Business processing method and device based on financial website robot, and electronic equipment

Abstract

The invention discloses a business processing method and device based on a financial website robot and electronic equipment, and relates to the field of financial science and technology, and the method comprises the steps: receiving multi-mode identification information and an operation log sent by a financial application program in a business interaction process between the financial website robot and a user, mapping the multi-modal identification information and the operation log to a predetermined interaction vector space, outputting a continuous interaction vector, performing dynamic weight adjustment on historical interaction data in the continuous interaction vector through an interaction attenuation coefficient, establishing an incidence matrix of each specified service type and an interaction state based on the continuous interaction vector after the weight adjustment, and sending the incidence matrix to a server; and outputting a response strategy by adopting the incidence matrix, and controlling the financial network robot to output a service response action. According to the invention, the technical problem that the customer satisfaction is reduced due to the fact that the information identification channel is single and repeated questioning or invalid operation is easily caused in the business interaction process of the financial network robot in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of financial technology or other related fields, and more specifically, to a business processing method and apparatus, and electronic equipment based on a financial branch robot. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, especially breakthroughs in natural language processing, computer vision, and deep learning, the financial services industry has increasingly applied robots and automated systems to improve customer experience, reduce costs, and increase efficiency. Financial branch robots, as part of the service window, interact with customers and provide personalized services through identity verification and business processing interactions. However, during business interactions, the identification of customer interaction information by these financial network robots is mostly limited to a single channel, such as face-to-face interaction, without considering changes in the user's state during the interaction. Furthermore, they cannot adjust service strategies based on unfinished transactions or operations by customers in online channels, easily leading to repeated questions or ineffective operations, thus reducing customer satisfaction.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a business processing method, apparatus, and electronic device based on a financial branch robot, to at least solve the technical problem in the related art that financial branch robots have a single information identification channel during business interaction, which easily leads to repeated questions or invalid operations and reduces customer satisfaction.

[0005] To achieve the above objectives, according to one aspect of this application, a business processing method based on a financial branch robot is provided, comprising: during business interaction between the financial branch robot and a user, receiving multimodal recognition information from the financial branch robot after feature integration and feature fusion, and an operation log sent by a financial application installed on the user's terminal, wherein the operation log includes time-series operation information of the terminal page during the user's authorization to use the financial application; mapping the multimodal recognition information and the operation log to a predetermined interaction vector space, outputting a continuous interaction vector, and dynamically adjusting the weights of historical interaction data in the continuous interaction vector using a pre-acquired interaction attenuation coefficient; establishing an association matrix between each specified business type and interaction state based on the weighted continuous interaction vector, and outputting a response strategy using the association matrix, wherein the response strategy is used to control the business response actions output by the financial branch robot.

[0006] Optionally, before receiving the multimodal recognition information from the financial branch robot after feature integration and feature fusion, and the operation log sent by the financial application installed on the user terminal, the method further includes: collecting user interaction data from the financial branch robot during the business interaction process to obtain initial multimodal information; extracting features from the user interaction data using a multimodal data processing engine to obtain spatiotemporal synchronization engine feature data; and using a Transformer architecture module based on a self-attention mechanism to perform feature fusion on the spatiotemporal synchronization engine feature data to obtain the multimodal recognition information.

[0007] Optionally, before receiving the multimodal recognition information of the financial branch robot after feature integration and feature fusion, and the operation log sent by the financial application installed in the user terminal, the method further includes: after obtaining user authorization, the financial application installed in the user terminal receives the user's operation behavior data, operation delay data, page interruption frequency data, business operation duration data, and timestamps during the historical process of using the financial application to conduct business interactions, and obtains the operation log.

[0008] Optionally, the step of mapping the multimodal recognition information and the operation log to a predetermined interaction vector space to output a continuous interaction vector includes: selecting the predetermined interaction vector space as the predetermined interaction vector space, wherein the predetermined interaction vector space is configured with interaction vectors of three continuous dimensions; based on a pre-established operation behavior-interaction feature mapping table, mapping the operation behavior data and business operation duration data in the operation log to the predetermined interaction vector space to obtain a first type of interaction vector and discrete interaction labels; based on a pre-established activity state-interaction feature mapping table, mapping the user interaction data in the multimodal recognition information to the predetermined interaction vector space to obtain a second type of interaction vector and discrete interaction labels; and combining the first type of interaction vector, the second type of interaction vector, and the discrete interaction labels to output a continuous interaction vector.

[0009] Optionally, the step of dynamically adjusting the weights of historical interaction data in the continuous interaction vector using a pre-acquired interaction attenuation coefficient includes: obtaining the business risk level of the target financial business in which the financial branch robot interacts with the user, wherein the business risk level is determined based on the business type and importance level of the target financial business; adjusting the interaction attenuation coefficient according to the business risk level; and inputting the interaction attenuation coefficient and the interval between historically identified interaction states into the interaction attenuation model, so that the interaction attenuation model can dynamically adjust the weights of historical interaction data in the continuous interaction vector.

[0010] Optionally, the step of using the association matrix to output a response strategy includes: determining the specified business type involved in the interaction between the financial branch robot and the user at the current time; obtaining the business interaction identification result and dwell time of the interaction between the financial branch robot and the user at the current time; calling the association matrix, indexing the robot response strategy corresponding to the specified business type, the business interaction identification result and the dwell time, and outputting the response strategy.

[0011] Optionally, the business processing method based on financial branch robots further includes: using a federated learning algorithm to obtain abnormal interaction identification samples and interaction identification model parameters of each financial branch, wherein the abnormal interaction identification samples include at least: abnormal interaction images, abnormal interaction features, and robot response strategies; inputting the abnormal interaction identification samples into a preset generative adversarial network, whereby the preset generative adversarial network adjusts the established correlation matrix and adjusts the output response strategy.

[0012] Optionally, the business processing method based on the financial branch robot further includes: acquiring the dynamic interaction status of the user identified during the business interaction between the financial branch robot and the user; inputting the dynamic interaction status into the business association module, which then executes a false trigger circuit breaker mechanism for judgment, wherein, under the false trigger circuit breaker mechanism, it is detected and determined whether a risk control action has been triggered and the number of triggers; if the number of triggers of risk control actions reaches a preset threshold within a specified time period, the service process of the financial branch server is switched to a manual service process.

[0013] According to another aspect of the present invention, a business processing apparatus based on a financial branch robot is also provided, comprising: an operation data receiving unit, configured to receive multimodal recognition information of the financial branch robot after feature integration and feature fusion, and operation logs sent by a financial application installed on a user terminal during business interaction between the financial branch robot and a user, wherein the operation logs include time-series operation information of the terminal page during the user's authorization to use the financial application; an interaction mapping unit, configured to map the multimodal recognition information and the operation logs to a predetermined interaction vector space, output a continuous interaction vector, and dynamically adjust the weights of historical interaction data in the continuous interaction vectors using a pre-acquired interaction attenuation coefficient; and a business response unit, configured to establish an association matrix between each specified business type and interaction state based on the weighted continuous interaction vectors, and output a response strategy using the association matrix, wherein the response strategy is used to control the business response actions output by the financial branch robot.

[0014] Optionally, the business processing device based on the financial branch robot further includes: a multimodal information acquisition unit, used to collect user interaction data during the business interaction process between the financial branch robot and the user before receiving the multimodal recognition information after feature integration and feature fusion from the financial branch robot and the operation log sent by the financial application installed in the user terminal, to obtain initial multimodal information; a feature extraction unit, used to extract features from the user interaction data through a multimodal data processing engine to obtain spatiotemporal synchronization engine feature data; and a feature fusion unit, used to perform feature fusion on the spatiotemporal synchronization engine feature data using a Transformer architecture module based on a self-attention mechanism to obtain the multimodal recognition information.

[0015] Optionally, the business processing device based on the financial branch robot further includes: an operation data acquisition unit, used to receive, before receiving the multimodal recognition information of the financial branch robot after feature integration and feature fusion and the operation log sent by the financial application installed in the user terminal, after the financial application installed in the user terminal obtains user authorization, the user's operation behavior data, operation delay data, page interruption frequency data, business operation duration data and timestamp during the historical process of using the financial application to conduct business interactions, and obtain the operation log.

[0016] Optionally, the interaction mapping unit includes: a selection module, configured to select the predetermined interaction vector space as the predetermined interaction vector space, wherein the predetermined interaction vector space is configured with interaction vectors of three consecutive dimensions; a first mapping module, configured to map the operation behavior data and business operation duration data in the operation log to the predetermined interaction vector space based on a pre-established operation behavior-interaction feature mapping table, to obtain a first type of interaction vector and discrete interaction labels; a second mapping module, configured to map the user interaction data in the multimodal recognition information to the predetermined interaction vector space based on a pre-established activity state-interaction feature mapping table, to obtain a second type of interaction vector and discrete interaction labels; and an interaction vector output module, configured to integrate the first type of interaction vector, the second type of interaction vector, and the discrete interaction labels to output a continuous interaction vector.

[0017] Optionally, the interaction mapping unit includes: a business risk level acquisition module, used to acquire the business risk level of the target financial business in which the financial branch robot interacts with the user, wherein the business risk level is determined based on the business type and business importance level of the target financial business; a coefficient adjustment module, used to adjust the interaction attenuation coefficient according to the business risk level; and an interaction data adjustment module, used to input the interaction attenuation coefficient and the interval duration between historically identified interaction states into the interaction attenuation model, and the interaction attenuation model dynamically adjusts the weights of historical interaction data in the continuous interaction vector.

[0018] Optionally, the business response unit includes: a business type determination module, used to determine the specified business type involved in the interaction between the financial branch robot and the user at the current time; an interaction identification result acquisition module, used to acquire the business interaction identification result and dwell time of the interaction between the financial branch robot and the user at the current time; and a strategy indexing module, used to call the association matrix, index the robot response strategy corresponding to the specified business type, the business interaction identification result, and the dwell time, and output the response strategy.

[0019] Optionally, the business processing device based on the financial branch robot further includes: an abnormal interaction identification sample acquisition unit, used to acquire abnormal interaction identification samples and interaction identification model parameters of each financial branch using a federated learning algorithm, wherein the abnormal interaction identification sample includes at least: abnormal interaction image, abnormal interaction features, and robot response strategy; and a matrix adjustment unit, used to input the abnormal interaction identification sample into a preset generative adversarial network, and the preset generative adversarial network adjusts the established correlation matrix and adjusts the output response strategy.

[0020] Optionally, the business processing device based on the financial branch robot further includes: an interaction status acquisition unit, used to acquire the dynamic interaction status of the user identified during the business interaction between the financial branch robot and the user; a false trigger circuit breaker mechanism judgment unit, used to input the dynamic interaction status into the business association module, and the business association module executes the false trigger circuit breaker mechanism judgment, wherein, under the false trigger circuit breaker mechanism, it is detected and determined whether a risk control action is triggered and the number of triggers; and a service process switching unit, used to switch the financial branch server service process to a manual service process when the number of times the risk control action is triggered within a specified time period reaches a preset threshold.

[0021] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the business processing method based on the financial branch robot described above.

[0022] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the business processing method based on the financial branch robot described above.

[0023] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the business processing method based on a financial branch robot as described above.

[0024] In this disclosure, during the business interaction between a financial branch robot and a user, the system receives multimodal recognition information from the financial branch robot after feature integration and feature fusion, as well as operation logs sent by the financial application installed on the user's terminal. The operation logs include time-series operation information of the terminal page during the user's authorization to use the financial application. The multimodal recognition information and operation logs are mapped to a predetermined interaction vector space to output a continuous interaction vector. The historical interaction data in the continuous interaction vector is dynamically weighted using a pre-acquired interaction attenuation coefficient. Based on the weighted continuous interaction vector, an association matrix between each specified business type and interaction state is established. The association matrix is ​​used to output a response strategy, wherein the response strategy is used to control the business response actions output by the financial branch robot.

[0025] Based on the aforementioned publicly available information, by integrating multimodal data across channels and combining user operation logs recorded by financial applications on user terminals, a more comprehensive user interaction background can be provided for financial branch robots through multiple information channels. Furthermore, based on dynamic interaction status and business association matrix, financial branch robots can adjust their interaction strategies in real time, provide more personalized services, and improve customer satisfaction. This solves the technical problem in related technologies where financial branch robots, during business interactions, often rely on a single information channel, leading to repetitive questions or invalid operations and reduced customer satisfaction. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0027] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a business processing method based on a financial branch robot is shown.

[0028] Figure 2 This is a flowchart of an optional business processing method based on a financial branch robot according to an embodiment of the present invention;

[0029] Figure 3 This is a flowchart of an optional cross-channel interaction status recognition and business processing method based on bank branch robot interaction according to an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of an optional business processing device based on a financial branch robot according to an embodiment of the present invention;

[0031] Figure 5 This is a structural block diagram of an electronic device that performs a business processing method based on a financial branch robot, according to an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0035] The Transformer Model (TM) is a neural network architecture suitable for processing sequential data such as text and speech. It utilizes a self-attention mechanism to process the input sequence, enabling the model to focus on different parts of the input sequence in parallel, effectively capturing long-range dependencies. In this invention, the Transformer model is used for feature fusion of multimodal signals, improving the accuracy and efficiency of interactive state recognition.

[0036] In this invention, a cross-channel interactive state fusion module is defined in the interactive vector space. This module combines real-time multimodal recognition information with historical operation logs on the user terminal financial application and generates the user's dynamic interactive state through interactive space mapping.

[0037] The circuit breaker mechanism is a safety measure to prevent financial branch robots from making inappropriate responses due to misunderstandings of customer interactions. If multiple misjudgments occur consecutively within a certain period, the system will automatically switch to human assistance to reduce the risk of service interruption and customer dissatisfaction.

[0038] Federated Learning (FL) is a distributed machine learning framework that allows parties to collaboratively train a model without sharing the original data. In this invention, federated learning enables institutions in different regions to train interaction recognition models locally, and then upload the model parameters (rather than the data itself) to a unified system. These parameters are then aggregated to update the global model, thus protecting data privacy while improving the model's generalization ability.

[0039] It should be noted that the business processing method and apparatus based on financial branch robots in this disclosure can be used in the financial technology field to realize cross-channel interactive status recognition and business interaction based on financial branch robots, and can also be used in any field other than the financial technology field to realize cross-channel interactive status recognition and business interaction based on financial branch robots. This disclosure does not limit the application field of the business processing method and apparatus based on financial branch robots.

[0040] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected in this public disclosure are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0041] It should be noted that in this disclosure, customer information is collected and analyzed, and users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0042] The following embodiments of the present invention can be applied to various systems / applications / devices for business processing based on financial branch robots. The present invention can be applied to scenarios integrating financial institution branches and online banking services. For example, in a bank branch robot interaction system, the present invention can achieve cross-channel identification and migration of user interaction states between the robot and online channels such as mobile banking applications (APPs), improving customer service quality, optimizing business processing flows, and enhancing the robot's humanized interactive experience. Taking bank branch service as an example, when a customer enters a bank branch and interacts with the service robot, the present invention can integrate the customer's previous operational behavior and interaction state response (or user feedback tendency) on the mobile financial application, providing the robot with a more comprehensive user interaction background, thereby making a more appropriate service response. Combined with the user interaction background, the present invention can quickly identify abnormal interaction state changes when customers are performing high-risk operations (such as large-scale fund transfers), thereby triggering additional identity verification or risk assessment processes in a timely manner, enhancing financial security.

[0043] By integrating multimodal data across channels, this invention can capture user interaction states in a wider range of scenarios, providing more comprehensive and accurate interaction recognition capabilities than single-channel methods. Based on dynamic interaction states and business association rules, this invention enables the robot to adjust its interaction strategies in real time, providing more personalized services and improving customer satisfaction.

[0044] This invention introduces a business association matrix and a circuit breaker mechanism for accidental triggering, enabling dynamic assessment of business risks based on customer interactions and automatic adjustment of service responses or switching to human assistance, effectively avoiding potential risks and operational errors. Through a federated learning framework, this invention achieves efficient utilization of multi-channel data while ensuring the security and privacy of user data, meeting the high data protection standards of the financial industry.

[0045] The present invention will now be described in detail with reference to various embodiments.

[0046] Example 1

[0047] According to an embodiment of the present invention, an embodiment of a business processing method based on a financial branch robot is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0048] The business processing method based on financial branch robots provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a business processing method based on a financial branch robot is shown. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1 (Illustrated as 102a, 102b, ..., 102n) Processor 102 (processor 102 may include, but is not limited to, a microprocessor MCU (Microcontroller Unit) or a programmable gate array (FPGA), etc.), memory 104 for storing data, and transmission device 106 for communication functions. In addition, it may include: a display, input / output interface (I / O interface), Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), network interface, power supply, and / or camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0049] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the business processing method based on financial branch robots in this embodiment of the application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned business processing method based on financial branch robots. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0052] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0053] Under the aforementioned operating environment, this application provides the following: Figure 2 The diagram illustrates a business processing method based on a financial branch robot. Figure 2 This is a flowchart of an optional business processing method based on a financial branch robot according to an embodiment of the present invention, such as... Figure 2As shown, the method includes the following steps S201 to S203, which will be explained in detail below.

[0054] Optionally, before receiving the multimodal recognition information from the financial branch robot after feature integration and feature fusion, and the operation logs sent by the financial application installed on the user terminal, the method further includes: collecting user interaction data from the financial branch robot during the business interaction process to obtain initial multimodal information; extracting features from the user interaction data through a multimodal data processing engine to obtain spatiotemporal synchronization engine feature data; and using a Transformer architecture module based on a self-attention mechanism to perform feature fusion on the spatiotemporal synchronization engine feature data to obtain multimodal recognition information.

[0055] In this embodiment, the financial branch robot (such as a fixed robot or a mobile chatbot pre-equipped in a bank branch) is equipped with various sensing devices, enabling it to receive interactive information (such as voice data) transmitted from the user terminal and obtain raw multimodal information. To ensure the accuracy and usability of the information, this embodiment employs a multimodal data processing engine to extract features and standardize the initially collected data, thereby transforming the complex and raw data information into a structured and easily analyzable form. Furthermore, this embodiment uses a multimodal data processing engine to convert the user's voice waveform and body movements into a series of data features.

[0056] It should be noted that the multimodal information collected in this embodiment is processed through a spatiotemporal synchronization engine to ensure that interaction features correspond to timestamps, thereby accurately aligning time-series data of different modalities and maintaining the temporal continuity and accuracy of interaction states. Furthermore, a Transformer architecture module is employed, based on a self-attention mechanism, to deeply integrate the feature data from the spatiotemporal synchronization engine. This self-attention mechanism allows the model to consider all input features simultaneously, rather than processing them sequentially.

[0057] Optionally, before receiving the multimodal recognition information of the financial branch robot after feature integration and feature fusion, and the operation log sent by the financial application installed in the user terminal, the method further includes: after the financial application installed in the user terminal obtains user authorization, receiving the user's operation behavior data, operation delay data, page interruption frequency data, business operation duration data and timestamps during the historical process of using the financial application for business interaction, and obtaining the operation log.

[0058] In this embodiment, with the explicit authorization of the user, a role can be established to collect the user's online behavior data through a financial application installed on the user's terminal (provided the user has authorized and the data collection interface is open). The financial application can record the user's operation history when using the financial application for business interactions in a historical process, including operation behavior, operation delay, page interruption frequency, business operation duration and corresponding operation timestamps, and generate operation logs.

[0059] The operation log records user behavior data within the app, such as checking balances, transferring funds, and purchasing financial products. Operation delay data includes the time difference between the user's response to various functions within the financial application, which may include the time difference between the user clicking the page and the actual time the page is displayed in the financial application. Business operation duration data records the time required for the user to complete a certain business, including the timestamp from the user entering a business page in the financial application to clicking the complete button. The timestamp refers to adding a time identifier to each of the aforementioned data items to ensure that the operation log can reflect the sequence and time of events. By collecting and analyzing this operation log data, this embodiment can construct the user's online behavior trajectory and combine it with multimodal information from offline robot interactions to realize a complete interactive state migration and recognition system that spans online and offline, creating conditions for providing more coherent and personalized financial services.

[0060] Step S201: During the business interaction between the financial branch robot and the user, the system receives multimodal recognition information after feature integration and feature fusion from the financial branch robot, as well as operation logs sent by the financial application installed on the user's terminal. The operation logs include the timing operation information of the terminal page during the user's authorization to use the financial application.

[0061] In this embodiment, when the financial branch robot interacts with the user, the content it receives includes not only multimodal recognition information from the robot's end but also detailed operation logs sent by the financial application on the user's terminal. For example, the multimodal recognition information includes the comprehensive interactive action analysis results of user voice data and body movements collected with authorization during the interaction with the user. Through a professional multimodal data processing engine, these data are feature extracted and integrated, and then feature fusion is performed using the self-attention mechanism of the Transformer architecture. The generated multimodal recognition information includes the user's real-time interactive state during face-to-face interaction. As for the operation logs, with the user's explicit authorization, the financial application in this embodiment records a series of time-series operation information of the user on the terminal page (including but not limited to the type of operation performed by the user, operation delay, page interruption frequency, and operation duration). By collecting this online behavioral data, the user's operation trajectory during the use of the financial application can be constructed.

[0062] It should be noted that the financial application receives user operation logs, and the result of feature fusion with the Transformer fusion module can be input into the cross-channel interaction status fusion module. The cross-channel interaction status module uses a preset spatial mapping strategy to align the multi-channel data and performs dynamic data processing with the interaction decay model.

[0063] Step S202: Map the multimodal recognition information and operation log to a predetermined interaction vector space, output a continuous interaction vector, and dynamically adjust the weights of the historical interaction data in the continuous interaction vector using a pre-acquired interaction decay coefficient.

[0064] Optionally, the step of mapping multimodal recognition information and operation logs to a predetermined interaction vector space to output continuous interaction vectors includes: selecting a predetermined interaction vector space, wherein the predetermined interaction vector space is configured with interaction vectors of three continuous dimensions; mapping operation behavior data and business operation duration data in the operation logs to the predetermined interaction vector space based on a pre-established operation behavior-interaction feature mapping table to obtain a first type of interaction vector and discrete interaction labels; mapping user interaction data in multimodal recognition information to the predetermined interaction vector space based on a pre-established activity state-interaction feature mapping table to obtain a second type of interaction vector and discrete interaction labels; and combining the first type of interaction vector, the second type of interaction vector, and the discrete interaction labels to output continuous interaction vectors.

[0065] After receiving multimodal recognition information and operation logs, this embodiment further maps these data to a predetermined interaction vector space (in this embodiment, the PAD interaction space can be selected, which is an expression model for user interaction recognition, mapping user interaction states to multiple continuous dimensions. By converting interaction data from different sources into vectors in the PAD space, comparison and fusion of multi-channel interaction states can be achieved, providing more detailed quantitative indicators for interaction state recognition), to output continuous interaction vectors, transforming discrete interaction tags and behavioral data into continuous interaction vectors, facilitating more detailed interaction state analysis and fusion.

[0066] It should be noted that the predetermined interaction vector space mentioned in this embodiment may include an interaction vector space configured with three consecutive dimensions: a first dimension (corresponding to the first type of interaction vector), a second dimension (corresponding to the second type of interaction vector), and a third dimension (corresponding to the third type of interaction vector).

[0067] It should also be noted that the first type of interaction vector in this embodiment originates from the operation log mapping. Through a pre-established operation behavior-interaction feature mapping table, the user's operation behavior on the financial application is transformed into interaction vectors in a predetermined interaction vector space. The second type of interaction vector originates from the multimodal recognition information mapping, through a pre-established activity state-interaction feature mapping table. Then, by combining the first type of interaction vector, the second type of interaction vector, and discrete interaction tags, this embodiment generates continuous interaction vectors using a fusion algorithm (such as weighted averaging). The fusion process considers the multi-dimensional and multi-source characteristics of the user interaction process, ensuring the comprehensiveness and accuracy of the output interaction vectors.

[0068] Optionally, the step of dynamically adjusting the weights of historical interaction data in the continuous interaction vector using a pre-acquired interaction attenuation coefficient includes: obtaining the business risk level of the target financial business in which the financial branch robot interacts with the user, wherein the business risk level is determined based on the business type and importance level of the target financial business; adjusting the interaction attenuation coefficient according to the business risk level; and inputting the interaction attenuation coefficient and the interval between historically identified interaction states into the interaction attenuation model, so that the interaction attenuation model can dynamically adjust the weights of historical interaction data in the continuous interaction vector.

[0069] In this embodiment, the interaction decay model is used to process historical interaction data in continuous interaction vectors, and dynamically adjusts the weight of historical interaction information according to the business risk level of the current financial business, so as to ensure that the robot can reasonably integrate the user's latest interaction status and past interaction status traces based on the real-time business environment and risk considerations during the interaction process, thereby making a more accurate and appropriate response.

[0070] First, this embodiment obtains the business risk level of the target financial business from a database or real-time business information system. The business risk level is determined based on the business type (e.g., small-amount transfers, wealth management product consultations, account loss reporting, etc.) and the importance of the business (scope of impact, scale of funds, etc.), representing the priority of the business in interaction management. Then, based on the obtained business risk level, this embodiment adjusts the interaction attenuation coefficient. The interaction attenuation coefficient (defined as λ0) is a parameter that controls the proportion of historical interaction data in the current interaction state (or user feedback tendency). The larger the value of λ, the lower the contribution of historical interaction data, and vice versa. This adjustment mechanism considers the possibility and speed of dynamic changes in user interaction under different business scenarios, ensuring that the integration of historical interaction data is more realistic. It should be noted that for high-risk operations, because the user interaction state may change rapidly due to the urgency and complexity of the business, the value of λ will be increased, meaning that the impact of historical interaction state is less significant, and the robot relies more on... The system responds to user interactions in real-time. For low-risk operations, user interactions are more stable, resulting in a smaller λ value. This allows the continuous interaction vector to retain more weight from historical interactions, helping the robot understand long-term trends and provide more consistent service. Finally, based on the adjusted interaction decay coefficient λ and the interval between historical interaction data and the currently identified interaction state, this embodiment inputs these parameters into the interaction decay model. The interaction decay model uses a specific mathematical algorithm (such as an exponential decay function) to dynamically adjust the weights of historical interaction data in the continuous interaction vector, ensuring that the continuous interaction vector accurately reflects the timeliness and importance of user interactions.

[0071] Step S203: Based on the weighted continuous interaction vector, establish an association matrix between each specified business type and interaction state, and use the association matrix to output a response strategy, wherein the response strategy is used to control the business response actions output by the financial branch robot.

[0072] In this embodiment, step S203 establishes a correlation matrix between specific business types and interaction states based on the weighted continuous interaction vector, and then uses this matrix to guide the robot to output appropriate business response actions, thereby achieving sensitivity and responsiveness to user feedback tendencies and providing personalized services and accurate positioning.

[0073] Optionally, the step of using the association matrix to output the response strategy includes: determining the specified business type involved in the interaction between the financial branch robot and the user at the current moment; obtaining the business interaction identification result and dwell time of the interaction between the financial branch robot and the user at the current moment; calling the association matrix, indexing the robot response strategy corresponding to the specified business type, business interaction identification result and dwell time, and outputting the response strategy.

[0074] First, this embodiment requires collecting a large amount of historical case data, covering changes in user interaction states under different business types. Based on this data analysis, a business association matrix is ​​established. This matrix records in detail the association rules between specific business types and user interaction states, as well as the optimal response strategy under this association. The association matrix can include more detailed identity security verification steps and reassuring language response strategies. Once the financial branch robot begins interacting with the user and determines the current business type, the robot can also simultaneously obtain the user's feedback tendency identification results and dwell time. Using this information as an index, the robot calls the pre-established association matrix to find the response strategy matching the current situation and outputs the corresponding business response action accordingly. The strategy output based on user interaction state (or user feedback tendency) and business type can significantly improve service effectiveness and customer satisfaction.

[0075] During the interaction with the user, the robot in this embodiment can quickly identify the specific business type that the user is trying to perform (the identification process may involve multiple information sources such as voice command parsing, screen display content analysis, and business options manually selected by the user), ensuring the robot's accurate judgment of the business type.

[0076] Based on the current business type, interaction status recognition results, and dwell time, the robot accesses the association matrix through an index to find the most suitable response strategy. It includes optimal response guidelines for various business-interaction status combinations. For example, if a user shows a high tendency to provide feedback during financial consultation but the dwell time exceeds a predetermined threshold, the robot can proactively provide more in-depth product details or suggest scheduling a face-to-face meeting with a financial advisor to meet the user's interests and needs.

[0077] Through the above steps, during the business interaction between the financial branch robot and the user, the system can receive multimodal recognition information (after feature integration and fusion) from the financial branch robot and operation logs sent by the financial application installed on the user's terminal. The operation logs include the time-series operation information of the terminal page during the user's authorization to use the financial application. The multimodal recognition information and operation logs are mapped to a predetermined interaction vector space to output continuous interaction vectors. The historical interaction data in the continuous interaction vectors are dynamically weighted using a pre-acquired interaction attenuation coefficient. Based on the weighted continuous interaction vectors, an association matrix between each specified business type and interaction state is established. The association matrix is ​​used to output a response strategy, whereby the response strategy is used to control the business response actions output by the financial branch robot. In this embodiment, multimodal data can be integrated across channels, and user operation logs recorded by financial applications in user terminals can be combined to provide a more comprehensive background of user interaction status for financial branch robots through multiple information channels. Based on dynamic interaction status and business association matrix, financial branch robots can adjust their interaction strategies in real time, provide more personalized services, and improve customer satisfaction. This solves the technical problem in related technologies where financial branch robots have a single information channel for identification during business interactions, which can easily lead to repeated questions or invalid operations and reduce customer satisfaction.

[0078] In this embodiment, the federated learning algorithm is applied to obtain abnormal interaction identification samples and corresponding interaction identification model parameters of each financial branch, which not only enhances the robustness and generalization ability of the model, but also ensures the security and privacy protection of user data. Optionally, the business processing method based on the financial branch robot further includes: using the federated learning algorithm to obtain abnormal interaction identification samples and interaction identification model parameters of each financial branch, wherein the abnormal interaction identification samples include at least: abnormal interaction images, abnormal interaction features, and robot response strategies; inputting the abnormal interaction identification samples into a preset generative adversarial network, which adjusts the established correlation matrix and the output response strategy.

[0079] In acquiring abnormal interaction identification samples and model parameters from various financial institutions, this embodiment employs a federated learning algorithm. Federated learning is a distributed machine learning framework that allows multiple institutions to collaboratively train a model without sharing raw data. Each institution (such as bank branches) trains its model locally and then only uploads the model parameters to a central server for aggregation. This avoids directly exposing sensitive user data, thus protecting user privacy while ensuring the effectiveness of model training. Through the abnormal interaction identification samples and interaction identification model parameters collected via federated learning, this embodiment can periodically update and optimize the correlation matrix, ensuring the optimization of the robot's response strategy. This improves the accuracy of the robot's interaction state identification in specific business scenarios and makes its response strategy more personalized and human-like, better adapting to the specific needs of different user groups.

[0080] Among them, abnormal interaction identification samples can include users' abnormal feedback tendencies in specific business scenarios and corresponding robot response strategies. The samples may include, but are not limited to: abnormal interaction images, abnormal interaction features, and robot response strategies designed for these abnormal interaction states.

[0081] To further optimize the correlation matrix and response strategy, this embodiment employs a pre-defined Generative Adversarial Network (GAN). A GAN is a deep learning model comprising two neural networks: a generator and a discriminator. The generator aims to generate fake samples, while the discriminator aims to distinguish real samples from the fake samples generated by the generator. Through this competitive mechanism, the GAN can learn the latent distribution of the data and generate samples similar to real data. In this embodiment, abnormal interaction identification samples collected from various financial institutions are input into the pre-defined GAN. The generator attempts to synthesize data similar to real abnormal interaction identification samples, while the discriminator evaluates the realism of these synthesized samples. This helps enhance the model's ability to identify abnormal interaction states, especially when real data is scarce or difficult to obtain. The GAN can compensate for this deficiency by synthesizing data, enriching the diversity of training samples.

[0082] By generating abnormal interaction identification samples using a generative adversarial network, this embodiment can adjust the established association matrix. Specifically, the adjustment can include fine-tuning the robot response strategy under specific interaction state combinations to improve its applicability and effectiveness in abnormal interaction scenarios.

[0083] As the model is continuously optimized, the correlation matrix and response strategy will also be dynamically adjusted to adapt to the ever-changing business environment and user feedback tendencies. The robot can temporarily adjust its response strategy and add state steps with increased feedback tendency, such as providing a clearer explanation of loan interest rates or extending the decision support time, to help users make more comfortable choices.

[0084] Optionally, the business processing method based on the financial branch robot further includes: outputting the dynamic interaction status of the user identified during the business interaction between the financial branch robot and the user through the cross-channel interaction status module; inputting the dynamic interaction status into the business association module, which then executes a false trigger circuit breaker mechanism for judgment, wherein, under the false trigger circuit breaker mechanism, it is detected and determined whether a risk control action has been triggered and the number of triggers; if the number of triggers of risk control actions reaches a preset threshold within a specified time period, the service process of the financial branch server is switched to a manual service process.

[0085] In the business process of this embodiment, the cross-channel interaction status module aggregates information based on multimodal recognition (such as voice data, body movements, etc.) and financial application operation logs from user terminals. Through predetermined interaction vector space mapping and interaction decay model processing, it generates continuous interaction vectors to accurately reflect fluctuations in the user's interaction status. The cross-channel interaction status module continuously analyzes changes in the user's interaction status during the interaction process and outputs dynamic interaction status data. The data not only includes immediate interaction status feedback but also considers the decay effect of the user's historical interaction status, forming a comprehensive, time-evolving description of the user's interaction status. The dynamic interaction status data is then passed to the business association module. The module analyzes the correlation between the interaction status and the current business context and executes a circuit breaker mechanism to prevent the robot from over-interfering or taking inappropriate business processing actions due to misunderstanding the user's interaction status, thereby improving the stability and security of the service. The business association module detects whether risk control actions have been triggered and the number of times such actions are triggered within a unit of time. Risk control actions refer to security measures or additional verification steps initiated based on the user's feedback tendency status.

[0086] Under the circuit breaker mechanism for false triggers, if the number of times a risk control action is triggered within a specified time period reaches a preset threshold, it means that the robot may be stuck in a loop of repeatedly performing the same risk control actions. This not only increases the user's processing time but also leads to a decline in user experience. At this time, the system can automatically determine that a false trigger has occurred and trigger the service process switch of the financial branch server, that is, switch from the robot's automated service process to the human service process, in order to resolve potential service deadlocks or errors in interaction status recognition.

[0087] When the service process switches to human assistance, branch staff will take over the interaction with customers. By observing user behavior, listening to user statements, and using professional judgment, staff can more accurately understand user feedback and business needs, providing personalized solutions and care to ensure customers receive an efficient, safe, and satisfactory service experience.

[0088] The following describes in detail another optional implementation method.

[0089] The present invention overcomes the shortcomings of existing robot interaction status recognition and the lack of interoperability and synchronization between the two channels of bank branch service APP, and provides a fast and comprehensive solution that can transfer user interaction status data across channels, enabling customers to have a more personalized and satisfactory service experience through financial branch robots.

[0090] This invention, based on the existing Transformer-based multimodal interaction status recognition, adds a cross-channel interaction status fusion module and a business association module to adapt to the service model of bank branches.

[0091] Figure 3 This is a flowchart of an optional cross-channel interaction status recognition and business processing method based on bank branch robot interaction according to an embodiment of the present invention, such as... Figure 3 As shown, it includes:

[0092] A [Multimodal data: speech data / action] --> B [Feature extraction];

[0093] B --> C [Transformer fusion];

[0094] C-->D [Cross-channel interaction status fusion module];

[0095] E[Application Operation Log] --> D;

[0096] D-->F [Dynamic Interactive State];

[0097] F-->G [Business Interaction Related Module];

[0098] G-->H [Incorrectly triggered circuit breaker mechanism];

[0099] H -->|Exception|I [Switch to human service];

[0100] H-->|Normal|J[Robot Interaction].

[0101] like Figure 3 As shown, the multimodal data (including voice, body movements, etc.) received by the robot is input into the feature extraction module. After feature extraction, it is input into the Transformer fusion module for feature fusion. The financial application receives the user's operation logs, which, together with the result of feature fusion by the Transformer fusion module, are input into the cross-channel interaction state fusion module. The cross-channel interaction state module includes multi-channel data alignment using PAD space mapping and dynamic data processing (i.e., data processing for interaction state transitions) using an interaction decay model.

[0102] The cross-channel interaction status module outputs the user's dynamic feedback tendency status, which is then input into the business association module. Based on the module's output, a circuit breaker mechanism is determined for false triggering. For example, if the same risk control action is triggered three times consecutively within 10 minutes, human service is automatically switched, and the financial branch robot tells the user, "I'm a little tired, *** will serve you next." If the circuit breaker mechanism determines that there is no abnormality, the robot interacts based on the results from the business association module.

[0103] This implementation method establishes a circuit breaker mechanism to prevent excessive robot intervention and provide customers with more satisfactory service.

[0104] In the cross-channel interaction status integration module, the process steps include:

[0105] The first step is multi-channel data collection and alignment: A spatiotemporal synchronization engine records the interaction status data of each terminal, and the mobile banking app operation logs (click delay / interruption frequency) and Transformer-based multimodal interaction status recognition results are uniformly mapped to a predetermined interaction vector space, outputting a continuous interaction vector. By extending discrete interaction tags into a continuous space, we can model complex emotions with finer granularity.

[0106] In the process of aligning multi-channel data collection, the operation behavior-interaction feature mapping table provided in Table 1 below can be used to uniformly map the interaction status data of each terminal to a predetermined interaction vector space and output a continuous interaction vector.

[0107] Table 1. Mapping Table of Operational Behaviors and Interaction Features of Financial Applications

[0108]

[0109] The second step is dynamic data processing for interaction state transitions: An interaction decay coefficient λ is set so that the weights of historical interaction states decrease exponentially over time. The decay function is designed as: γ(t) = e^(-λ / t) -λ·Δt , where λ is the interaction decay coefficient and Δt is the interval time.

[0110] Construct an interaction decay model to update and correct the interaction state: A real-time parameter tuning mechanism is added. When a high-risk operation is detected (such as a large transfer), the system adjusts the λ value to increase it. When a low-risk or risk-free operation is detected (such as a user chatting with a robot), the system adjusts the λ value to decrease it.

[0111] In the business association module, the data training steps include:

[0112] First, it is necessary to construct the business type-interaction status association matrix as shown in Table 2 below, and then connect different robot response actions.

[0113] Table 2. Business Type - Interaction Status Correlation Matrix

[0114]

[0115] Next, in this embodiment of the invention, a pre-set generative adversarial network (such as StyleGAN network) + federated learning strategy can be used to perform cross-back-to-back trial transfer learning. The pre-set generative adversarial network synthesizes abnormal interaction identification samples that are difficult for branch offices to obtain. The scarce interaction state samples are added to the training for fine-tuning. Each branch office trains the model locally and uploads the model parameters to the unified headquarters for aggregation, thereby solving the problem of scarce data in bank branch scenarios.

[0116] The above implementation method can establish a multi-channel interaction status recognition system that combines mobile banking applications and robot perception, which is more comprehensive than the current single-channel interaction status recognition (which only processes real-time data from the robot side and ignores the interaction status trajectory of the application channel).

[0117] In addition, this invention takes into account the dynamic changes in business scenarios. By establishing a dynamic attenuation model driven by business risks, it reduces response rigidity. For scenarios with high business risk levels, it reduces the retention of historical interaction states; for scenarios with low business risk levels, it increases the retention of historical interaction states.

[0118] The following is a detailed description with reference to another embodiment.

[0119] Example 2

[0120] The business processing device based on a financial branch robot provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above. The specific implementation method and beneficial effects can be referred to the aforementioned method embodiment, and will not be repeated here.

[0121] Figure 4 This is a schematic diagram of an optional business processing device based on a financial branch robot according to an embodiment of the present invention, such as... Figure 4 As shown, the business processing device based on the financial branch robot may include: an operation data receiving unit 41, an interactive mapping unit 42, and a business response unit 43.

[0122] The operation data receiving unit 41 is used to receive multimodal recognition information after feature integration and feature fusion of the financial branch robot and operation logs sent by the financial application installed in the user terminal during the business interaction between the financial branch robot and the user. The operation logs include the time sequence operation information of the terminal page during the user's authorization to use the financial application.

[0123] The interactive mapping unit 42 is used to map multimodal recognition information and operation logs to a predetermined interactive vector space, output continuous interactive vectors, and dynamically adjust the weights of historical interactive data in the continuous interactive vectors using a pre-acquired interactive decay coefficient.

[0124] The business response unit 43 is used to establish an association matrix between each specified business type and the interaction state based on the weighted continuous interaction vector, and to output a response strategy using the association matrix. The response strategy is used to control the business response actions output by the financial branch robot.

[0125] The aforementioned business processing device based on a financial branch robot can receive, through the operation data receiving unit 41, multimodal recognition information after feature integration and feature fusion from the financial branch robot and operation logs sent by the financial application installed on the user terminal during business interaction between the financial branch robot and the user. The operation logs include the time sequence operation information of the terminal page during the user's authorization to use the financial application. The interaction mapping unit 42 maps the multimodal recognition information and operation logs to a predetermined interaction vector space, outputs a continuous interaction vector, and dynamically adjusts the weights of historical interaction data in the continuous interaction vector using a pre-acquired interaction attenuation coefficient. The business response unit 43 establishes an association matrix between each specified business type and interaction state based on the weighted continuous interaction vector, and outputs a response strategy using the association matrix. The response strategy is used to control the business response actions output by the financial branch robot. In this embodiment, multimodal data can be integrated across channels, and user operation logs recorded by financial applications on user terminals can be combined to provide financial branch robots with a more comprehensive user emotional background through multiple information channels. Based on dynamic interaction status and business emotional association rules, the financial branch robot can adjust its interaction strategy in real time, provide more personalized services, and improve customer satisfaction. This solves the technical problem in related technologies where financial branch robots have a single information channel for identification during business interactions, which can easily lead to repeated questions or invalid operations and reduce customer satisfaction.

[0126] Optionally, the business processing device based on the financial branch robot further includes: a multimodal information acquisition unit, used to collect user interaction data during the business interaction process between the financial branch robot and the user before receiving the multimodal recognition information after feature integration and feature fusion from the financial branch robot and the operation log sent by the financial application installed in the user terminal, to obtain initial multimodal information; a feature extraction unit, used to extract features from the user interaction data through the multimodal data processing engine to obtain spatiotemporal synchronization engine feature data; and a feature fusion unit, used to perform feature fusion on the spatiotemporal synchronization engine feature data based on the self-attention mechanism using the Transformer architecture module to obtain multimodal recognition information.

[0127] Optionally, the business processing device based on the financial branch robot further includes: an operation data acquisition unit, which, before receiving the multimodal recognition information of the financial branch robot after feature integration and feature fusion, and the operation log sent by the financial application installed in the user terminal, receives, after the financial application installed in the user terminal obtains user authorization, the user's operation behavior data, operation delay data, page interruption frequency data, business operation duration data, and timestamp during the historical process of using the financial application for business interaction, and obtains the operation log.

[0128] Optionally, the interaction mapping unit includes: a selection module, used to select a predetermined interaction vector space as the predetermined interaction vector space, wherein the predetermined interaction vector space is configured with interaction vectors of three continuous dimensions; a first mapping module, used to map operation behavior data and business operation duration data in the operation log to the predetermined interaction vector space based on a pre-established operation behavior-interaction feature mapping table, to obtain a first type of interaction vector and discrete interaction labels; a second mapping module, used to map user interaction data in multimodal recognition information to the predetermined interaction vector space based on a pre-established activity state-interaction feature mapping table, to obtain a second type of interaction vector and discrete interaction labels; and an interaction vector output module, used to integrate the first type of interaction vector, the second type of interaction vector, and the discrete interaction labels to output a continuous interaction vector.

[0129] Optionally, the interactive mapping unit includes: a business risk level acquisition module, used to acquire the business risk level of the target financial business that the financial branch robot interacts with the user, wherein the business risk level is determined based on the business type and importance level of the target financial business; a coefficient adjustment module, used to adjust the interaction attenuation coefficient according to the business risk level; and an interaction data adjustment module, used to input the interaction attenuation coefficient and the interval between historically identified interaction states into the interaction attenuation model, and the interaction attenuation model dynamically adjusts the weights of historical interaction data in the continuous interaction vector.

[0130] Optionally, the business response unit includes: a business type determination module, used to determine the specified business type involved in the interaction between the financial branch robot and the user at the current moment; an interaction recognition result acquisition module, used to acquire the business interaction recognition result and dwell time of the interaction between the financial branch robot and the user at the current moment; and a strategy indexing module, used to call the association matrix, index the robot response strategy corresponding to the specified business type, business interaction recognition result and dwell time, and output the response strategy.

[0131] Optionally, the business processing device based on the financial branch robot further includes: an abnormal interaction identification sample acquisition unit, used to acquire abnormal interaction identification samples and interaction identification model parameters of each financial branch using a federated learning algorithm, wherein the abnormal interaction identification sample includes at least: abnormal interaction image, abnormal interaction features and robot response strategy; and a matrix adjustment unit, used to input the abnormal interaction identification sample into a preset generative adversarial network, and the preset generative adversarial network adjusts the established correlation matrix and adjusts the output response strategy.

[0132] Optionally, the business processing device based on the financial branch robot further includes: an interaction status acquisition unit, used to acquire the dynamic interaction status of the user identified during the business interaction between the financial branch robot and the user; a false trigger circuit breaker mechanism judgment unit, used to input the dynamic interaction status into the business association module, and the business association module executes the false trigger circuit breaker mechanism judgment, wherein, under the false trigger circuit breaker mechanism, it is detected and determined whether a risk control action is triggered and the number of triggers; and a service process switching unit, used to switch the financial branch server service process to a manual service process when the number of times the risk control action is triggered within a specified time period reaches a preset threshold.

[0133] The aforementioned business processing device based on financial branch robots may also include a processor and a memory. The aforementioned operation data receiving unit 41, interactive mapping unit 42, business response unit 43, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0134] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters enables cross-channel interactive status recognition and business services based on financial branch robots.

[0135] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0136] Example 3

[0137] Embodiments of this application may provide an electronic device. Figure 5 This is a structural block diagram of an electronic device that executes a business processing method based on a financial branch robot, according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 Only one of the components is shown: processor 502, memory 504, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module, and display.

[0138] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the business processing method and apparatus based on financial branch robots in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned business processing method based on financial branch robots. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0139] The processor can access information and applications stored in the memory via a transmission device to execute the following steps: During business interaction between the financial branch robot and the user, the processor receives multimodal recognition information from the financial branch robot after feature integration and feature fusion, as well as operation logs sent by the financial application installed on the user's terminal. The operation logs include time-series operation information of the terminal page during the user's authorization to use the financial application. The processor maps the multimodal recognition information and operation logs to a predetermined interaction vector space, outputs continuous interaction vectors, and dynamically adjusts the weights of historical interaction data in the continuous interaction vectors using a pre-acquired interaction attenuation coefficient. Based on the weighted continuous interaction vectors, the processor establishes an association matrix between each specified business type and interaction state, and outputs a response strategy using the association matrix. The response strategy is used to control the business response actions output by the financial branch robot.

[0140] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0141] Those skilled in the art will understand that all or part of the steps in the various business processing methods based on financial branch robots in the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0142] Example 4

[0143] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the business processing method based on the financial branch robot provided in Embodiment 1.

[0144] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the business processing method based on the financial branch robot of any one of the above embodiments.

[0145] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0146] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the business processing method based on a financial branch robot described in various embodiments of this application.

[0147] This application also provides a computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the business processing method based on a financial branch robot described in various embodiments of this application.

[0148] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0149] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0153] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0154] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A business processing method based on a financial branch robot, characterized in that, include: During the business interaction between the financial branch robot and the user, the system receives multimodal recognition information after feature integration and feature fusion from the financial branch robot, as well as operation logs sent by the financial application installed on the user's terminal. The operation logs include time-series operation information of the terminal page during the user's authorization to use the financial application. The multimodal recognition information and the operation log are mapped to a predetermined interaction vector space to output a continuous interaction vector. The historical interaction data in the continuous interaction vector is dynamically weighted by a pre-acquired interaction decay coefficient. Based on the weighted continuous interaction vector, an association matrix is ​​established between each specified business type and interaction state. The association matrix is ​​used to output a response strategy, wherein the response strategy is used to control the business response actions output by the financial branch robot.

2. The business processing method based on a financial branch robot according to claim 1, characterized in that, Before receiving the multimodal recognition information from the financial branch robot after feature integration and fusion, and the operation logs sent by the financial application installed on the user terminal, the process also includes: The initial multimodal information is obtained by collecting user interaction data during the business interaction process between the financial branch robot and the user. The spatiotemporal synchronization engine feature data is obtained by extracting features from the user interaction data through a multimodal data processing engine. The Transformer architecture module is used to perform feature fusion on the feature data of the spatiotemporal synchronization engine based on the self-attention mechanism to obtain the multimodal recognition information.

3. The business processing method based on a financial branch robot according to claim 1, characterized in that, Before receiving the multimodal recognition information from the financial branch robot after feature integration and fusion, and the operation logs sent by the financial application installed on the user terminal, the process also includes: After obtaining user authorization, the financial application installed on the user terminal receives data on the user's operational behavior, operation delay, page interruption frequency, business operation duration, and timestamps during the historical process of using the financial application to conduct business interactions, and obtains the operation log.

4. The business processing method based on a financial branch robot according to claim 1, characterized in that, The step of mapping the multimodal recognition information and the operation log to a predetermined interaction vector space and outputting continuous interaction vectors includes: The predetermined interaction vector space is selected as the predetermined interaction vector space, wherein the predetermined interaction vector space is configured with interaction vectors of three consecutive dimensions; Based on the pre-established operation behavior-interaction feature mapping table, the operation behavior data and business operation duration data in the operation log are mapped to the predetermined interaction vector space to obtain the first type of interaction vector and discrete interaction label; Based on a pre-established activity state-interaction feature mapping table, the user interaction data in the multimodal recognition information is mapped to the predetermined interaction vector space to obtain the second type of interaction vector and discrete interaction labels; The continuous interaction vector is output by combining the first type of interaction vector, the second type of interaction vector, and the discrete interaction tags.

5. The business processing method based on a financial branch robot according to claim 1, characterized in that, The step of dynamically adjusting the weights of historical interaction data in the continuous interaction vector using a pre-acquired interaction decay coefficient includes: The business risk level of the target financial business that the financial branch robot interacts with the user is obtained, wherein the business risk level is determined based on the business type and business importance level of the target financial business; Adjust the interaction attenuation coefficient according to the business risk level; Based on the interaction decay coefficient and the interval between historically identified interaction states, the interaction decay coefficient and the interval are input into the interaction decay model, and the interaction decay model dynamically adjusts the weights of the historical interaction data in the continuous interaction vector.

6. The business processing method based on a financial branch robot according to claim 1, characterized in that, The steps of using the correlation matrix to output the response strategy include: Determine the specific business type involved in the interaction between the financial branch robot and the user at the current moment; Obtain the identification results of the business interaction between the financial branch robot and the user at the current moment, as well as the dwell time; The association matrix is ​​invoked to index the robot response strategy corresponding to the specified business type, the business interaction identification result, and the dwell time, and the response strategy is output.

7. The business processing method based on a financial branch robot according to claim 1, characterized in that, Also includes: A federated learning algorithm is used to obtain abnormal interaction identification samples and interaction identification model parameters of each financial institution. The abnormal interaction identification samples include at least: abnormal interaction images, abnormal interaction features, and robot response strategies. The abnormal interaction identification samples are input into a preset generative adversarial network, which adjusts the established correlation matrix and the output response strategy.

8. The business processing method based on a financial branch robot according to claim 1, characterized in that, Also includes: Acquire the dynamic interaction status of users identified during business interactions between financial branch robots and users; The dynamic interactive state is input into the business association module, which then executes the false trigger circuit breaker mechanism for judgment. Under the false trigger circuit breaker mechanism, it is detected and determined whether a risk control action has been triggered and the number of times it has been triggered. If the number of times a risk control action is triggered reaches a preset threshold within a specified time period, the service process will switch from the financial branch server to a manual service process.

9. A business processing device based on a financial branch robot, characterized in that, include: The operation data receiving unit is used to receive multimodal recognition information after feature integration and feature fusion of the financial branch robot and operation logs sent by the financial application installed in the user terminal during the business interaction between the financial branch robot and the user. The operation logs include the time sequence operation information of the terminal page during the user's authorization to use the financial application. An interactive mapping unit is used to map the multimodal recognition information and the operation log to a predetermined interactive vector space, output a continuous interactive vector, and dynamically adjust the weight of historical interactive data in the continuous interactive vector using a pre-acquired interactive attenuation coefficient. The business response unit is used to establish an association matrix between each specified business type and the interaction state based on the weighted continuous interaction vector, and to output a response strategy using the association matrix. The response strategy is used to control the business response actions output by the financial branch robot.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the business processing method based on a financial branch robot as described in any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the business processing method based on a financial branch robot as described in any one of claims 1 to 8.

Images