Banking outlet service method and system based on multi-modal large model

Abstract

The invention provides a bank outlet service method and system based on a multi-modal large model in the technical field of financial science and technology. The method comprises the steps that S1, a server collects bank outlet operation data from outlet terminals of all bank outlets and stores the bank outlet operation data in a database; s2, acquiring a service query request input by a user terminal, analyzing the service query request to obtain a service demand and real-time position information, and analyzing the service demand through a multi-modal large model to obtain a user intention and an intention parameter; s3, generating an outlet recommendation list based on the user intention, the intention parameter, the real-time position information and the bank outlet operation data, and pushing the outlet recommendation list to the user terminal; and S4, the user terminal generates a reservation order based on the website recommendation list and sends the reservation order to the server, and the server forwards the reservation order to the corresponding website terminal and sends prompt information to the user terminal based on the reservation order. The method has the advantages that the intelligent level and the individuation capability of bank outlet services are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a bank branch service method and system based on a multimodal large model. Background Technology

[0002] As the digital transformation of the banking industry deepens, optimizing the service efficiency and customer experience of bank branches, as key nodes in offline financial services, has become a core issue for industry development. Currently, most banks have established online service channels, such as mobile banking applications and WeChat official accounts, enabling users to check basic branch information (including address and business hours) and make appointments for some services. In addition, some third-party information service platforms have also integrated publicly available information on bank branches within their region, providing basic branch search functions.

[0003] At the data application level, existing bank branch service systems mainly rely on data resources within a single bank, only able to obtain static information about the bank's branches and limited user data. In terms of service functions, most existing systems only support branch information display and simple business appointments, lacking the ability to make accurate branch recommendations based on factors such as the user's real-time geographical location, specific business needs, and real-time queuing status at each branch. At the intelligent interaction level, traditional customer service systems have limited ability to understand and process multimodal inputs (such as text, voice, and images), making it difficult to accurately interpret user intent and trigger corresponding service processes. In terms of customer group mining at the bank's end, existing analysis methods are mostly based on internal bank transaction data and have failed to effectively integrate external data such as users' life scenarios, resulting in insufficient accuracy in identifying potential target customer groups.

[0004] Therefore, how to provide a bank branch service method and system based on a multimodal large model to improve the intelligence and personalization capabilities of bank branch services has become an urgent technical problem to be solved. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a bank branch service method and system based on a multimodal large model, so as to improve the intelligence level and personalization capability of bank branch services.

[0006] In a first aspect, the present invention provides a bank branch service method based on a multimodal large model, comprising the following steps: Step S1: The server collects bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of special services, branch queuing status data and user behavior data. After preprocessing the bank branch operation data, the server stores it in the database. Step S2: The server obtains a business query request input from the user terminal, which carries at least business requirements and real-time location information. It parses the business query request to obtain the business requirements and real-time location information, and uses a pre-trained multimodal large model to parse the business requirements to obtain the user intent and intent parameters. Step S3: The server generates a branch recommendation list based on the user intent, intent parameters, real-time location information and bank branch operation data, and pushes the branch recommendation list to the user terminal. Step S4: The user terminal generates a reservation order based on the network recommendation list and sends it to the server. The server forwards the received reservation order to the corresponding network terminal and sends a prompt message to the user terminal based on the reservation order. Step S5: The server records user signaling data in real time, creates user profiles based on the user signaling data, and pushes the user profiles to the associated branch terminals.

[0007] Furthermore, step S1 specifically includes: The server collects bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of featured services, current queuing status data, and user behavior data. The basic branch information includes at least the branch name, address, contact information, business hours, and size. The featured service details include at least the featured service items and application requirements. Featured services include at least precious metals trading, corporate lending, and personal financial management. The current queuing status data includes at least the current number of people in the queue, waiting time for each service type, and the number of open service windows. The user behavior data includes at least historical transaction records, query preferences, and user signaling data. The operational data of each bank branch is preprocessed, including at least deduplication, error processing, missing data processing, data structuring, and data standardization, and the preprocessed operational data of each bank branch is stored in a distributed database.

[0008] Furthermore, step S2 specifically includes: The user terminal obtains the user's input business requirements based on the trigger signals of the text input box, voice input button, or image upload button, collects the real-time location information of the local machine, and sends a business query request to the server carrying at least the business requirements and real-time location information. The server receives the business query request in real time, parses the business query request to obtain the business requirements and real-time location information, and uses a pre-trained multimodal large model to parse the business requirements to obtain the user intent and intent parameters.

[0009] Furthermore, step S3 specifically includes: The server uses a hybrid intelligent algorithm based on collaborative filtering and geographic weighted regression to calculate the distance between the real-time location information and each bank branch. It combines the current queuing status data, user intent, and intent parameters to give each bank branch a comprehensive score. Based on the branch scores, a preset number of bank branches are selected to generate a branch recommendation list. The branch recommendation list is then converted into natural language form through the multimodal big data model and pushed to the user terminal.

[0010] Furthermore, step S4 specifically includes: The user terminal displays the received list of recommended branches, generates an appointment order for the corresponding bank branch based on the input selection signal and business needs, and sends it to the server. The server forwards the received appointment order to the corresponding branch terminal and sends a prompt message to the user terminal based on the appointment order, including at least a list of required documents, the processing procedure, and the branch navigation route.

[0011] Secondly, the present invention provides a bank branch service system based on a multimodal large model, comprising the following modules: The bank branch operation data collection module is used by the server to collect bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of special services, branch queuing status data and user behavior data. The bank branch operation data is preprocessed and then stored in the database. The business query request parsing module is used by the server to obtain a business query request input by the user terminal, which carries at least business requirements and real-time location information. The server parses the business query request to obtain the business requirements and real-time location information, and uses a pre-trained multimodal large model to parse the business requirements to obtain the user intent and intent parameters. The branch recommendation module is used by the server to generate a branch recommendation list based on the user intent, intent parameters, real-time location information and bank branch operation data, and push the branch recommendation list to the user terminal. The appointment order sending module is used by the user terminal to generate an appointment order based on the network recommendation list and send it to the server. The server forwards the received appointment order to the corresponding network terminal and sends a prompt message to the user terminal based on the appointment order. The user profile creation module is used by the server to record user signaling data in real time, create user profiles based on the user signaling data, and push the user profiles to the associated branch terminals.

[0012] Furthermore, the bank branch operation data collection module is specifically used for: The server collects bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of featured services, current queuing status data, and user behavior data. The basic branch information includes at least the branch name, address, contact information, business hours, and size. The featured service details include at least the featured service items and application requirements. Featured services include at least precious metals trading, corporate lending, and personal financial management. The current queuing status data includes at least the current number of people in the queue, waiting time for each service type, and the number of open service windows. The user behavior data includes at least historical transaction records, query preferences, and user signaling data. The operational data of each bank branch is preprocessed, including at least deduplication, error processing, missing data processing, data structuring, and data standardization, and the preprocessed operational data of each bank branch is stored in a distributed database.

[0013] Furthermore, the business query request parsing module is specifically used for: The user terminal obtains the user's input business requirements based on the trigger signals of the text input box, voice input button, or image upload button, collects the real-time location information of the local machine, and sends a business query request to the server carrying at least the business requirements and real-time location information. The server receives the business query request in real time, parses the business query request to obtain the business requirements and real-time location information, and uses a pre-trained multimodal large model to parse the business requirements to obtain the user intent and intent parameters.

[0014] Furthermore, the outlet recommendation module is specifically used for: The server uses a hybrid intelligent algorithm based on collaborative filtering and geographic weighted regression to calculate the distance between the real-time location information and each bank branch. It combines the current queuing status data, user intent, and intent parameters to give each bank branch a comprehensive score. Based on the branch scores, a preset number of bank branches are selected to generate a branch recommendation list. The branch recommendation list is then converted into natural language form through the multimodal big data model and pushed to the user terminal.

[0015] Furthermore, the appointment order sending module is specifically used for: The user terminal displays the received list of recommended branches, generates an appointment order for the corresponding bank branch based on the input selection signal and business needs, and sends it to the server. The server forwards the received appointment order to the corresponding branch terminal and sends a prompt message to the user terminal based on the appointment order, including at least a list of required documents, the processing procedure, and the branch navigation route.

[0016] The advantages of this invention are: 1. The system collects bank branch operation data from branch terminals at various bank outlets via a server. This data includes at least basic branch information, details of featured services, current queuing status data, and user behavior data. After preprocessing the operational data, it is stored in a database. Next, the system acquires business query requests from user terminals, which at least include business requirements and real-time location information. These requests are parsed to obtain the business requirements and real-time location information. A multimodal big data model is used to parse the business requirements, obtaining user intent and intent parameters. Based on the user intent, intent parameters, real-time location information, and bank branch operation data, a branch recommendation list is generated and pushed to the user terminal. The user terminal generates an appointment order based on the branch recommendation list and sends it to the server. The server then forwards the appointment order to the corresponding branch. The system automatically sends notifications to user terminals based on appointment orders; the server records user signaling data in real time, creates user profiles based on this data, and pushes these profiles to associated branch terminals; it uses a pre-trained multimodal large model to accurately analyze user business needs expressed in text, voice, or image formats, gaining a deep understanding of their intentions; it then integrates real-time user location, branch static information, and dynamic queuing data, using intelligent algorithms to generate a personalized branch recommendation list; finally, it proactively provides full-process guidance after the appointment is completed, and empowers branches by building dynamic profiles based on user signaling data. This achieves a leap from passive inquiry to proactive intelligent recommendation, and from single information display to full-process personalized service, ultimately greatly improving the intelligence and personalization capabilities of bank branch services.

[0017] 2. A pre-trained multimodal large model is used to parse user input business requirements, supporting multiple input methods such as text, voice, and images. This not only enhances the flexibility and convenience of user interaction, but also accurately extracts user intent and parameters through advanced natural language processing technology. This multimodal processing capability reduces the limitations of traditional single input methods, improves the accuracy and efficiency of intent recognition, and thus provides users with more accurate service matching. This reflects the innovative application of artificial intelligence in the financial services field and enhances the system's intelligence level and user experience.

[0018] 3. The server collects multi-dimensional operational data from the terminals of various bank branches, including basic information, special services, queuing status, and user behavior data. It then performs preprocessing steps such as duplication processing, error correction, missing data filling, structuring, and standardization before storing the data in a distributed database. This comprehensive data management ensures high-quality and consistent data, providing a reliable foundation for subsequent recommendation algorithms. At the same time, the use of a distributed database enhances the system's scalability and fault tolerance, which is beneficial for handling large-scale real-time data and improves the overall system stability and performance.

[0019] 4. By combining a hybrid intelligent algorithm of collaborative filtering and geographic weighted regression, a network point recommendation list is generated based on user intent, real-time location, and network point data (such as queue length and waiting time), and a comprehensive score is applied. This algorithm not only considers geographic proximity but also incorporates user preferences and real-time conditions, achieving highly personalized and dynamic recommendations, effectively reducing user waiting time and optimizing network point resource allocation. In addition, the recommendation results are converted into natural language output, enhancing user-friendliness, improving service efficiency, and demonstrating the advanced nature and practicality of data-driven decision-making.

[0020] 5. The system automates the entire process from user selection of a recommended branch to the generation of an appointment order. The server automatically forwards the order to the corresponding branch terminal and sends detailed prompts to the user, including a list of required materials, procedures, and navigation routes. This simplifies the cumbersome steps of traditional manual appointments, reduces human error, and improves processing speed. At the same time, by providing comprehensive prompts, it helps users prepare in advance, thereby improving business processing efficiency and customer satisfaction. This demonstrates the significant advantages of automation technology in improving the efficiency of bank service operations.

[0021] 6. The server records user signaling data in real time and creates user profiles based on this data, which are then pushed to the terminals of associated branches. This enables banks to continuously learn and adapt to user behavior, providing personalized recommendations and optimizations for future services, such as customizing services based on historical preferences.

[0022] 7. By integrating pre-trained multimodal large models to process various input methods such as text, voice, and images, the system intelligently interprets user intent and combines comprehensively collected bank branch operation data (including basic information, special services, queuing status, and user behavior data) for efficient preprocessing and distributed storage. A hybrid algorithm of collaborative filtering and geographic weighted regression is used to generate personalized, real-time branch recommendation lists, automate appointment order processing and notification message sending, and build user profiles based on real-time user signaling data and push them to branch terminals. This significantly improves the intelligence level of banking services, data processing reliability, recommendation accuracy, operational efficiency, personalized experience, and real-time system response capabilities, thereby optimizing the overall user experience and bank operational efficiency. Attached Figure Description

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] Figure 1 This is a flowchart of a bank branch service method based on a multimodal large model according to the present invention.

[0025] Figure 2 This is a schematic diagram of the structure of a bank branch service system based on a multimodal large model according to the present invention. Detailed Implementation

[0026] The technical solution in this application embodiment follows the following general approach: It uses a multimodal large model to accurately analyze the business needs expressed by users in the form of text, voice, or images, gaining a deep understanding of their intentions; then, it integrates the user's real-time location, static branch information, and dynamic queuing data, using intelligent algorithms to generate a personalized branch recommendation list; finally, it proactively provides full-process guidance after the appointment is completed, and empowers branches by building dynamic profiles based on user signaling data. This achieves a leap from passive querying to proactive intelligent recommendation, and from single information display to full-process personalized service, thereby improving the intelligence level and personalization capabilities of bank branch services.

[0027] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the bank branch service method based on a multimodal large model of the present invention includes the following steps: Step S1: The server collects bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of special services, branch queuing status data and user behavior data. After preprocessing the bank branch operation data, the server stores it in the database. Step S2: The server obtains a business query request input from the user terminal, which carries at least business requirements and real-time location information. It parses the business query request to obtain the business requirements and real-time location information, and uses a pre-trained multimodal large model to parse the business requirements to obtain the user intent and intent parameters. Step S3: The server generates a branch recommendation list based on the user intent, intent parameters, real-time location information and bank branch operation data, and pushes the branch recommendation list to the user terminal. Step S4: The user terminal generates a reservation order based on the network recommendation list and sends it to the server. The server forwards the received reservation order to the corresponding network terminal and sends a prompt message to the user terminal based on the reservation order. Step S5: The server records user signaling data in real time, creates user profiles based on the user signaling data, and pushes the user profiles to associated branch terminals. In practice, users whose signaling data appears a preset threshold number of times within a 3-kilometer radius of the target branch can be selected to identify potential customer groups. The user signaling data can also be used to create movement trajectories to categorize them as belonging to a specific branch. Further analysis of the selected users can be conducted, combining historical business preferences and spending power data to construct potential customer profiles, clarifying the age distribution, business needs, and consumption habits of the customer group.

[0028] Based on the data format provided by the user signaling data, the latitude and longitude range of the area within 3 kilometers of the network point is calculated using the Haversine formula. The calculation formula is as follows: ; Where (Lng1,Lat1) represents the latitude and longitude of point A; (Lng2,Lat2) represents the latitude and longitude of point B; a = Lat1 - Lat2 represents the difference in latitude between the two points; b = Lng1 - Lng2 represents the difference in longitude between the two points.

[0029] When the coordinates (26.09367, 119.30712) were selected as the pilot site address, a 3-kilometer geographical area was defined, with latitude ranging from [26.06672, 26.1206] and longitude ranging from [119.27711, 119.33712]. This geographical area was defined as the site ecosystem, and user signaling data within this latitude and longitude range was filtered. After processing, only 3 user signaling data entries remained in the training set. Considering the small data size, the data retrieval distance was adjusted to 30 kilometers, with latitude ranging from [25.82417, 26.36316] and longitude ranging from [119.00704, 119.6072]. After filtering, 93,775 signaling data entries were retained, averaging approximately 18,000 entries per day. Based on the selected signaling data, the frequency of overlap between signaling and the service area of ​​the outlet is counted using mobile phone numbers as IDs to identify identity information. The average daily stopover time of individual signaling data is also counted. Customers with an overlap frequency of 4 or more and an average daily stopover time of more than half an hour are selected as potential customers.

[0030] This invention integrates operational data from all bank branches within a region via a server, breaking down data silos. It introduces intelligent algorithms that combine real-time location information, business needs (required service types), and queuing status at each branch to recommend the nearest and most suitable branch to users, providing online appointment booking, queue prediction, and personalized service guidance. A multimodal model is embedded in the intelligent customer service module to parse user input (text, voice, images, etc.), accurately identifying the user's multi-turn dialogue intent and extracting the parameters required for corresponding functions, enabling automated service calls and efficient intelligent interaction. The bank terminal utilizes user signaling data obtained from the server to analyze the overlap between the user's living area and the branch location (signaling data appearing multiple times within a three-kilometer geographical area centered on the branch), identifying potential customer groups for the branch and supporting precise marketing for the bank.

[0031] This invention integrates data from multiple banks and of various types within a region through a server, providing comprehensive and rich data support for subsequent intelligent services. It solves the problem of single data resources in existing technologies, breaking down data silos and achieving data resource integration. An intelligent recommendation algorithm combining user location, service needs, and branch queuing status can match users with the nearest and most suitable branch. Combined with online appointments, queue prediction, and personalized guidance services, it significantly reduces user waiting time and improves the offline service experience. The application of a multimodal large model enables accurate parsing of text, voice, and image inputs and accurate recognition of user intent. It can quickly extract key parameters and complete service calls, improving the interaction efficiency and service quality of intelligent customer service. Based on user signaling data and community analysis, it can accurately locate potential customer groups that highly overlap with the branch's coverage area, providing precise marketing directions for bank branches, improving marketing conversion rates, and enhancing the bank's market competitiveness.

[0032] Step S1 specifically involves: The server collects bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of featured services, current queuing status data, and user behavior data. The basic branch information includes at least the branch name, address, contact information, business hours, and size. The featured service details include at least the featured service items and application requirements. Featured services include at least precious metals trading, corporate lending, and personal financial management. The current queuing status data includes at least the current number of people in the queue, waiting time for each service type, and the number of open service windows. The user behavior data includes at least historical transaction records, query preferences, and user signaling data. The operational data of each bank branch undergoes preprocessing, including at least deduplication, error handling, missing data handling, data structuring, and data standardization. The preprocessed operational data of each bank branch is then stored in a distributed database to ensure data security and scalability.

[0033] For example, the text descriptions and user voice interaction content in the details of featured services are structured and converted into text data; the geographic location data is standardized and uniformly adopted in the latitude and longitude coordinate format.

[0034] Step S2 specifically involves: The user terminal obtains the user's input business requirements based on the trigger signals of the text input box, voice input button, or image upload button, collects the real-time location information of the local machine, and sends a business query request to the server carrying at least the business requirements and real-time location information; the collection of the real-time location information requires prior user authorization. The server receives the business query requests in real time, parses them to obtain business requirements and real-time location information, and then uses a pre-trained multimodal large model to parse the business requirements to obtain user intent and intent parameters. In practice, dialogue interaction can be performed through the multimodal large model.

[0035] For voice input, it is first converted into text information using speech recognition technology; for image input (such as photos of business documents), text information is extracted using OCR technology; then, the parsed text information is combined with the context and dialogue history to identify the user's intent (such as branch appointment, business consultation, queue inquiry, etc.), and the intent parameters corresponding to the intent are extracted.

[0036] An example of a dialogue process in a multimodal large model is as follows: ① Customer question: "I want to book a personal banking appointment at the nearest bank branch"; ② Multimodal large model recognition content: "Intent: Branch appointment", "Parameters": "Bank, Business type: Private, Filtering condition: Recent, Customer location: (120.123456, 20.654321)"; ③ Smart reply: "Do you need to schedule a personal banking appointment at ** branch?"; ④ After the customer confirms again, the multimodal big model calls the service corresponding to the intent to complete the branch reservation.

[0037] Step S3 specifically involves: The server uses a hybrid intelligent algorithm based on collaborative filtering and geographic weighted regression to calculate the distance between the real-time location information and each bank branch. It combines the current queuing status data, user intent, and intent parameters to give each bank branch a comprehensive score. Based on the branch scores, a preset number of bank branches are selected to generate a branch recommendation list. The branch recommendation list is then converted into natural language form through the multimodal big data model and pushed to the user terminal.

[0038] Step S4 specifically involves: The user terminal displays the received list of recommended branches, generates an appointment order for the corresponding bank branch based on the input selection signal and business needs, and sends it to the server. The server forwards the received appointment order to the corresponding branch terminal and sends a prompt message to the user terminal based on the appointment order, including at least a list of required documents, the processing procedure, and the branch navigation route.

[0039] In practice, the reservation order will be synchronized to the queuing and calling system of the corresponding outlet terminal, and a reservation success notification and reservation number will be sent to the user terminal. At the same time, based on historical queuing data and real-time queuing status, a time series prediction algorithm will be used to predict the waiting time for the user after arriving at the outlet.

[0040] A preferred embodiment of a bank branch service system based on a multimodal large model according to the present invention includes the following modules: The bank branch operation data collection module is used by the server to collect bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of special services, branch queuing status data and user behavior data. The bank branch operation data is preprocessed and then stored in the database. The business query request parsing module is used by the server to obtain a business query request input by the user terminal, which carries at least business requirements and real-time location information. The server parses the business query request to obtain the business requirements and real-time location information, and uses a pre-trained multimodal large model to parse the business requirements to obtain the user intent and intent parameters. The branch recommendation module is used by the server to generate a branch recommendation list based on the user intent, intent parameters, real-time location information and bank branch operation data, and push the branch recommendation list to the user terminal. The appointment order sending module is used by the user terminal to generate an appointment order based on the network recommendation list and send it to the server. The server forwards the received appointment order to the corresponding network terminal and sends a prompt message to the user terminal based on the appointment order. The user profile creation module is used by the server to record user signaling data in real time, create user profiles based on the signaling data, and push the user profiles to associated branch terminals. In practice, users whose signaling data appears a preset threshold number of times within a three-kilometer radius of the target branch can be selected to identify potential customer groups. The signaling data can also be used to create movement trajectories to assign users to a specific branch. Further analysis of the selected users, combined with their historical business preferences and spending power, can construct potential customer profiles, clarifying characteristics such as age distribution, business needs, and consumption habits.

[0041] Based on the data format provided by the user signaling data, the latitude and longitude range of the area within 3 kilometers of the network point is calculated using the Haversine formula. The calculation formula is as follows: ; Where (Lng1,Lat1) represents the latitude and longitude of point A; (Lng2,Lat2) represents the latitude and longitude of point B; a = Lat1 - Lat2 represents the difference in latitude between the two points; b = Lng1 - Lng2 represents the difference in longitude between the two points.

[0042] When the coordinates (26.09367, 119.30712) were selected as the pilot site address, a 3-kilometer geographical area was defined, with latitude ranging from [26.06672, 26.1206] and longitude ranging from [119.27711, 119.33712]. This geographical area was defined as the site ecosystem, and user signaling data within this latitude and longitude range was filtered. After processing, only 3 user signaling data entries remained in the training set. Considering the small data size, the data retrieval distance was adjusted to 30 kilometers, with latitude ranging from [25.82417, 26.36316] and longitude ranging from [119.00704, 119.6072]. After filtering, 93,775 signaling data entries were retained, averaging approximately 18,000 entries per day. Based on the selected signaling data, the frequency of overlap between signaling and the service area of ​​the outlet is counted using mobile phone numbers as IDs to identify identity information. The average daily stopover time of individual signaling data is also counted. Customers with an overlap frequency of 4 or more and an average daily stopover time of more than half an hour are selected as potential customers.

[0043] This invention integrates operational data from all bank branches within a region via a server, breaking down data silos. It introduces intelligent algorithms that combine real-time location information, business needs (required service types), and queuing status at each branch to recommend the nearest and most suitable branch to users, providing online appointment booking, queue prediction, and personalized service guidance. A multimodal model is embedded in the intelligent customer service module to parse user input (text, voice, images, etc.), accurately identifying the user's multi-turn dialogue intent and extracting the parameters required for corresponding functions, enabling automated service calls and efficient intelligent interaction. The bank terminal utilizes user signaling data obtained from the server to analyze the overlap between the user's living area and the branch location (signaling data appearing multiple times within a three-kilometer geographical area centered on the branch), identifying potential customer groups for the branch and supporting precise marketing for the bank.

[0044] This invention integrates data from multiple banks and of various types within a region through a server, providing comprehensive and rich data support for subsequent intelligent services. It solves the problem of single data resources in existing technologies, breaking down data silos and achieving data resource integration. An intelligent recommendation algorithm combining user location, service needs, and branch queuing status can match users with the nearest and most suitable branch. Combined with online appointments, queue prediction, and personalized guidance services, it significantly reduces user waiting time and improves the offline service experience. The application of a multimodal large model enables accurate parsing of text, voice, and image inputs and accurate recognition of user intent. It can quickly extract key parameters and complete service calls, improving the interaction efficiency and service quality of intelligent customer service. Based on user signaling data and community analysis, it can accurately locate potential customer groups that highly overlap with the branch's coverage area, providing precise marketing directions for bank branches, improving marketing conversion rates, and enhancing the bank's market competitiveness.

[0045] The bank branch operation data collection module is specifically used for: The server collects bank branch operation data from the branch terminals of each bank branch, including at least basic branch information, details of featured services, current queuing status data, and user behavior data. The basic branch information includes at least the branch name, address, contact information, business hours, and size. The featured service details include at least the featured service items and application requirements. Featured services include at least precious metals trading, corporate lending, and personal financial management. The current queuing status data includes at least the current number of people in the queue, waiting time for each service type, and the number of open service windows. The user behavior data includes at least historical transaction records, query preferences, and user signaling data. The operational data of each bank branch undergoes preprocessing, including at least deduplication, error handling, missing data handling, data structuring, and data standardization. The preprocessed operational data of each bank branch is then stored in a distributed database to ensure data security and scalability.

[0046] For example, the text descriptions and user voice interaction content in the details of featured services are structured and converted into text data; the geographic location data is standardized and uniformly adopted in the latitude and longitude coordinate format.

[0047] The business query request parsing module is specifically used for: The user terminal obtains the user's input business requirements based on the trigger signals of the text input box, voice input button, or image upload button, collects the real-time location information of the local machine, and sends a business query request to the server carrying at least the business requirements and real-time location information; the collection of the real-time location information requires prior user authorization. The server receives the business query requests in real time, parses them to obtain business requirements and real-time location information, and then uses a pre-trained multimodal large model to parse the business requirements to obtain user intent and intent parameters. In practice, dialogue interaction can be performed through the multimodal large model.

[0048] For voice input, it is first converted into text information using speech recognition technology; for image input (such as photos of business documents), text information is extracted using OCR technology; then, the parsed text information is combined with the context and dialogue history to identify the user's intent (such as branch appointment, business consultation, queue inquiry, etc.), and the intent parameters corresponding to the intent are extracted.

[0049] An example of a dialogue process in a multimodal large model is as follows: ① Customer question: "I want to book a personal banking appointment at the nearest bank branch"; ② Multimodal large model recognition content: "Intent: Branch appointment", "Parameters": "Bank, Business type: Private, Filtering condition: Recent, Customer location: (120.123456, 20.654321)"; ③ Smart reply: "Do you need to schedule a personal banking appointment at ** branch?"; ④ After the customer confirms again, the multimodal big model calls the service corresponding to the intent to complete the branch reservation.

[0050] The outlet recommendation module is specifically used for: The server uses a hybrid intelligent algorithm based on collaborative filtering and geographic weighted regression to calculate the distance between the real-time location information and each bank branch. It combines the current queuing status data, user intent, and intent parameters to give each bank branch a comprehensive score. Based on the branch scores, a preset number of bank branches are selected to generate a branch recommendation list. The branch recommendation list is then converted into natural language form through the multimodal big data model and pushed to the user terminal.

[0051] The reservation order sending module is specifically used for: The user terminal displays the received list of recommended branches, generates an appointment order for the corresponding bank branch based on the input selection signal and business needs, and sends it to the server. The server forwards the received appointment order to the corresponding branch terminal and sends a prompt message to the user terminal based on the appointment order, including at least a list of required documents, the processing procedure, and the branch navigation route.

[0052] In practice, the reservation order will be synchronized to the queuing and calling system of the corresponding outlet terminal, and a reservation success notification and reservation number will be sent to the user terminal. At the same time, based on historical queuing data and real-time queuing status, a time series prediction algorithm will be used to predict the waiting time for the user after arriving at the outlet.

[0053] In summary, the advantages of this invention are as follows: 1. The system collects bank branch operation data from branch terminals at various bank outlets via a server. This data includes at least basic branch information, details of featured services, current queuing status data, and user behavior data. After preprocessing the operational data, it is stored in a database. Next, the system acquires business query requests from user terminals, which at least include business requirements and real-time location information. These requests are parsed to obtain the business requirements and real-time location information. A multimodal big data model is used to parse the business requirements, obtaining user intent and intent parameters. Based on the user intent, intent parameters, real-time location information, and bank branch operation data, a branch recommendation list is generated and pushed to the user terminal. The user terminal generates an appointment order based on the branch recommendation list and sends it to the server. The server then forwards the appointment order to the corresponding branch. The system automatically sends notifications to user terminals based on appointment orders; the server records user signaling data in real time, creates user profiles based on this data, and pushes these profiles to associated branch terminals; it uses a pre-trained multimodal large model to accurately analyze user business needs expressed in text, voice, or image formats, gaining a deep understanding of their intentions; it then integrates real-time user location, branch static information, and dynamic queuing data, using intelligent algorithms to generate a personalized branch recommendation list; finally, it proactively provides full-process guidance after the appointment is completed, and empowers branches by building dynamic profiles based on user signaling data. This achieves a leap from passive inquiry to proactive intelligent recommendation, and from single information display to full-process personalized service, ultimately greatly improving the intelligence and personalization capabilities of bank branch services.

[0054] 2. A pre-trained multimodal large model is used to parse user input business requirements, supporting multiple input methods such as text, voice, and images. This not only enhances the flexibility and convenience of user interaction, but also accurately extracts user intent and parameters through advanced natural language processing technology. This multimodal processing capability reduces the limitations of traditional single input methods, improves the accuracy and efficiency of intent recognition, and thus provides users with more accurate service matching. This reflects the innovative application of artificial intelligence in the financial services field and enhances the system's intelligence level and user experience.

[0055] 3. The server collects multi-dimensional operational data from the terminals of various bank branches, including basic information, special services, queuing status, and user behavior data. It then performs preprocessing steps such as duplication processing, error correction, missing data filling, structuring, and standardization before storing the data in a distributed database. This comprehensive data management ensures high-quality and consistent data, providing a reliable foundation for subsequent recommendation algorithms. At the same time, the use of a distributed database enhances the system's scalability and fault tolerance, which is beneficial for handling large-scale real-time data and improves the overall system stability and performance.

[0056] 4. By combining a hybrid intelligent algorithm of collaborative filtering and geographic weighted regression, a network point recommendation list is generated based on user intent, real-time location, and network point data (such as queue length and waiting time), and a comprehensive score is applied. This algorithm not only considers geographic proximity but also incorporates user preferences and real-time conditions, achieving highly personalized and dynamic recommendations, effectively reducing user waiting time and optimizing network point resource allocation. In addition, the recommendation results are converted into natural language output, enhancing user-friendliness, improving service efficiency, and demonstrating the advanced nature and practicality of data-driven decision-making.

[0057] 5. The system automates the entire process from user selection of a recommended branch to the generation of an appointment order. The server automatically forwards the order to the corresponding branch terminal and sends detailed prompts to the user, including a list of required materials, procedures, and navigation routes. This simplifies the cumbersome steps of traditional manual appointments, reduces human error, and improves processing speed. At the same time, by providing comprehensive prompts, it helps users prepare in advance, thereby improving business processing efficiency and customer satisfaction. This demonstrates the significant advantages of automation technology in improving the efficiency of bank service operations.

[0058] 6. The server records user signaling data in real time and creates user profiles based on this data, which are then pushed to the terminals of associated branches. This enables banks to continuously learn and adapt to user behavior, providing personalized recommendations and optimizations for future services, such as customizing services based on historical preferences.

[0059] 7. By integrating pre-trained multimodal large models to process various input methods such as text, voice, and images, the system intelligently interprets user intent and combines comprehensively collected bank branch operation data (including basic information, special services, queuing status, and user behavior data) for efficient preprocessing and distributed storage. A hybrid algorithm of collaborative filtering and geographic weighted regression is used to generate personalized, real-time branch recommendation lists, automate appointment order processing and notification message sending, and build user profiles based on real-time user signaling data and push them to branch terminals. This significantly improves the intelligence level of banking services, data processing reliability, recommendation accuracy, operational efficiency, personalized experience, and real-time system response capabilities, thereby optimizing the overall user experience and bank operational efficiency.

[0060] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A bank outlet service method based on a multi-modal large model, characterized in that: The method comprises the following steps: Step S1, the server collects bank outlet operation data including at least outlet basic information, special service details, outlet queuing status data and user behavior data from the outlet terminals of each bank outlet, pre-processes each of the bank outlet operation data and stores it into a database; Step S2, the server obtains a service query request carrying at least service demand and real-time location information input by a user terminal, analyzes the service query request to obtain service demand and real-time location information, and analyzes the service demand through a pre-trained multi-modal large model to obtain user intent and intent parameters; Step S3, the server generates an outlet recommendation list based on the user intent, intent parameters, real-time location information and bank outlet operation data, and pushes the outlet recommendation list to the user terminal; Step S4, the user terminal generates a reservation order based on the outlet recommendation list and sends it to the server, the server forwards the received reservation order to the corresponding outlet terminal, and sends prompt information to the user terminal based on the reservation order; Step S5, the server records user signaling data in real time, creates a user portrait based on each of the user signaling data, and pushes the user portrait to the associated outlet terminal.

2. The bank outlet service method based on a multi-modal large model according to claim 1, characterized in that: The step S1 is specifically: The server collects bank outlet operation data including at least outlet basic information, special service details, outlet queuing status data and user behavior data from the outlet terminals of each bank outlet; the outlet basic information at least includes outlet name, outlet address, outlet contact information, outlet business hours and outlet scale; the special service details at least include special service items and handling conditions; the special service items at least include precious metal transactions, enterprise credit and personal financial management; the outlet queuing status data at least includes current number of people in line, waiting time for each business type and number of windows open; the user behavior data at least includes historical business handling records, query preferences and user signaling data; The pre-processing of each of the bank outlet operation data at least includes duplicate data processing, error data processing, missing data processing, data structuring and data standardization, and each of the pre-processed bank outlet operation data is stored into a distributed database. 3.The bank outlet service method based on a multi-modal large model according to claim 1, characterized in that: The step S2 is specifically: The user terminal obtains service demand input by a user based on a trigger signal of a text input box, a voice recording button or a picture uploading button, collects real-time location information of the local machine, and sends a service query request carrying at least the service demand and real-time location information to the server; The server receives the service query request in real time, analyzes the service query request to obtain service demand and real-time location information, and analyzes the service demand through a pre-trained multi-modal large model to obtain user intent and intent parameters. 4.The bank outlet service method based on a multi-modal large model according to claim 1, wherein: The step S3 is specifically: The server calculates the distance between the real-time location information and each bank outlet based on a hybrid intelligent algorithm of collaborative filtering and geographic weighted regression, combines outlet queuing status data, user intent and intent parameters to comprehensively score each bank outlet to obtain an outlet score, filters a preset number of bank outlets based on the outlet score to generate an outlet recommendation list, and pushes the outlet recommendation list in natural language form to the user terminal after conversion by the multi-modal large model. 5.The bank outlet service method based on a multi-modal large model according to claim 1, wherein: The step S4 specifically includes: The user terminal displays the received outlet recommendation list, generates a reservation order for the corresponding bank outlet based on the input selection signal and business demand, and sends the reservation order to the server. The server forwards the received reservation order to the corresponding outlet terminal and sends prompt information including at least a material list, a handling process and an outlet navigation route to the user terminal based on the reservation order.

6. A bank outlet service system based on a multi-modal large model, characterized in that: The method comprises the following modules: A bank outlet operation data collection module is configured to collect, by the server, bank outlet operation data including at least bank outlet basic information, special service details, outlet queuing status data and user behavior data from outlet terminals of each bank outlet, and store the preprocessed bank outlet operation data to a database; A business query request analysis module is configured to obtain, by the server, a business query request including at least a business demand and real-time location information input by the user terminal, analyze the business query request to obtain the business demand and real-time location information, and analyze the business demand by a pre-trained multi-modal large model to obtain user intent and intent parameters; An outlet recommendation module is configured to generate, by the server, an outlet recommendation list based on the user intent, intent parameters, real-time location information and bank outlet operation data, and push the outlet recommendation list to the user terminal; A reservation order sending module is configured to generate, by the user terminal, a reservation order based on the outlet recommendation list and send the reservation order to the server. The server forwards the received reservation order to the corresponding outlet terminal and sends prompt information to the user terminal based on the reservation order; A user portrait creation module is configured to record, by the server, user signaling data in real time, create a user portrait based on each user signaling data, and push the user portrait to the associated outlet terminal.

7. The bank outlet service system based on a multi-modal large model according to claim 6, characterized in that: The bank outlet operation data collection module is specifically configured to: The server collects bank outlet operation data including at least bank outlet basic information, special service details, outlet queuing status data and user behavior data from outlet terminals of each bank outlet. The bank outlet basic information includes at least outlet name, outlet address, outlet contact information, outlet business hours and outlet scale. The special service details include at least special service items and handling conditions. The special service items include at least precious metal transactions, enterprise credit and personal financial management. The outlet queuing status data includes at least current number of people in line, waiting time for each business type and number of open windows. The user behavior data includes at least historical business handling records, query preferences and user signaling data. The bank point operation data is preprocessed, including at least repeated data processing, error data processing, missing data processing, data structuring and data standardization, and the preprocessed bank point operation data is stored in a distributed database.

8. The bank outlet service system based on a multi-modal large model of claim 6, wherein: The business query request analysis module is specifically used for: The user terminal obtains the business demand input by the user based on the trigger signal of the text input box, the voice recording button or the picture uploading button, collects real-time location information of the local terminal, and sends a business query request carrying at least the business demand and the real-time location information to the server; The server receives the business query request in real time, analyzes the business query request to obtain the business demand and the real-time location information, and analyzes the business demand by using a pre-trained multi-modal large model to obtain a user intention and an intention parameter.

9. The bank outlet service system based on a multi-modal large model of claim 6, wherein: The point recommendation module is specifically used for: The server calculates the distance between the real-time location information and each bank point based on a hybrid intelligent algorithm combining collaborative filtering and geographical weighted regression, combines the bank point queuing status data, the user intention and the intention parameter to score each bank point to obtain a point score, screens a preset number of bank points based on the point score to generate a point recommendation list, and pushes the point recommendation list converted into a natural language form to the user terminal by using the multi-modal large model.

10. The bank outlet service system based on a multi-modal large model of claim 6, wherein: The reservation order sending module is specifically used for: The user terminal displays the received point recommendation list, generates a reservation order of the corresponding bank point based on the input selection signal and the business demand, and sends the reservation order to the server, the server forwards the received reservation order to the corresponding point terminal, and sends prompt information including at least a material list, a handling process and a point navigation route to the user terminal based on the reservation order.

Images