Business adaptive resource scheduling method and device for bank counter video audit system

Abstract

The present invention discloses a business adaptive resource scheduling method and device for a bank counter video audit system, which relates to the field of video audit technology. The head office data center collects the audio streams of the call machines of each branch in real time, extracts the counter numbers through voice recognition and natural language understanding, dynamically establishes the corresponding counter audio and video transmission channels and allocates AI computing resources; extracts the business type based on real-time voice recognition and natural language understanding algorithms, and only calls the compliance audit algorithm of the business, and releases the resources after the business is completed. This solution reduces network transmission redundancy through a real-time association mechanism between business needs and resource allocation; and avoids redundant computing loads through a dynamic mapping mechanism between business types and audit algorithms. Compared with traditional solutions, it achieves dual optimization of network bandwidth and AI computing resources, significantly improving system resource utilization.

Description

Technical Field

[0001] The present invention relates to the technical field of video auditing, and in particular to a business adaptive resource scheduling method and device for a bank counter video auditing system. Background Art

[0002] The bank counter video audit system uses AI video recognition technology to monitor the entire process, establishing a "real-time monitoring and tracing" risk control system. This system accurately captures unusual transactions and irregularities, and simultaneously generates intelligent audit reports to mitigate operational risks. Existing bank counter video audit systems typically employ a centralized deployment approach, centrally deploying a high-performance AI server cluster in the head office data center. Each branch transmits video streams back to the head office data center in real time via a wide area network for auditing. While this model ensures centralized scheduling of AI computing resources, it also poses the problem of persistent network bandwidth usage. In-depth analysis revealed that this architecture wastes resources in two ways.

[0003] 1) In the time dimension, even during the 8-hour business hours when no customers are conducting business (such as when there are no customers at the branch or a certain counter), the system continues to transmit video streams and call AI computing resources to execute the audit algorithm, resulting in a double waste of network resources and AI computing resources.

[0004] 2) In the business dimension, a dynamic mapping mechanism between business types and audit rules has not been established. All audit algorithms are mechanically loaded for all business scenarios, generating unnecessary computing load and leading to a waste of AI computing resources.

[0005] In view of the above situation, there is an urgent need to design a business adaptive resource scheduling method and device for a bank counter video audit system to solve the above-mentioned double resource waste problem. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention provides a method and device for business adaptive resource scheduling of a bank counter video audit system. The following technical solutions are adopted:

[0007] The business adaptive resource scheduling method of the bank counter video audit system includes the following steps:

[0008] Step 1: When a bank branch opens in the morning, the head office data center allocates network resources I and AI computing resources I;

[0009] Step 2: The head office data center uses network resource I to obtain audio stream data from the queuing machines at all branches. The head office data center then runs a counter number extraction algorithm on AI computing resource I to extract the counter number from the audio stream data and marks the extracted counter number as bank counter Y.

[0010] Step 3: Allocate network resource II to establish a network link between the head office data center and bank counter Y obtained in step 2. The head office data center uses network resource II to collect audio and video stream data from bank counter Y in real time.

[0011] Step 4: Allocate AI computing resource II-A. The head office data center uses AI computing resource II-A to run a service etiquette audit algorithm to analyze the audio and video stream data from bank counter Y to check whether the teller's service process meets reception etiquette standards.

[0012] Step 5: Allocate AI computing resource II-B. The head office data center uses AI computing resource II-B to run a business type extraction algorithm to extract the business type from the audio and video stream of bank counter Y, obtaining business type Z. This process completes the process of handling business Z for the customer.

[0013] Step 6: Release AI computing resources II-B;

[0014] Step 7: Allocate AI computing resources II-C. The head office data center uses AI computing resources II-C to run the compliance audit algorithm for business type Z to check whether the processes in the audio and video streams of bank counter Y are compliant.

[0015] The head office data center uses AI computing resource II-C to run the transaction completion detection algorithm for transaction type Z. This algorithm detects whether the customer transaction process at bank counter Y has ended, as seen in the audio and video streams. If so, the algorithm stops step 7 and proceeds to step 8.

[0016] Step 8: Release network resources II and AI computing resources II-A and II-C;

[0017] Step 9: If the branch is still in business hours, return to step 2; if the branch has ended business for the day, the head office data center releases AI computing resources I and network resources I, and waits until business opens the next day before returning to step 1.

[0018] By adopting this technical solution, network transmission redundancy is reduced through a real-time correlation mechanism between business needs and resource allocation, and redundant computing loads are avoided through a dynamic mapping mechanism between business types and audit algorithms. Compared to traditional solutions, this achieves dual optimization of network bandwidth and AI computing resources, significantly improving system resource utilization.

[0019] Optionally, step 2 includes the following sub-steps:

[0020] Step 21: The head office data center uses network resource 1 to obtain audio stream data from all call machines at all branches. Suppose there are N branches, corresponding to N call machines and N channels of audio stream data from call machines.

[0021] Step 22: The head office data center runs the counter number extraction algorithm on AI computing resource I to extract the counter number information from the N-channel call machine audio stream data.

[0022] Optionally, step 22 includes the following sub-steps:

[0023] Step 221: Using a speech recognition algorithm, transcribe the N-way call machine audio stream data into N text records;

[0024] Step 222: Use a natural language understanding algorithm to extract the counter number information of the customer from the N transcribed text records, and mark the extracted counter number as bank counter Y.

[0025] By adopting the above technical solution, the counter number information can be automatically and efficiently obtained, providing a basis for subsequent resource allocation; the speech recognition algorithm in step 222 can be open source FunASR, PaddleSpeech, etc.

[0026] Optionally, if bank counter Y that will handle business for customer X is extracted, steps 3 to 8 are performed on bank counter Y; if multiple counter number information is extracted at the same time, steps 3 to 8 are performed independently and in parallel on all the extracted counter number information; if no counter number information is extracted, go to step 9.

[0027] Optionally, step 5 includes the following sub-steps:

[0028] Step 51: Use a speech recognition algorithm to transcribe the audio in the bank counter Y audio and video stream into text in real time;

[0029] Step 52: Use a natural language understanding algorithm to extract the business type from the transcribed text, and mark the extracted business type as business type Z.

[0030] By adopting the above technical solution, automated extraction of business types is achieved, providing a basis for subsequent audits. The speech recognition algorithm in step 51 can be open-source algorithms such as FunASR and PaddleSpeech. In step 52, natural language understanding technology can extract the type of customer business from text records using regular expressions, UIE algorithms, or a large language model.

[0031] Optionally, in step 7, the service completion detection algorithm includes the following sub-steps:

[0032] Step 71: Use a speech recognition algorithm to transcribe the audio stream in the audio and video stream of bank counter Y into text, and use a natural language understanding algorithm to analyze the text to determine whether transaction Z has been completed;

[0033] Step 72: Use target detection and target tracking technology to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the transaction has been completed.

[0034] Step 73: The results of step 71 and step 72 are integrated to obtain a detection result of whether the service type Z is terminated.

[0035] By adopting the above technical solution, the business completion detection can be realized efficiently and accurately. The natural language understanding algorithm in step 71 can use the regular expression method to extract the information of the business completion from the text record, or use a large model to extract it.

[0036] Optionally, in step 1, the head office data center allocates network resources I and AI computing resources I after the bank branch starts operating.

[0037] By adopting the above technical solution, network resources and computing resources are allocated only after the bank branch starts business, avoiding the double waste of network resources and AI computing resources.

[0038] The business adaptive resource scheduling device of the bank counter video audit system is used to implement the business adaptive resource scheduling method of the bank counter video audit system. The adaptive resource scheduling device includes an initial resource allocation module, a counter number extraction module, a dynamic transmission channel module, a dynamic transmission channel module, a service etiquette audit module, a business type extraction module, a business type resource release module, a compliance audit and termination detection module, a resource release control module and a loop control module.

[0039] Optionally, the initial resource allocation module is configured to execute step 1: when the outlet begins operations, allocate network resources I and AI computing resources I;

[0040] The counter number extraction module is used to execute step 2: extract the counter number information through voice recognition and natural language understanding of the audio stream of the call machine;

[0041] The dynamic transmission channel module is used to execute step 3: allocate network resources II to bank counter Y and establish an audio and video stream transmission link;

[0042] The service etiquette audit module is used to execute step 4: allocate AI computing resources II-A to run the service etiquette detection algorithm;

[0043] The business type extraction module is used to execute step 5: allocate AI computing resources II-B and extract business type Z through real-time speech transcription and text parsing;

[0044] The service type resource release module is used to execute step 6: release AI computing resources II-B;

[0045] The compliance audit and termination detection module is used to execute step 7: allocate AI computing resources II-C to run the compliance algorithm and termination detection algorithm of business Z;

[0046] The resource release control module is used to execute step 8: releasing network resources II and AI computing resources II-A and II-C;

[0047] The loop control module is used to execute step 9: cyclically trigger step 2 or release initial resources according to the business status of the outlets.

[0048] Optionally, each module of the adaptive resource scheduling device is implemented based on a computer server in the head office data center.

[0049] In summary, the present invention includes at least the following beneficial technical effects:

[0050] The present invention can provide a business adaptive resource scheduling method and device for a bank counter video audit system. The head office data center collects the audio streams of the call machines at each branch in real time, extracts the counter numbers through voice recognition and natural language understanding, dynamically establishes the corresponding counter audio and video transmission channels, and allocates AI computing resources. Based on real-time voice transcription to identify the business type, only the compliance audit algorithm for the business is called, and the resources are released after the business is completed. This solution reduces network transmission redundancy through a real-time association mechanism between business needs and resource allocation; and avoids redundant computing loads through a dynamic mapping mechanism between business types and audit algorithms. Compared with traditional solutions, it achieves dual optimization of network bandwidth and AI computing resources, significantly improving system resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The present invention is a flow chart of a business adaptive resource scheduling method for a bank counter video audit system. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below with reference to the accompanying drawings.

[0053] The embodiments of the present invention disclose a method and device for adaptively scheduling business resources in a bank counter video audit system.

[0054] Reference Figure 1 The business adaptive resource scheduling method of the bank counter video audit system includes the following steps:

[0055] Step 1: When a bank branch opens in the morning, the head office data center allocates network resources I and AI computing resources I;

[0056] Step 2: The head office data center uses network resource I to obtain audio stream data from the queuing machines at all branches. The head office data center then runs a counter number extraction algorithm on AI computing resource I to extract the counter number from the audio stream data and marks the extracted counter number as bank counter Y.

[0057] Step 3: Allocate network resource II to establish a network link between the head office data center and bank counter Y obtained in step 2. The head office data center uses network resource II to collect audio and video stream data from bank counter Y in real time.

[0058] Step 4: Allocate AI computing resource II-A. The head office data center uses AI computing resource II-A to run a service etiquette audit algorithm to analyze the audio and video stream data from bank counter Y to check whether the teller's service process meets reception etiquette standards.

[0059] Step 5: Allocate AI computing resource II-B. The head office data center uses AI computing resource II-B to run a business type extraction algorithm to extract the business type from the audio and video stream of bank counter Y, obtaining business type Z.

[0060] Step 6: Release AI computing resources II-B;

[0061] Step 7: Allocate AI computing resources II-C. The head office data center uses AI computing resources II-C to run the compliance audit algorithm for business type Z. This algorithm checks the compliance of the audio and video streams at bank counter Y until the customer completes business type Z.

[0062] The head office data center uses AI computing resource II-C to run the transaction completion detection algorithm for transaction type Z. This algorithm detects whether the customer transaction process at bank counter Y has ended, as seen in the audio and video streams. If so, the algorithm stops step 7 and proceeds to step 8.

[0063] Step 8: Release network resources II and AI computing resources II-A and II-C;

[0064] Step 9: If the branch is still in business hours, return to step 2; if the branch has ended business for the day, the head office data center releases AI computing resources I and network resources I, and waits until business opens the next day before returning to step 1.

[0065] By linking business needs with resource allocation in real time, network transmission redundancy is reduced; by dynamically mapping business types with audit algorithms, redundant computing loads are avoided. Compared to traditional solutions, this achieves dual optimization of network bandwidth and AI computing resources, significantly improving system resource utilization.

[0066] Step 2 includes the following sub-steps:

[0067] Step 21: The head office data center uses network resource 1 to obtain audio stream data from all call machines at all branches. Suppose there are N branches, corresponding to N call machines and N channels of audio stream data from call machines.

[0068] Step 22: The head office data center runs the counter number extraction algorithm on AI computing resource I to extract the counter number information from the N-channel call machine audio stream data.

[0069] Step 22 includes the following sub-steps:

[0070] Step 221: Using a speech recognition algorithm, transcribe the N-way call machine audio stream data into N text records;

[0071] Step 222: Use a natural language understanding algorithm to extract the counter number information of the customer from the N transcribed text records, and mark the extracted counter number as bank counter Y.

[0072] The automatic and efficient acquisition of counter number information can be achieved, providing a basis for subsequent resource allocation; the speech recognition algorithm in step 222 can be open source FunASR, PaddleSpeech, etc.

[0073] If bank counter Y that will handle business for customer X is retrieved, then steps 3 to 8 are performed on bank counter Y. If multiple counter number information is retrieved at the same time, then steps 3 to 8 are performed in parallel on all the retrieved counter number information. If no counter number information is retrieved, then go to step 9.

[0074] Step 5 includes the following sub-steps:

[0075] Step 51: Use a speech recognition algorithm to transcribe the audio in the bank counter Y audio and video stream into text in real time;

[0076] Step 52: Use a natural language understanding algorithm to extract the business type from the transcribed text, and mark the extracted business type as business type Z.

[0077] Automatically extract the business type, providing a basis for subsequent audits. The speech recognition algorithm in step 51 can be open-source FunASR, PaddleSpeech, etc. In step 52, natural language understanding technology can use regular expressions to extract the type of business performed by the customer from the text record, or it can use the UIE algorithm.

[0078] In step 7, the service completion detection algorithm includes the following sub-steps:

[0079] Step 71: Use a speech recognition algorithm to transcribe the audio stream in the audio and video stream of bank counter Y into text, and use a natural language understanding algorithm to analyze the text to determine whether transaction Z has been completed;

[0080] Step 72: Use target detection and target tracking technology to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the transaction has been completed.

[0081] Step 73: The detection results of step 71 and step 72 are integrated to obtain a detection result of whether the service type Z is terminated.

[0082] The business completion detection can be realized efficiently and accurately. The natural language understanding algorithm in step 71 can extract the business completion information from the text record using a regular expression method, or it can use a large model to extract it.

[0083] In step 1, after the bank branch starts operating, the head office data center allocates network resources I and AI computing resources I.

[0084] Network resources and computing resources are allocated only after bank branches start operating, avoiding double waste of network resources and AI computing resources.

[0085] The business adaptive resource scheduling device of the bank counter video audit system is used to implement the business adaptive resource scheduling method of the bank counter video audit system. The adaptive resource scheduling device includes an initial resource allocation module, a counter number extraction module, a dynamic transmission channel module, a dynamic transmission channel module, a service etiquette audit module, a business type extraction module, a business type resource release module, a compliance audit and termination detection module, a resource release control module and a loop control module.

[0086] The initial resource allocation module is used to execute step 1: when the outlet starts operating, allocate network resources I and AI computing resources I;

[0087] The counter number extraction module is used to execute step 2: extract the counter number information through voice recognition and natural language understanding of the audio stream of the call machine;

[0088] The dynamic transmission channel module is used to execute step 3: allocate network resources II to bank counter Y and establish an audio and video stream transmission link;

[0089] The service etiquette audit module is used to execute step 4: allocate AI computing resources II-A to run the service etiquette detection algorithm;

[0090] The business type extraction module is used to execute step 5: allocate AI computing resources II-B and extract business type Z through real-time speech transcription and text parsing;

[0091] The service type resource release module is used to execute step 6: release AI computing resources II-B;

[0092] The compliance audit and termination detection module is used to execute step 7: allocate AI computing resources II-C to run the compliance algorithm and termination detection algorithm of business Z;

[0093] The resource release control module is used to execute step 8: releasing network resources II and AI computing resources II-A and II-C;

[0094] The loop control module is used to execute step 9: cyclically trigger step 2 or release initial resources according to the business status of the outlets.

[0095] Each module of the adaptive resource scheduling device is implemented based on the computer server of the head office data center.

[0096] The following uses a specific embodiment to illustrate the implementation principle of the business adaptive resource scheduling method and device of the bank counter video audit system:

[0097] A business adaptive resource scheduling method for a bank counter video audit system, such as Figure 1 The specific steps are as follows:

[0098] Step 1: After the branch opens in the morning, the head office data center allocates network resources I and AI computing resources I.

[0099] In one embodiment, a bank has 1,000 outlets. Each audio stream from the ticket-calling machine uses a 64kbps bandwidth, resulting in a total bandwidth requirement of 64kbps × 1,000 = 64Mbps. Network resource I is allocated as a 100Mbps dedicated channel (including redundancy). AI computing resource I is configured with 2 CPU cores and 2GB of memory for each audio stream from the ticket-calling machine, resulting in a total requirement of 2,000 CPU cores and 2,000GB of memory. A Kubernetes cluster is used to deploy 200 Pod instances (each Pod is allocated 10 cores and 10GB of memory).

[0100] In another embodiment, a provincial credit union sets all branches to open at 9:00, and the head office data center pre-allocates network resources I and AI computing resources I at 8:55 through a scheduled task.

[0101] In another embodiment, a national commercial bank dynamically allocates resources based on the local business hours of branches in each province. For example, network resource I and AI computing resource I are allocated to branch A at 8:55, and network resource I and AI computing resource I are allocated to branch B at 9:55.

[0102] Step 2: The head office data center retrieves the counter number information.

[0103] Step 21: The head office data center uses network resource I to obtain the audio stream data of the call machine of all branches. Suppose there are N branches, each branch has only one call machine, so there are N call machines and N channels of call machine audio stream data.

[0104] Step 22: The head office data center runs the counter number extraction algorithm on AI computing resource I.

[0105] Step 221: Use a speech recognition algorithm to transcribe the N-way call machine audio stream data into N text records. This is not limited to any speech recognition algorithm, such as Alibaba's open-source FunASR and PaddleSpeech.

[0106] Step 222: Use a natural language understanding algorithm to extract the counter number for the customer's upcoming transaction from the N transcribed text records. The text records contain branch service instructions, typically formatted as "Customer X, please go to bank counter Y." The natural language understanding algorithm used is not limited to a specific one; for example, regular expression algorithms, UIE (Universal Information Extraction) algorithms, or large language models can be used.

[0107] If the counter number information Y that will handle the business for customer X is extracted, then the bank counter Y is processed from step 3 to step 8. If multiple counter number information is extracted at the same time, all the extracted counter information needs to be processed from step 3 to step 8 in parallel.

[0108] If the counter number information is not retrieved, go to step 9.

[0109] In one embodiment, a general regular expression pattern is designed to match various typical statements based on the diverse voice broadcast formats of ticket machines at different outlets: "Please ask customer A102 to go to counter 5," "Please ask customer 23 to go to VIP counter 8," "Please ask customer 32 to go to counter 12," and "Please ask customer D110 with a gold card to go to room 2 in the VIP area." The Python code is as follows:

[0110] patterns = [

[0111] (r"Please ([A-Z0-9]+) number.*? Go to counter (\d+)", 2),

[0112] (r"(?: Arrive|Please go to)(\d+)[Number]?(?: Counter|VIP Counter|Counter)", 1),

[0113] (r"(directly)? Go to counter (\d{2,3}), 2),

[0114] (r"VIP [Area](\d+)", 1) ]

[0116] def extract_counter(text):

[0117] for pattern, group_idx in patterns:

[0118] match = re.search(pattern, text)

[0119] if match:

[0120] return match.group(group_idx)

[0121] return None

[0122] examples = [

[0123] 'Please ask customer A102 to go to counter 5'

[0124] 'Customer No. 23, please go to VIP counter No. 8 to handle your business',

[0125] 'Golden Sunflower 32 customers please go to counter 12 to handle your business',

[0126] "Please ask Gold Card customer D110 to go to VIP area room 2." ]

[0128] for text in examples:

[0129] counter = extract_counter(text)

[0130] print(f"{text}: {counter}");

[0131] The result of running the above code is:

[0132] Please go to counter 5 for customer A102:

[0133] Customer No. 23 please go to VIP counter No. 8 to handle business: 8

[0134] Please go to counter 12 for business at Sunflower 32:

[0135] Please go to VIP area room 2 at D110 for gold card customers:

[0136] In another example, the bank has a large number of foreign customers, so English announcements are used. Typical English announcements include: "Customer A102, please proceed to Counter 5 for service.", "Customer 23, please go to VIP Counter 8 for your transaction.", "Golden Sun flower customer 32, please approach Counter 12 for assistance.", and "Gold Card member D110, kindly visit Room 2 in the VIP area." To ensure accurate counter number extraction in multiple languages, a large language model is introduced to extract the counter number. In this case, a feasible prompt word is: "Please extract the counter number in the following description: xxxx."

[0137] In another embodiment, the software associated with the queuing machine has been optimized and modified so that when the queuing machine broadcasts the call information, it can also directly transmit the counter number information to the head office data center. This improvement allows steps 1 and 2 to be skipped in the business process. In other words, the head office data center no longer needs to extract the counter number information from the audio stream itself, but can directly receive the accurate counter number information sent by the queuing machine. This optimization not only simplifies the data processing process but also improves the accuracy and real-time performance of counter number recognition, further enhancing the system's responsiveness and service quality.

[0138] Step 3: Allocate network resources II to establish a network link between the head office data center and bank counter Y. The head office data center uses network resources II to collect audio and video stream data from bank counter Y in real time.

[0139] Step 4: Allocate AI computing resource II-A. The head office data center uses AI computing resource II-A to run the service etiquette audit algorithm to check whether the business process handled by teller Y (i.e., the teller at bank counter Y) complies with reception etiquette standards.

[0140] In one embodiment, to verify whether a teller is performing a hand salute, the system uses the OpenPose model to detect key points on the teller's body. Based on this key point data, a classifier pre-trained with the ResNet50 network is used to determine whether the teller has performed a hand salute.

[0141] In another example, to verify whether a teller smiles when providing customer service, the system first uses the YOLO (You Only Look Once) model to detect the teller's face. Once the face is detected, the system extracts the facial image and uses a classifier trained with a ResNet50 network to determine whether the teller smiles.

[0142] Step 5: Allocate AI computing resources II-B. The head office data center uses AI computing resources II-B to run the business type extraction algorithm.

[0143] Step 51: Use a speech recognition algorithm to transcribe the audio from the bank counter Y video stream into text in real time. This can be done using any speech recognition technology, such as Alibaba's open-source FunASR or Baidu's open-source PaddleSpeech.

[0144] In one embodiment, considering that bank branches are located in areas with high Mandarin penetration rates (such as first-tier cities), tellers and customers primarily communicate in standard Mandarin. In this case, the Mandarin speech recognition algorithm provided by the FunASR system performs audio transcription.

[0145] In another embodiment, for scenarios where bank branches are located in areas with a rich dialect diversity or low Mandarin penetration, where tellers may have a distinct local accent, the system employs the following optimization scheme to achieve high-precision speech recognition: First, data covering a variety of conversational scenarios is collected in the target dialect, and data augmentation techniques such as speech rate variation and background noise overlay are used to improve model robustness. Second, a speech recognition model based on the Transformer architecture is constructed, incorporating a CNN (convolutional neural network) and a self-attention mechanism to effectively capture local speech features and global contextual information, enhancing understanding of different accents. Furthermore, a dialect-specific language model is introduced to optimize understanding of grammatical structure and semantics, resulting in an end-to-end automatic speech recognition (ASR) system that predicts text directly from audio, streamlining the recognition process. Finally, a post-processing module is deployed to correct common recognition errors through rule matching and statistical learning methods. The model is continuously updated iteratively based on user feedback to ensure continuous optimization and adaptability of system performance.

[0146] Step 52: Use a natural language understanding algorithm to extract the business type from the transcribed text. Let the extracted business type be Z. The natural language understanding technology used is not limited to one method. For example, a regular expression method or a UIE algorithm can be used to extract the type of business performed by the customer from the text record.

[0147] In one embodiment, a regular expression is used to extract the business type. The following is the Python code for this implementation:

[0148] import re

[0149] #Define business types and corresponding regular expressions

[0150] business_patterns={

[0151] "Deposit": r"(deposit|deposit|deposit)\s*\d+yuan",

[0152] "Withdraw": r"(withdraw|withdraw|withdraw)\s*\d+yuan",

[0153] "transfer":r"(transfer|remittance|transfer out)\s*\d+yuan",

[0154] "Loan":r"(loan|borrowing|mortgage)\s*",

[0155] "Foreign Exchange": r" (Exchange|Purchase|Settlement)\s*\d+USD"

[0156] }

[0157] defextract_business_type(text):

[0158] for biz_type,pattern in business_patterns.items():

[0159] match=re.search(pattern,text)

[0160] if match:

[0161] return biz_type

[0162] return "other"

[0163] #Test example

[0164] texts=[

[0165] "I want to deposit 5,000 yuan into my current account.",

[0166] "Please help me withdraw 2,000 yuan from my account.",

[0167] "I want to transfer 10,000 yuan to Zhang San's account."

[0168] "I would like to apply for a housing loan of 200,000 yuan."

[0169] "I want to exchange $500." ]

[0171] for text in texts:

[0172] print(f"Input: {text}\nOutput: {extract_business_type(text)}\n")

[0173] The result of running the above code is:

[0174] Input: I want to deposit 5,000 yuan into my checking account.

[0175] Output: Deposit

[0176] Input: Please help me withdraw 2000 yuan from my account.

[0177] Output: Withdrawal

[0178] Input: I want to transfer 10,000 yuan to Zhang San’s account.

[0179] Output: Transfer

[0180] Input: I would like to apply for a housing loan of $200,000.

[0181] Output: Loans

[0182] Input: I want to exchange $500.

[0183] Output: Foreign exchange

[0184] Step 6: Release AI computing resources II-B.

[0185] In one embodiment, the software used to process transactions at branch counters has been modified so that transaction type information is instantly transmitted to the head office data center upon initiating customer transactions. This optimization allows steps 5 and 6 of the transaction process to be skipped. In other words, the head office data center no longer needs to extract and analyze transaction types on its own; instead, it can directly receive the exact transaction type information sent by the branch counters. This improvement not only simplifies the data processing process but also improves the accuracy and response speed of transaction type identification, further enhancing the efficiency and reliability of the system. This adjustment ensures that resources can be more accurately allocated according to actual business needs, achieving more flexible and efficient system operation.

[0186] Step 7: Allocate AI computing resources II-C. The head office data center uses AI computing resources II-C to run the compliance audit algorithm and business completion detection algorithm for business type Z.

[0187] Step 71: Allocate AI computing resources II-C.

[0188] Step 72: Run the compliance audit algorithm for business type Z. This algorithm is used to check whether the process of teller Y handling business Z for customer X is compliant. The input of the algorithm is the audio and video stream of bank counter Y.

[0189] In a specific embodiment, different compliance audit algorithms need to be developed for compliance requirements of different business types, which may include:

[0190] 1. Basic funding services, including current / fixed deposits, cash deposits and withdrawals, lump-sum deposits, fixed-term and flexible deposits, and other savings services, as well as personal / corporate loan applications, mortgage / car loan / consumer loan approval and issuance, and other credit services.

[0191] 2. Payment and settlement services, including interbank and interbank transfers, real-time remittances, check / draft / promissory note cashing and reporting loss, bank-to-securities transfers, and cross-border payment and settlement services. Some banks also offer precious metals trading, foreign exchange, and cross-border remittance services.

[0192] 3. Account management services involve the full life cycle management of personal / corporate accounts, including opening, changing, closing, reporting loss and unreporting, password reset, etc., as well as the activation and maintenance of electronic banking (online banking / mobile banking).

[0193] 4. Special financial services, including agency services (collection of utility bills, insurance / fund sales, and payroll), wealth management services (fund / wealth management product subscriptions and asset portfolio allocation), bill services (acceptance / discounting), and customized products such as education savings and certificate of deposit pledge loans.

[0194] 5. Corporate banking: We offer services such as checking account management, cash vault management, trade financing (letters of credit / guarantees), and corporate settlement (collection / letter of credit processing) to corporate customers. Some banks also support corporate loans and commercial property mortgages.

[0195] If step 52 identifies the transaction as a large-value cash transaction, the system initiates a large-value cash transaction compliance audit algorithm. This involves using a YOLOv5 model trained specifically for cash counters and a classifier trained with MobileNet to identify the cash counter's usage status; employing the HRNet hand keypoint detection algorithm to track the teller's hand contact with cash to determine whether the teller has prolonged contact with cash; and using the OpenPose pose estimation algorithm to analyze the presence and duration of two-person interaction within the authorization area. This series of operations ensures comprehensive auditing, from customer identity verification and device operating specifications to authorization process compliance.

[0196] If step 52 determines the transaction type is a cash drawer-tampering scenario, the system initiates a cash drawer-tampering compliance audit algorithm. This includes using the YOLOv5 model to detect the cash drawer and combining it with a classifier trained on MobileNet to determine its open or closed state. Similarly, the YOLOv5 model and MobileNet classifier are used to monitor cash counter usage. The HRNet hand keypoint detection algorithm is also used to track cash checkers' contact with cash in real time. Furthermore, a time series feature fusion algorithm is used to assess the integrity of the entire operational process, ensuring compliance with cash drawer-tampering management standards at all stages, from cash drawer opening and equipment use to cash handling.

[0197] Step 73: While step 72 is running, the service Z completion detection algorithm is running. This algorithm uses the audio and video streams from bank counter Y as input. It detects whether bank counter Y has completed service Z for customer X. If so, step 7 is stopped and the process proceeds to step 8. If not, steps 72 and 73 continue.

[0198] The business completion detection algorithm integrates the audio information and video information in the counter audio and video stream.

[0199] Step 731: Use a speech recognition algorithm to transcribe the audio stream from bank counter Y into text. Then, use a natural language understanding algorithm to analyze the text to determine whether transaction Z has been completed. The natural language understanding technology used is not limited to specific techniques. For example, regular expressions can be used to extract transaction completion information from the text record, or large models can be used.

[0200] Step 732: Use target detection and target tracking technology to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the business has been completed.

[0201] In one embodiment, the counter is located in a relatively private VIP reception room with a single staff. First, a static counter service area (ROI) is defined within the video feed as the effective area for customer interaction. A YOLOv8 model is used to detect the customer's body and dynamically analyze their positional changes. If the system continuously tracks a customer outside the ROI for three seconds, the customer is marked as away from the counter, indicating that the transaction has concluded.

[0202] In another embodiment, the counter is located in a bank lobby with high customer traffic. Similarly, the counter service area (ROI) is first statically set in the video image as the range of effective customer interaction. The YOLOv8 model is used to detect the customer's body and dynamically analyze its position changes. At the same time, a classifier trained with ResNet18 is used to identify the customer's orientation (facing or facing away from the counter). When the system continuously tracks and finds that a customer has not appeared in the ROI area within 3 seconds and is facing away from the counter, the customer will be marked as leaving the counter and the business will be deemed completed. In addition, based on the density of customer traffic, the system can adaptively adjust the judgment threshold, for example, extending the leaving the counter judgment time to 5 seconds during peak hours, thereby enhancing the detection robustness and accuracy in complex environments.

[0203] Step 733: The detection results of step 731 and step 732 are integrated to obtain the detection result of whether the service Z is terminated.

[0204] In one embodiment, when the audio and video stream quality is high (audio signal-to-noise ratio ≥ 30dB and video resolution ≥ 720P), the system uses a strict double confirmation mechanism. Only when both steps 731 and 732 determine that the service is completed will the service be finally confirmed as completed.

[0205] In another embodiment, if the audio and video stream quality is low (audio signal-to-noise ratio <30dB or video resolution <720P), the tolerance mechanism is activated. In this case, as long as either step 731 or step 732 determines that the service is completed, the service is confirmed to be completed.

[0206] In another embodiment, the system introduces a timeout mechanism. Specifically, if the processing time of service Z exceeds a predetermined time limit, or a new customer arrives at counter Y, or the ticket machine calls a new customer to go to counter Y for service, the system will automatically determine that the current service is terminated.

[0207] In one embodiment, the software used to process transactions at branch counters has been optimized and modified so that when a teller completes transaction Z, the end-of-transaction information is immediately transmitted to the head office data center. This improvement allows step 73 of the transaction process to be skipped. In other words, the head office data center no longer needs to rely on audio and video stream analysis to determine whether transaction Z has been completed. Instead, it can directly receive confirmation of the completion of transaction Z from the branch counter. This optimization not only simplifies the transaction completion detection process but also improves the accuracy and timeliness of the judgment, further enhancing the system's responsiveness and efficiency.

[0208] Step 8: Release network resources II and AI computing resources II-A and II-C.

[0209] Step 9: If the outlet is still in business hours, return to step 2; if the outlet has ended business for the day, release AI computing resources I and network resources I.

[0210] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A business adaptive resource scheduling method for a bank counter video audit system, characterized in that: The following steps are involved: Step 1: When a bank branch opens in the morning, the head office data center allocates network resources I and AI computing resources I; Step 2: The head office data center uses network resource I to obtain audio stream data from the queuing machines at all branches. The head office data center then runs a counter number extraction algorithm on AI computing resource I to extract the counter number from the audio stream data and marks the extracted counter number as bank counter Y. Step 3: Allocate network resource II to establish a network link between the head office data center and bank counter Y obtained in step 2. The head office data center uses network resource II to collect audio and video stream data from bank counter Y in real time. Step 4: Allocate AI computing resource II-A. The head office data center uses AI computing resource II-A to run a service etiquette audit algorithm to analyze the audio and video stream data from bank counter Y to check whether the teller's service process meets reception etiquette standards. Step 5: Allocate AI computing resource II-B. The head office data center uses AI computing resource II-B to run a business type extraction algorithm to extract the business type from the audio and video stream of bank counter Y, obtaining business type Z. Step 6: Release AI computing resources II-B; Step 7: Allocate AI computing resources II-C. The head office data center uses AI computing resources II-C to run the compliance audit algorithm for business type Z. This algorithm checks the compliance of the audio and video streams at bank counter Y until the customer completes business type Z. The head office data center uses AI computing resource II-C to run the transaction completion detection algorithm for transaction type Z. This algorithm detects whether the customer transaction process at bank counter Y has ended, as seen in the audio and video streams. If so, the algorithm stops step 7 and proceeds to step 8. Step 8: Release network resources II and AI computing resources II-A and II-C; Step 9: If the branch is still in business hours, return to step 2; if the branch has ended business for the day, the head office data center releases AI computing resources I and network resources I, and waits until business opens the next day before returning to step 1.

2. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: Step 2 includes the following sub-steps: Step 21: The head office data center uses network resource 1 to obtain audio stream data from all call machines at all branches. Suppose there are N branches, corresponding to N call machines and N channels of audio stream data from call machines. Step 22: The head office data center runs the counter number extraction algorithm on AI computing resource I to extract the counter number information from the N-channel call machine audio stream data.

3. The business adaptive resource scheduling method of the bank counter video audit system according to claim 2 is characterized in that: Step 22 includes the following sub-steps: Step 221: Using a speech recognition algorithm, transcribe the N-way call machine audio stream data into N text records; Step 222: Use a natural language understanding algorithm to extract the counter number information of the customer from the N transcribed text records, and mark the extracted counter number as bank counter Y.

4. The business adaptive resource scheduling method of the bank counter video audit system according to claim 3 is characterized in that: If bank counter Y that will handle business for customer X is retrieved, then steps 3 to 8 are performed on bank counter Y. If multiple counter number information is retrieved at the same time, then steps 3 to 8 are performed independently and in parallel on all the retrieved counter number information. If no counter number information is retrieved, then go to step 9.

5. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: Step 5 includes the following sub-steps: Step 51: Use a speech recognition algorithm to transcribe the audio in the bank counter Y audio and video stream into text in real time; Step 52: Use a natural language understanding algorithm to extract the business type from the transcribed text, and mark the extracted business type as business type Z.

6. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: In step 7, the service completion detection algorithm includes the following sub-steps: Step 71: Use a speech recognition algorithm to transcribe the audio stream in the bank counter Y audio and video stream into text, and use a natural language understanding algorithm to analyze the text to determine whether transaction Z has been completed; Step 72: Use target detection and target tracking technology to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the transaction has been completed. Step 73: The results of step 71 and step 72 are integrated to obtain a detection result of whether the service type Z is terminated.

7. The business adaptive resource scheduling method for the bank counter video audit system according to claim 1 is characterized in that: In step 1, after the bank branch starts operating, the head office data center allocates network resources I and AI computing resources I.

8. A business adaptive resource scheduling device for a bank counter video audit system, characterized by: A method for implementing the business adaptive resource scheduling of a bank counter video audit system as described in any one of claims 1 to 7, wherein the adaptive resource scheduling device includes an initial resource allocation module, a counter number extraction module, a dynamic transmission channel module, a service etiquette audit module, a business type extraction module, a business type resource release module, a compliance audit and termination detection module, a resource release control module, and a loop control module.

9. The service adaptive resource scheduling device for the bank counter video audit system according to claim 8, characterized in that: The initial resource allocation module is used to execute step 1: when the outlet starts operating, allocate network resources I and AI computing resources I; The counter number extraction module is used to execute step 2: extract the counter number information through voice recognition and natural language understanding of the audio stream of the call machine; The dynamic transmission channel module is used to execute step 3: allocate network resources II to bank counter Y and establish an audio and video stream transmission link; The service etiquette audit module is used to execute step 4: allocate AI computing resources II-A to run the service etiquette detection algorithm; The business type extraction module is used to execute step 5: allocate AI computing resources II-B and extract business type Z through real-time speech transcription and text parsing; The service type resource release module is used to execute step 6: release AI computing resources II-B; The compliance audit and termination detection module is used to execute step 7: allocate AI computing resources II-C to run the compliance algorithm and termination detection algorithm of business Z; The resource release control module is used to execute step 8: releasing network resources II and AI computing resources II-A and II-C; The loop control module is used to execute step 9: cyclically trigger step 2 or release initial resources according to the business status of the outlets.

10. The business adaptive resource scheduling device of the bank counter video audit system according to claim 9, characterized in that: Each module of the adaptive resource scheduling device is implemented based on the computer server of the head office data center.

Images