Business adaptive resource scheduling method and device of bank counter video auditing system

Abstract

The invention discloses a service adaptive resource scheduling method and device of a bank counter video auditing system, and relates to the technical field of video auditing. A head office data center collects audio streams of number calling machines of all network points in real time, and counter numbers are extracted through voice recognition and natural language understanding; dynamically establishing a corresponding counter audio and video transmission channel and allocating AI computing resources; a service type is extracted based on real-time speech recognition and a natural language understanding algorithm, only a compliance auditing algorithm of the service is called, and resources are released after the service is finished. According to the scheme, network transmission redundancy is reduced through a real-time association mechanism of service requirements and resource allocation; through a dynamic mapping mechanism of a service type and an auditing algorithm, redundant calculation load is avoided. Compared with a traditional scheme, double optimization of the network bandwidth and the AI computing resources is achieved, and the utilization rate of system resources is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video auditing, and in particular to a method and device for business adaptive resource scheduling of a bank counter video auditing system. Background Art

[0002] The bank counter video auditing system realizes full-process monitoring through AI video recognition technology, constructs a "real-time monitoring and tracing" risk control system, accurately captures abnormal transactions and violations, and simultaneously generates an intelligent auditing report to reduce operational risks. Existing bank counter video auditing systems usually adopt a centralized deployment scheme, that is, a high-performance AI server cluster is centrally deployed in the head office data center, and each branch transmits video streams to the head office data center for auditing in real time through a wide area network. Although this mode can ensure centralized scheduling of AI computing resources, there is a problem of continuous occupation of network bandwidth. In-depth analysis reveals that there are two-fold resource wastes in this architecture.

[0003] 1) In the time series dimension, even during the 8-hour business hours of each day but in periods when no customers are handling business (such as when there are no customers in the branch or at a certain counter), the system still continuously transmits video streams and invokes AI computing resources to execute auditing algorithms, resulting in double wastes of network resources and AI computing resources.

[0004] 2) In the business dimension, no dynamic mapping mechanism between business types and auditing rules is established, and all auditing algorithms are mechanically loaded for all business scenarios, generating unnecessary computing loads, thereby resulting in wastes of AI computing resources.

[0005] In view of the above situation, it is urgent to design a method and device for business adaptive resource scheduling of a bank counter video auditing system to solve the above two-fold resource waste problems. Summary of the Invention

[0006] To solve the above technical problems, the present invention provides a method and device for business adaptive resource scheduling of a bank counter video auditing system. The following technical solutions are adopted: A method for business adaptive resource scheduling of a bank counter video auditing system includes the following steps: Step 1: When a bank branch starts business in the morning, the head office data center allocates network resource I and AI computing resource I; Step 2: The head office data center uses network resource I to obtain the audio stream data of the queuing machines of all branches. The head office data center runs a counter number extraction algorithm on AI computing resource I to extract the counter number information in the audio stream data of the queuing machines, and marks the extracted counter number as bank counter Y; Step 3: Allocate Network Resource II to establish a network link between the head office data center and Bank Counter Y obtained in Step 2. The head office data center uses Network Resource II to collect the audio and video stream data of Bank Counter Y in real time; Step 4: Allocate AI Computing Resource II-A. The head office data center uses AI Computing Resource II-A to run the service etiquette auditing algorithm to analyze the audio and video stream data of Bank Counter Y, and check whether the process of the teller handling business at Bank Counter Y complies with the reception etiquette standards; Step 5: Allocate AI Computing Resource II-B. The head office data center uses AI Computing Resource II-B to run the business type extraction algorithm to extract the business type from the audio and video stream of Bank Counter Y, obtaining Business Type Z, until the process of handling Business Z for the customer ends; Step 6: Release AI Computing Resource II-B; Step 7: Allocate AI Computing Resource II-C. The head office data center uses AI Computing Resource II-C to run the compliance auditing algorithm for Business Type Z to detect whether the process in the audio and video stream of Bank Counter Y is compliant; The head office data center uses AI Computing Resource II-C to run the business handling end detection algorithm for Business Type Z to detect whether the process of handling business for the customer in the audio and video stream of Bank Counter Y has ended. If it has ended, stop Step 7 and proceed to Step 8; Step 8: Release Network Resource II, AI Computing Resources II-A and II-C; Step 9: If the branch is still within the business hours, go back to Step 2; if the branch has ended its business for the day, the head office data center releases AI Computing Resource I and Network Resource I, and when the business starts the next day, go to Step 1.

[0007] By adopting the above technical solution, through the real-time association mechanism between business requirements and resource allocation, network transmission redundancy is reduced; through the dynamic mapping mechanism between business types and auditing algorithms, redundant computing loads are avoided. Compared with the traditional solution, dual optimization of network bandwidth and AI computing resources is achieved, significantly improving the utilization rate of system resources.

[0008] Optionally, Step 2 includes the following sub-steps: Step 21: The head office data center uses Network Resource I to obtain the audio stream data of the queuing machines of all branches. There are N branches, corresponding to N queuing machines and N channels of queuing machine audio stream data; Step 22: The head office data center runs the counter number extraction algorithm on AI Computing Resource I to extract the counter number information from the N channels of queuing machine audio stream data respectively.

[0009] Optionally, Step 22 includes the following sub-steps: Step 221: Use a speech recognition algorithm to transcribe the audio stream data of N queuing machines into N text records; Step 222: Use a natural language understanding algorithm to extract the counter number information for which business will be handled for the customer from the N transcribed text records, and mark the extracted counter number as bank counter Y.

[0010] By adopting the above technical solution, automatic and efficient acquisition of counter number information can be achieved, providing a basis for subsequent resource allocation; the speech recognition algorithms in Step 222 can be open-source ones such as FunASR and PaddleSpeech.

[0011] Optionally, if bank counter Y for which business will be handled for customer X is extracted, then perform the processing of Steps 3 - 8 on bank counter Y; if multiple counter number information is extracted simultaneously, then independently perform the processing of Steps 3 - 8 on all the extracted counter number information in parallel; if no counter number information is extracted, then go to Step 9.

[0012] Optionally, Step 5 includes the following sub-steps: Step 51: Use a speech recognition algorithm to transcribe the audio in the audio-video stream of bank counter Y into text in real time; Step 52: Use a natural language understanding algorithm to extract the business type from the transcribed text, and mark the extracted business type as business type Z.

[0013] By adopting the above technical solution, automatic extraction of the business type is achieved, providing a basis for subsequent audit and inspection; the speech recognition algorithms in Step 51 can be open-source ones such as FunASR and PaddleSpeech. The natural language understanding technology in Step 52 can extract the type of business handled by the customer from the text record by using the method of regular expressions, or can be extracted by using the UIE algorithm, or by using a large language model.

[0014] Optionally, in Step 7, the business handling end detection algorithm includes the following sub-steps: Step 71: Use a speech recognition algorithm to transcribe the audio stream in the audio-video stream of bank counter Y into text, and use a natural language understanding algorithm to analyze the text to determine whether business Z has been completed; Step 72: Adopt object detection and object tracking technologies to detect whether the customer has left the counter. If the customer has left the counter, then determine that the business has been completed; Step 73: Integrate the results of Step 71 and Step 72 to obtain the detection result of whether business type Z has ended.

[0015] By adopting the above technical solutions, the end detection of business processing can be efficiently and accurately achieved. In step 71, the natural language understanding algorithm can extract the information of the end of business processing from the text record by using regular expressions, or can also be extracted by using a large model.

[0016] Optionally, in step 1, after the bank branch starts business, the head office data center allocates network resource I and AI computing resource I.

[0017] By adopting the above technical solutions, the allocation of network resources and computing resources starts only after the bank branch starts business, avoiding double waste of network resources and AI computing resources.

[0018] The business adaptive resource scheduling device of the bank counter video auditing system is used to implement the business adaptive resource scheduling method of the bank counter video auditing system. The adaptive resource scheduling device includes an initial resource allocation module, a counter number extraction module, a dynamic transmission channel module, a dynamic transmission channel module, a service etiquette auditing module, a business type extraction module, a business type resource release module, a compliance auditing and end detection module, a resource release control module, and a loop control module.

[0019] Optionally, the initial resource allocation module is used to execute step 1; when the branch starts business, it allocates network resource I and AI computing resource I; The counter number extraction module is used to execute step 2; it extracts the counter number information through voice recognition and natural language understanding of the queuing machine audio stream; The dynamic transmission channel module is used to execute step 3; it allocates network resource II for bank counter Y and establishes an audio and video stream transmission link; The service etiquette auditing module is used to execute step 4; it allocates AI computing resource II-A to run the service etiquette detection algorithm; The business type extraction module is used to execute step 5; it allocates AI computing resource II-B and extracts business type Z through real-time speech transcription and text parsing; The business type resource release module is used to execute step 6; it releases AI computing resource II-B; The compliance auditing and end detection module is used to execute step 7; it allocates AI computing resource II-C to run the compliance algorithm and end detection algorithm of business Z; The resource release control module is used to execute step 8: release network resource II, AI computing resources II-A and II-C; The loop control module is used to execute step 9: trigger step 2 or release the initial resources in a loop according to the business status of the branch.

[0020] Optionally, each module of the adaptive resource scheduling device is implemented based on the computer servers of the head office data center.

[0021] In summary, the present invention includes at least the following beneficial technical effects: The present invention can provide a method and device for business-adaptive resource scheduling of a bank counter video auditing system. The head office data center collects the audio streams of the queuing machines at each branch in real time, extracts the counter numbers through speech recognition and natural language understanding, dynamically establishes corresponding audio and video transmission channels for the counters and allocates AI computing resources; based on real-time speech-to-text recognition of the business type, only the compliance auditing algorithms for that business are invoked, and the resources are released after the business ends. This solution reduces network transmission redundancy through a real-time association mechanism between business requirements and resource allocation; and avoids redundant computing loads through a dynamic mapping mechanism between business types and auditing algorithms. Compared with traditional solutions, it realizes the dual optimization of network bandwidth and AI computing resources, and significantly improves the utilization rate of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic flowchart of the method for business-adaptive resource scheduling of the bank counter video auditing system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The present invention will be further described in detail below with reference to the accompanying drawings.

[0024] The embodiments of the present invention disclose a method and device for business-adaptive resource scheduling of a bank counter video auditing system.

[0025] Referring to Figure 1 , the method for business-adaptive resource scheduling of a bank counter video auditing system includes the following steps: Step 1: When the branches start business in the morning, the head office data center allocates network resource I and AI computing resource I; Step 2: The head office data center uses network resource I to obtain the audio stream data of the queuing machines at all branches. The head office data center runs a counter number extraction algorithm on AI computing resource I to extract the counter number information in the audio stream data of the queuing machines, and marks the extracted counter number as bank counter Y; Step 3: Allocate network resource II for establishing a network connection between the head office data center and bank counter Y obtained in Step 2. The head office data center uses network resource II to collect the audio and video stream data of bank counter Y in real time; Step 4: Allocate AI computing resource II-A. The head office data center runs a service etiquette auditing algorithm on AI computing resource II-A to analyze the audio and video stream data of bank counter Y and check whether the process of the teller handling business at bank counter Y meets the reception etiquette standards; Step 5: Allocate AI computing resource II-B. The head office data center uses AI computing resource II-B to run the service type extraction algorithm to extract the service type from the Y audio-video stream of bank counter, and obtain service type Z. Step 6: Release AI computing resource II-B. Step 7: Allocate AI computing resource II-C. The head office data center uses AI computing resource II-C to run the compliance auditing algorithm for service type Z to detect whether the process in the audio-video stream of bank counter Y is compliant until the process of handling service Z for the customer ends. The head office data center uses AI computing resource II-C to run the service handling end detection algorithm for service type Z to detect whether the process of handling business for the customer in the audio-video stream of bank counter Y ends. If it ends, stop Step 7 and enter Step 8. Step 8: Release network resource II, AI computing resources II-A and II-C. Step 9: If the branch is still in the business hours, go back to Step 2; if the branch has ended the business for the day, the head office data center releases AI computing resource I and network resource I. Wait until the business starts the next day and go to Step 1.

[0026] Through the real-time association mechanism between business requirements and resource allocation, reduce network transmission redundancy; through the dynamic mapping mechanism between service types and auditing algorithms, avoid redundant computing loads. Compared with the traditional solution, it realizes the dual optimization of network bandwidth and AI computing resources, and significantly improves the system resource utilization rate.

[0027] Step 2 includes the following sub-steps: Step 21: The head office data center uses network resource I to obtain the audio stream data of the queuing machines of all branches. Suppose there are N branches, corresponding to N queuing machines and N audio stream data of queuing machines. Step 22: The head office data center runs the counter number extraction algorithm on AI computing resource I to extract the counter number information from the N audio stream data of queuing machines respectively.

[0028] Step 22 includes the following sub-steps: Step 221: Use the speech recognition algorithm to transcribe the N audio stream data of queuing machines into N text records. Step 222: Use the natural language understanding algorithm to extract the counter number information of the counter where the business will be handled for the customer from the N transcribed text records, and mark the extracted counter number as bank counter Y.

[0029] It can realize the efficient and automated acquisition of counter number information, providing a basis for subsequent resource allocation; the speech recognition algorithm in Step 222 can be open-source ones such as FunASR and PaddleSpeech.

[0030] If the bank counter Y for handling business for customer No. X is extracted, then perform the processing of steps 3 to 8 on bank counter Y; if multiple counter number information is extracted simultaneously, perform the processing of steps 3 to 8 on all the extracted counter number information in parallel; if no counter number information is extracted, go to step 9.

[0031] Step 5 includes the following sub-steps: Step 51: Use the speech recognition algorithm to transcribe the audio in the audio-video stream of bank counter Y into text in real time; Step 52: Use the natural language understanding algorithm to extract the business type from the transcribed text, and mark the extracted business type as business type Z.

[0032] Realize the automatic extraction of the business type, providing a basis for subsequent audit and inspection; the speech recognition algorithm in step 51 can be open-source FunASR, PaddleSpeech, etc. The natural language understanding technology in step 52 can use regular expressions to extract the type of business handled by the customer from the text record, or can use the UIE algorithm to extract.

[0033] In step 7, the business handling end detection algorithm includes the following sub-steps: Step 71: Use the speech recognition algorithm to transcribe the audio stream in the audio-video stream of bank counter Y into text, and use the natural language understanding algorithm to analyze the text to determine whether business Z has been completed; Step 72: Use object detection and object tracking technologies to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the business has been completed; Step 73: Integrate the detection results of step 71 and step 72 to obtain the detection result of whether business type Z has ended.

[0034] It can efficiently and accurately realize the detection of the end of business handling. The natural language understanding algorithm in step 71 can use regular expressions to extract the information on the end of business handling from the text record, or can use a large model to extract.

[0035] In step 1, after the bank branch starts business, the head office data center allocates network resource I and AI computing resource I.

[0036] The allocation of network resources and computing resources starts only after the bank branch starts business, avoiding double waste of network resources and AI computing resources.

[0037] A business adaptive resource scheduling device for a bank counter video auditing system, which is used to implement the business adaptive resource scheduling method of the bank counter video auditing system. The adaptive resource scheduling device includes an initial resource allocation module, a counter number extraction module, a dynamic transmission channel module, a service etiquette auditing module, a business type extraction module, a business type resource release module, a compliance auditing and end detection module, a resource release control module, and a loop control module.

[0038] The initial resource allocation module is used to execute Step 1; when the network point starts business, allocate network resource I and AI computing resource I; The counter number extraction module is used to execute Step 2; extract the counter number information through speech recognition and natural language understanding of the call machine audio stream; The dynamic transmission channel module is used to execute Step 3; allocate network resource II for bank counter Y and establish an audio and video stream transmission link; The service etiquette auditing module is used to execute Step 4; allocate AI computing resource II-A to run the service etiquette detection algorithm; The business type extraction module is used to execute Step 5; allocate AI computing resource II-B and extract business type Z through real-time speech transcription and text parsing; The business type resource release module is used to execute Step 6; release AI computing resource II-B; The compliance auditing and end detection module is used to execute Step 7; allocate AI computing resource II-C to run the compliance algorithm and end detection algorithm for business Z; The resource release control module is used to execute Step 8: release network resource II, AI computing resources II-A and II-C; The loop control module is used to execute Step 9: loop-trigger Step 2 or release the initial resources according to the business status of the network point.

[0039] Each module of the adaptive resource scheduling device is implemented based on the computer server of the head office data center.

[0040] The following uses specific embodiments to illustrate the implementation principles of the business adaptive resource scheduling method and device of the bank counter video auditing system: A business adaptive resource scheduling method for a bank counter video auditing system, as Figure 1 shown, the specific steps are as follows: Step 1: After the network point starts business in the morning, the head office data center allocates network resource I and AI computing resource I.

[0041] In one embodiment, a certain row has 1000 network points. Each queuing machine audio stream uses a bandwidth of 64 kbps, and the total bandwidth requirement is 64 kbps × 1000 = 64 Mbps. The network resource I is allocated as a 100 Mbps dedicated channel (including redundancy); the AI computing resource I is configured with 2-core CPUs + 2 GB of memory for each queuing machine audio stream, and the total requirement is 2000-core CPUs + 2000 GB of memory. 200 Pod instances are deployed using a Kubernetes cluster (each Pod is allocated 10 cores and 10 GB).

[0042] In another embodiment, a certain provincial credit union sets all network points to start business uniformly at 9:00. The head office data center pre-allocates network resource I and AI computing resource I at 8:55 through a scheduled task.

[0043] In another embodiment, a certain national commercial bank dynamically allocates resources according to the local business hours of network points in each province. For example, network resource I and AI computing resource I are allocated to Branch A at 8:55, and network resource I and AI computing resource I are allocated to Branch B at 9:55.

[0044] Step 2: The head office data center extracts the counter number information.

[0045] Step 21: The head office data center uses network resource I to obtain the queuing machine audio stream data of all network points. Suppose there are N network points, and each network point has only one queuing machine, so there are N queuing machines and N queuing machine audio stream data.

[0046] Step 22: The head office data center runs the counter number extraction algorithm on AI computing resource I.

[0047] Step 221: Use a speech recognition algorithm to transcribe the N queuing machine audio stream data into N text records. There is no limitation on which speech recognition algorithm to use, such as FunASR and PaddleSpeech open-sourced by Alibaba, etc.

[0048] Step 222: Use a natural language understanding algorithm to extract the counter number information of the counter that will handle business for the customer from the N transcribed text records. The text records contain service guidance information of the network points, and the typical format is "Please customer No. X go to bank counter Y for handling". There is no limitation on which natural language understanding algorithm to use. For example, a regular expression algorithm can be used, or a UIE (Universal Information Extraction) algorithm can be used, or a large language model can also be used.

[0049] If the counter number information Y for handling business for customer No. X is extracted, then the counter Y of the bank is processed in steps 3 - 8. If multiple counter number information is extracted simultaneously, then all the extracted counter information needs to be processed in steps 3 - 8 in parallel.

[0050] If the counter number information is not extracted, go to step 9.

[0051] In one embodiment, for the diverse voice broadcast formats of queuing machines at different outlets, a general regular expression pattern is designed to match multiple typical statements: "Please customer No. A102 go to counter No. 5 for service", "Please customer No. 23 go to VIP counter No. 8 for service", "Please customer Jinkuihua 32 go to counter No. 12 for service", "Please gold card customer D110 go to Room 2 in the VIP area". The Python code is as follows: patterns = (r"Please ([A-Z0-9]+) number.*? go to (\d+)th counter", 2), (r"(?:Go to|Please go to) (\d+)[st]?[nd]?[rd]?[th]? (?:counter|VIP counter|Counter)", 1), (r"(Directly)? go to (\d{2,3})th counter", 2), (r"VIP [area / room] (\d+)", 1) def extract_counter(text): for pattern, group_idx in patterns: match = re.search(pattern, text) if match: return match.group(group_idx) return None examples = 'Please customer No. A102 go to counter No. 5 for service', 'Please customer No. 23 go to VIP counter No. 8 for service', 'Please customer Jinkuihua 32 go to counter No. 12 for service', 'Please gold card customer D110 go to Room 2 in the VIP area' for text in examples: counter = extract_counter(text) print(f"{text}: {counter}"); The running result of the above code is: Please customer No. A102 go to counter No. 5 for service: 5 ​​Customer No. 23, please go to VIP Counter 8 to handle your business: 8 Please ask Golden Sunflower customer No. 32 to go to Counter 12 to handle your business: 12 Please ask Gold Card customer D110 to go to Room 2 in the VIP area: 2 In another embodiment, the bank has a large number of foreign customers, so English announcements are involved. Typical English announcement content is: "Customer A102, please proceed to Counter 5 for service.", "Customer 23, please go to VIP Counter 8 for your transaction.", "Golden Sunflower customer 32, please approach Counter 12 for assistance.", "Gold Card member D110, kindly visit Room 2 in the VIP area.". In order to ensure the correct extraction of counter numbers under multi-language conditions, a large language model is introduced to extract counter numbers. At this time, a feasible prompt is: "Please extract the counter label in the following description: xxxx." In another embodiment, the software related to the queuing machine is optimized and transformed so that when the queuing machine broadcasts queuing information, it can synchronously transmit the counter number information directly to the head office data center. Through this improvement, steps 1 and 2 in the business process can be skipped. That is to say, the head office data center no longer needs to extract the counter number information in the audio stream by itself, but can directly receive the accurate counter number information sent by the queuing machine. This optimization not only simplifies the data processing process, but also improves the accuracy and real-time performance of counter number recognition, further enhancing the response efficiency and service quality of the system.

[0052] Step 3: Allocate network resource II to establish a network link between the head office data center and Bank Counter Y. The head office data center uses network resource II to collect audio and video stream data of Bank Counter Y in real time.

[0053] Step 4: Allocate AI computing resource II-A. The head office data center uses AI computing resource II-A to run the service etiquette auditing algorithm to check whether the process of handling business by the teller at Counter Y (i.e., the teller of Bank Counter Y) complies with the reception etiquette standards.

[0054] In one embodiment, in order to audit whether the teller gives a salute to the customer, the system uses the OpenPose model to detect the key point positions of the teller's body. Based on this key point data, a classifier pre-trained by the ResNet50 network is further used to judge whether the teller has performed a salute.

[0055] In another embodiment, in order to audit whether the teller provides smiling service to customers, the system first uses the YOLO (You Only Look Once) model to detect the facial position of the teller. Once the face is detected, the system will extract these facial images and use a classifier trained with the ResNet50 network to determine whether the teller provides smiling service.

[0056] Step 5: Allocate AI computing resource II-B. The head office data center uses AI computing resource II-B to run the business type extraction algorithm.

[0057] Step 51: Use the speech recognition algorithm to transcribe the audio in the Y audio-video stream of the bank counter into text in real time. There is no limitation on the speech recognition technology used. For example, FunASR open-sourced by Alibaba, PaddleSpeech open-sourced by Baidu, etc.

[0058] In one embodiment, considering that bank branches are located in areas with a high penetration rate of Mandarin (such as first-tier cities), the communication between tellers and customers mainly uses standard Mandarin. In this case, the system uses the Mandarin speech recognition algorithm provided by FunASR for audio transcription.

[0059] In another embodiment, for the scenario where bank branches are located in areas with rich dialects or a low penetration rate of Mandarin, and tellers may have obvious local accents, the system adopts the following optimization solutions to achieve high-precision speech recognition: First, collect target dialect data covering various dialogue scenarios, and enhance the robustness of the model through data augmentation techniques such as speed change and background noise superposition. Second, build a speech recognition model based on the Transformer architecture, combine CNN (Convolutional Neural Network) and self-attention mechanism to effectively capture the local features and global context information of speech, and enhance the ability to understand different accents. Further, introduce a dialect-specific language model to optimize the understanding of grammar structure and semantics, build an end-to-end automatic speech recognition (ASR) system, directly predict text from audio, and simplify the recognition process. Finally, deploy a post-processing module to correct common recognition errors through rule matching and statistical learning methods, and continuously iterate and update the model based on user feedback to ensure the continuous optimization and adaptability improvement of the system performance.

[0060] Step 52: Use the natural language understanding algorithm to extract the business type from the transcribed text. Let the extracted business type be Z. There is no limitation on the natural language understanding technology used. For example, the method of regular expressions can be used to extract the type of business handled by customers from text records, or the UIE algorithm can be used for extraction.

[0061] In one embodiment, regular expressions are used to extract the business type. The following is the implemented python code import re # Define business types and their corresponding regular expressions business_patterns = { "Deposit": r"(Deposit|Deposit into|Savings)\s*\d+ yuan", "Withdrawal": r"(Withdraw|Withdraw from|Cash withdrawal)\s*\d+ yuan", "Transfer": r"(Transfer|Remittance|Transfer out)\s*\d+ yuan", "Loan": r"(Loan|Borrow|Mortgage)\s*", "Foreign exchange": r"(Exchange|Buy foreign exchange|Sell foreign exchange)\s*\d+ US dollars" } def extract_business_type(text): for biz_type, pattern in business_patterns.items(): match = re.search(pattern, text) if match: return biz_type return "Other" # Test examples texts = "I want to deposit 5000 yuan into my current account.", "Please help me withdraw 2000 yuan from my account.", "I want to transfer 10000 yuan to Zhang San's account.", "I want to apply for a 200,000 yuan housing loan.", "I want to exchange 500 US dollars." for text in texts: print(f"Input: {text}\nOutput: {extract_business_type(text)}\n") The running result of the above code is: Input: I want to deposit 5000 yuan into my current account.

[0062] Output: Deposit Input: Please help me withdraw 2000 yuan from my account.

[0063] Output: Withdrawal Input: I want to transfer 10000 yuan to Zhang San's account.​

[0064] Output: Transfer Input: I want to apply for a housing loan of $200,000.

[0065] Output: Loans Input: I want to exchange 500 USD.

[0066] Output: Foreign exchange Step 6: Release AI computing resources II-B.

[0067] In one embodiment, the software for handling business at the branch counter is modified so that the business type information can be immediately transmitted to the head office data center at the beginning of processing the customer business. Through this optimization, steps 5 and 6 can be skipped in the business process. In other words, the head office data center no longer needs to extract and analyze the business type by itself, but can directly receive the exact business type information sent by the branch counter. This improvement not only simplifies the data processing process, but also improves the accuracy and response speed of business type identification, thereby further enhancing the efficiency and reliability of the system. Such adjustments ensure that resources can be more accurately scheduled according to actual business needs, achieving more flexible and efficient system operation.

[0068] Step 7: Allocate AI computing resources II-C. The head office data center uses AI computing resources II-C to run the compliance audit algorithm and business completion detection algorithm for business type Z.

[0069] Step 71: Allocate AI computing resources II-C.

[0070] Step 72, running the compliance audit algorithm of business type Z, which is used to check whether the process of teller Y handling business Z for customer X is compliant, and its input is the audio and video stream of bank counter Y.

[0071] In a specific embodiment, different compliance audit algorithms need to be developed for compliance requirements of different business types, which may include: 1. Basic capital business, including current / fixed deposit handling, cash deposit and withdrawal, savings business such as fixed deposit and fixed-term deposit, as well as credit services such as personal / corporate loan application, housing loan / car loan / consumer loan approval and issuance.

[0072] 2. Payment and settlement services, including inter-bank transfers, real-time remittances, payment and loss reporting of checks, bills of exchange, and promissory notes, bank-securities transfers, cross-border payment and settlement, etc. Some banks also provide precious metal trading, foreign exchange conversion, and cross-border remittance services.

[0073] 3. Account management services cover the full life cycle management of personal / enterprise account opening, modification, closing, reporting and lifting of loss, password reset, etc., as well as the opening and maintenance of e-banking (online banking / mobile banking).

[0074] 4. Special financial services include agency services (collection of water and electricity bills, agency insurance / fund sales, payroll disbursement), wealth management services (subscription of funds / wealth management products, asset portfolio allocation), bill services (acceptance / discount), and customized products such as education savings and certificate of deposit pledge loans.

[0075] 5. Corporate services provide services such as checking account management, cash vault management, trade financing (letter of credit / guarantee), and corporate settlement (collection / letter of credit processing) for corporate customers. Some banks also support corporate loans and commercial housing mortgage services.

[0076] If step 52 identifies the business type as a large cash transaction, the system will activate the large cash transaction compliance auditing algorithm. This includes: using the YOLOv5 model specifically trained for cash counting machines and a classifier trained with MobileNet to identify the usage status of the cash counting machine; applying the HRNet hand key point detection algorithm to track the hand trajectory of the teller's contact with cash to determine whether the teller has long-term contact with cash; analyzing the two-person operation situation and its duration within the authorization area through the OpenPose pose estimation algorithm. This series of operations ensures a comprehensive audit from customer identity verification, equipment operation specifications to authorization process compliance.

[0077] If step 52 determines that the business type is the scenario of cash inventory verification, the system will activate the cash inventory verification compliance auditing algorithm. This includes: using the YOLOv5 model to detect the cash box and combining it with a classifier trained with MobileNet to judge the opening and closing status of the cash box; similarly, using the YOLOv5 model and the MobileNet classifier to monitor the usage of the cash counting machine; applying the HRNet hand key point detection algorithm to track the contact behavior between the inventory checker and cash in real time. In addition, a time series feature fusion algorithm will be used to evaluate the integrity of the entire operation process to ensure that all links from cash box opening, equipment use to cash contact meet the cash inventory management standards.

[0078] Step 73: While step 72 is running, run the business Z handling end detection algorithm. The input of this algorithm is the audio-video stream of bank counter Y, which is used to detect whether the process of handling business Z for customer X at bank counter Y has ended. If it has ended, stop step 7 and enter step 8. If not, continue to execute step 72 and step 73.

[0079] The business handling end detection algorithm integrates the audio information and video information in the counter audio-video stream.

[0080] Step 731: Use a speech recognition algorithm to transcribe the audio stream in the audio-video stream of bank counter Y into text, and use a natural language understanding algorithm to analyze the text to determine whether business Z has been completed. There is no limitation on the natural language understanding technology used. For example, the method of regular expressions can be used to extract information on the completion of business processing from the text record, or a large model can be used for extraction.

[0081] Step 732: Use object detection and object tracking technologies to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the business has been completed.

[0082] In one embodiment, the counter is located in the VIP reception room, where the environment is relatively private and the personnel are single. First, statically set the counter service area (ROI) in the video frame as the range of effective interaction with the customer. Use the YOLOv8 model to detect the customer's body and dynamically analyze its position change. Once the system continuously tracks that a certain customer has not appeared in the ROI area within 3 seconds, the customer is marked as the off-counter state, and it is determined that the business has been completed.

[0083] In another embodiment, the counter is set in the bank hall with a high passenger flow. Similarly, first statically set the counter service area (ROI) in the video frame as the range of effective interaction with the customer. Use the YOLOv8 model to detect the customer's body and dynamically analyze its position change, and at the same time use a classifier trained by ResNet18 to identify the customer's orientation (facing or turning back to the counter). When the system continuously tracks and finds that a certain customer has not appeared in the ROI area within 3 seconds and is in the state of turning back to the counter, the customer will be marked as the off-counter state, and it is determined that the business has been completed. In addition, according to the density of the passenger flow, the system can adaptively adjust the determination threshold. For example, during peak hours, the off-counter judgment time is extended to 5 seconds to enhance the detection robustness and accuracy in complex environments.

[0084] Step 733: Integrate the detection results of Step 731 and Step 732 to obtain the detection result of whether business Z has ended.

[0085] In one embodiment, when the quality of the audio-video stream is high (audio signal-to-noise ratio ≥ 30 dB and video resolution ≥ 720P), the system adopts a strict double confirmation mechanism. Only when both Step 731 and Step 732 determine that the business has been completed, will the business be finally confirmed as completed.

[0086] In another embodiment, if the quality of the audio-video stream is low (audio signal-to-noise ratio < 30 dB or video resolution < 720P), a lenient mechanism is enabled. At this time, as long as any one of Step 731 or Step 732 determines that the business has been completed, the business can be confirmed as completed.

[0087] In another embodiment, the system introduces a timeout mechanism. Specifically, if the processing time of Service Z exceeds the predetermined time limit, or a new customer arrives at Counter Y, or the queuing machine calls a new customer to go to Counter Y to handle the service, the system will automatically determine that the current service has ended.

[0088] In one embodiment, the software for handling services at the branch counter is optimized and transformed so that when the teller completes Service Z, the information indicating the end of the service can be immediately transmitted to the head office data center. Through this improvement, Step 73 in the service process can be skipped. That is to say, the head office data center no longer needs to rely on audio and video stream analysis to determine whether Service Z has ended, but can directly receive the exact notice of the end of Service Z from the branch counter. This optimization not only simplifies the process of service end detection, but also improves the accuracy and timeliness of the judgment, further enhancing the response speed and efficiency of the system.

[0089] Step 8: Release network resource II, AI computing resource II-A, and II-C.

[0090] Step 9: If the branch is still within the business hours, go back to Step 2; if the branch has ended its business for the day, release AI computing resource I and network resource I.

[0091] The above are all preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A business adaptive resource scheduling method for a bank counter video audit system, characterized in that: The following steps are involved: Step 1: When the bank branch opens in the morning, the head office data center allocates network resources I and AI computing resources I; Step 2: The head office data center uses network resource I to obtain the audio stream data of the number calling machines of all outlets. The head office data center runs the counter number extraction algorithm on AI computing resource I to extract the counter number information in the audio stream data of the number calling machine, and marks the extracted counter number as bank counter Y. Step 3: Allocate network resource II to establish a network link between the head office data center and bank counter Y obtained in step 2. The head office data center uses network resource II to collect audio and video stream data of bank counter Y in real time. Step 4: Allocate AI computing resource II-A. The head office data center uses AI computing resource II-A to run the service etiquette audit algorithm to analyze the audio and video stream data of bank counter Y to check whether the teller at bank counter Y meets the reception etiquette standards when handling business. Step 5: Allocate AI computing resources II-B. The head office data center uses AI computing resources II-B to run the business type extraction algorithm to extract the business type from the audio and video stream of bank counter Y and obtain business type Z. Step 6: Release AI computing resources II-B; Step 7: Allocate AI computing resources II-C. The head office data center uses AI computing resources II-C to run the compliance audit algorithm for business type Z to detect whether the process in the audio and video stream of bank counter Y is compliant until the process of handling business Z for the customer is completed. The head office data center uses AI computing resource II-C to run the business processing end detection algorithm for business type Z to detect whether the process of handling business for customers in the audio and video stream of bank counter Y is completed. If it is completed, stop step 7 and go to step 8; Step 8: Release network resources II, AI computing resources II-A and II-C; Step 9: If the branch is still in business hours, go back to step 2; if the branch has ended business for the day, the head office data center releases AI computing resources I and network resources I, and waits until business starts the next day before going to step 1.

2. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: Step 2 includes the following sub-steps: Step 21: The head office data center uses network resource I to obtain audio stream data of the number calling machines of all outlets. If there are N outlets, corresponding to N number calling machines, there are N channels of audio stream data of number calling machines; Step 22: The head office data center runs the counter number extraction algorithm on the AI ​​computing resource I to extract the counter number information of the N-channel call machine audio stream data.

3. The business adaptive resource scheduling method of the bank counter video audit system according to claim 2 is characterized in that: Step 22 includes the following sub-steps: Step 221: using a speech recognition algorithm to transcribe the audio stream data of the N-channel calling machine into N text records; Step 222: Use a natural language understanding algorithm to extract the counter number information of the customer from the N transcribed text records, and mark the extracted counter number as bank counter Y.

4. The business adaptive resource scheduling method of the bank counter video audit system according to claim 3 is characterized in that: If bank counter Y that will handle business for customer X is extracted, step 3 to step 8 are performed on bank counter Y; if multiple counter number information is extracted at the same time, step 3 to step 8 are performed independently and in parallel on all the extracted counter number information; if no counter number information is extracted, go to step 9.

5. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: Step 5 includes the following sub-steps: Step 51: Use a speech recognition algorithm to transcribe the audio in the audio and video stream of bank counter Y into text in real time; Step 52: Use a natural language understanding algorithm to extract the business type from the transcribed text, and mark the extracted business type as business type Z.

6. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: In step 7, the service completion detection algorithm includes the following sub-steps: Step 71: Use a speech recognition algorithm to transcribe the audio stream in the audio and video stream of the bank counter Y into text, and use a natural language understanding algorithm to analyze the text to determine whether the service Z has been completed; Step 72: Use target detection and target tracking technology to detect whether the customer has left the counter. If the customer has left the counter, it is determined that the service has been completed. Step 73: The results of step 71 and step 72 are integrated to obtain a detection result of whether service type Z is terminated.

7. The business adaptive resource scheduling method of the bank counter video audit system according to claim 1 is characterized in that: In step 1, after the bank branch starts operating, the head office data center allocates network resources I and AI computing resources I.

8. A business adaptive resource scheduling device for a bank counter video audit system, characterized in that: A business adaptive resource scheduling method for a bank counter video audit system used to implement any one of claims 1-7, the adaptive resource scheduling device includes an initial resource allocation module, a counter number extraction module, a dynamic transmission channel module, a dynamic transmission channel module, a service etiquette audit module, a business type extraction module, a business type resource release module, a compliance audit and termination detection module, a resource release control module and a loop control module.

9. The business adaptive resource scheduling device of the bank counter video audit system according to claim 8, characterized in that: The initial resource allocation module is used to execute step 1: when the outlet starts business, allocate network resources I and AI computing resources I; The counter number extraction module is used to execute step 2: extract the counter number information through voice recognition and natural language understanding of the audio stream of the calling machine; The dynamic transmission channel module is used to execute step 3: allocate network resources II to bank counter Y and establish an audio and video stream transmission link; The service etiquette audit module is used to execute step 4; allocate AI computing resources II-A to run the service etiquette detection algorithm; The business type extraction module is used to execute step 5; allocate AI computing resources II-B, and extract business type Z through real-time speech transcription and text parsing; The service type resource release module is used to execute step 6: release AI computing resources II-B; The compliance audit and end detection module is used to execute step 7: allocate AI computing resources II-C to run the compliance algorithm and end detection algorithm of business Z; The resource release control module is used to execute step 8: release network resources II, AI computing resources II-A and II-C; The loop control module is used to execute step 9: loop triggering step 2 or releasing initial resources according to the business status of the outlets.

10. The business adaptive resource scheduling device of the bank counter video audit system according to claim 9, characterized in that: Each module of the adaptive resource scheduling device is implemented based on the computer server of the head office data center.

Images