Method and device for automatically determining auditing item list for double-recording auditing system

Abstract

The invention discloses a method and a device for automatically determining an auditing item list for a double-recording auditing system, which relate to the technical field of double-recording auditing in the financial industry and comprise a model training stage and a double-recording auditing item list determining stage. The model training stage comprises training of a whole-section verbal skill link division model, training of a link verbal skill classification model and training of an action information extraction model. According to the method, the whole section of verbal skill is subdivided, so that compliance check can be carried out on each link in the double-recording process more finely, and each link is ensured to meet the specification requirement; according to different link verbal skill types, the method can adaptively determine the corresponding auditing items, thereby greatly reducing the huge workload of a financial institution for setting which auditing items are adopted to carry out compliance detection on hundreds of financial products.

Description

Technical Field

[0001] The present invention relates to the technical field of double-recording auditing in the financial industry, and in particular to a method and device for automatically determining an audit item list in a double-recording auditing system. Background Art

[0002] ‌Double recording‌, or "simultaneous audio and video recording", refers to recording the sales process of financial products through audio and video recording. This practice is intended to ensure the transparency of the sales process and the integrity of the evidence. Double recording is an important business operation specification that can regulate sales behavior, protect customer rights and interests, and improve the transparency of the sales process. Through audio and video recording, it can ensure that the communication between sales staff and customers is fully recorded to avoid misleading or fraudulent behavior during the sales process.

[0003] In the context of dual recording (simultaneous audio and video recording) becoming the standard operating specification of the financial industry, the dual recording audit system came into being and became an important tool to ensure the compliance of the dual recording process. The dual recording audit system conducts a comprehensive audit of the recorded dual recording videos. Summarizing the application scenarios of dual recording such as financial management, insurance, trust, and funds, the dual recording audit system needs to conduct compliance checks on the dual recording videos from the following aspects: the integrity of the salesperson's speech, whether the salesperson has any illegal speech, whether the salesperson has any idle chat content unrelated to the dual recording, whether the salesperson has shown his or her sales qualification certificate, whether the salesperson has shown the customer relevant product introductions, risk warnings and other documents, whether the customer has answered the salesperson and whether he or she has given an affirmative answer, whether the customer has shown the portrait side and national emblem side of his or her second-generation ID card, whether the customer has read the product manual, risk warnings and other related paper or electronic documents, and whether the customer has signed these documents.

[0004] In the actual application of the double-recording audit system, for a specific financial product, it is necessary to conduct a detailed analysis of the entire script, and then set which audit items to use for compliance testing. Specifically, it is necessary to analyze how many links the double-recording process contains; analyze what scripts the salesperson needs to say in each link; analyze whether the customer needs to answer questions in each link, and make it clear what the customer's answer is compliant (for example, if the salesperson asks "Are you clear?", if the customer answers "yes" or "clear", it is the correct answer, if the customer answers "agree" or "unclear", it is a controversial answer.); analyze the qualifications that the salesperson needs to show in each link, and which qualifications need to be shown (such as showing a financial planner qualification certificate, a fund practitioner qualification certificate, an insurance agent qualification certificate, an insurance agent qualification certificate, an insurance broker qualification certificate, a trust practitioner qualification certificate, a fund practitioner qualification certificate, an investment advisor qualification certificate, a securities practitioner qualification certificate, a futures practitioner qualification certificate, a private equity fund manager registration certificate certificate, private equity fund practitioner qualification certificate, etc.); analyze whether the customer is required to show the second-generation ID card at each stage, and whether to show the portrait side or the national emblem side; analyze whether the salesperson needs to show the product documents at each stage, and specifically what kind of documents to show (such as risk disclosure statement, product manual, product agreement, product confirmation letter (public version), individual investor electronic signature agreement, electronic contract, electronic risk disclosure statement, trust contract, fund contract, insurance law, exemption of insurer liability, explanation of exemption of insurer liability clauses, customer rights and interests, product suitability assessment, liability exemption notice, insurance liability exemption, exemption of insurer liability clauses, asset management contract, confirmation of exemption of insurer liability, private equity fund contract, etc.); analyze whether the customer needs to sign at each stage, and analyze what the signature contains.

[0005] A financial institution needs to sell hundreds of financial products and needs to regularly or irregularly upgrade the entire sales pitch for these products. As you can imagine, it is a huge workload to set which audit items to use for compliance testing for all financial products.

[0006] Therefore, there is an urgent need for a method for automatically determining a list of audit items for a dual-recording audit system. This method can automatically determine which audit items are needed to perform compliance checks on dual-recording videos based on the entire speech. Summary of the invention

[0007] In order to solve the technical problem of automatically determining which audit items are needed to perform compliance checks on dual-recording videos, the present invention provides a method and device for automatically determining an audit item list for a dual-recording audit system. The following technical solution is adopted: A method for automatically determining an audit item list for a double-entry audit system, comprising a model training phase and a double-entry audit item list determination phase; The model training phase includes training the segment division model of the entire speech, training the segment speech classification model and training the action information extraction model; Training the segment segmentation model for the entire segment includes the following sub-steps: Step 111, collecting the entire speech of the dual recording application scenario, and constructing a training set of the segment division model of the entire speech; Step 112, training a segment division model for the entire speech; The training phase of the speech classification model includes the following sub-steps: Step 121, obtaining a training data set for a speech classification model in a training phase; Step 122, training session speech classification model; Training the motion information extraction model consists of the following sub-steps: Step 131, collecting a training data set for the motion information extraction model; Step 132, training a motion information extraction model; The stage of determining the double-record audit item list includes the following steps: Step 21, using the whole-segment speech segment division model to divide the whole-segment speech segment; Step 22, using the link speech classification model to classify each link speech into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech; Step 23, determine the list of double-recording audit items for speech, and add corresponding audit items to the list of double-recording audit items for speech according to the segment speech classification results determined in step 22.

[0008] By adopting the above technical solution and segmenting the entire speech, we can conduct a more detailed compliance check on each link in the dual recording process to ensure that each link meets the regulatory requirements; By using the trained segment division model of the entire speech, the entire speech can be automatically divided into segment speech according to different segments of the double recording, which improves work efficiency and reduces human errors; Through the training phase speech classification model, it can automatically identify and classify different types of speech, such as sales staff statements, action instructions, identity verification and customer answer instructions; For the action instruction-type speech, the action information extraction model can accurately extract key information such as the action performer, action type and action execution object, further improving the accuracy and automation of the audit; According to the different types of scripts in different links, this method can adaptively determine the corresponding audit items, thereby greatly reducing the huge workload of financial institutions in setting which audit items to use for compliance testing of hundreds of financial products.

[0009] Optional, the whole speech refers to all the speech that the salesperson needs to express to the customer in the process of selling finance. The whole speech segment division is to divide the whole speech into several segments according to the double recording links, and each double recording link corresponds to a segment of speech.

[0010] Optionally, the specific method of step 111 is: mark the training set as trainset={full_text_i, i=1,2,3,...,M}, where M is the number of training samples, full_text_i is the i-th whole paragraph; full_text_i={text_i_j, j=1,2,...,Ni}, text_i_j is the j-th link of the i-th whole paragraph full_text_i, and Ni is the number of link speeches of full_text_i.

[0011] Optionally, the specific method of step 121 is: manually label text_i_j, i=1,2,3,...,M, j=1,2,...,Ni obtained in step 111 to obtain tag_i_j, where the value of tag_i_j is sales staff statement speech, action instruction speech, identity verification speech or customer answer instruction speech.

[0012] Optionally, in step 131, the link speeches belonging to the action instruction type speeches obtained in step 121 are manually labeled one by one, and three key information including action executor Actor, action type ActionType, and action execution object Object are assigned to obtain a training data set for the action information extraction model.

[0013] By adopting the above technical solutions, we have made comprehensive applications of sample data. Not only have we annotated the entire conversation process in detail, but the same set of data can also serve three different model trainings at the same time: the segment division model of the entire conversation, the classification model of the conversation within the segment, and the action information extraction model. This improves the utilization rate of data.

[0014] Optionally, the training data set of the action information extraction model is represented as trainset_action={(text_k, actor_k, type_k, object_k), k=1, 2, ..., K}, where K is the number of samples, text_k is the link speech of the k-th action instruction type speech, actor_k, type_k, object_k are the corresponding action executor, action type, and action execution object.

[0015] Optionally, the stage of determining the double-record audit item list includes the following specific steps: Step 21: Use the trained segment segmentation model to divide the input full_text into several segment segments. The segmentation result can be expressed as {text_n, n=1,2,3,...,N}, where N is the number of segments and text_n is the segment segment of the nth segment. Step 22: Use the trained segment speech classification model to classify each segment speech text_n into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech. The speech classification result is represented as tag_n. Thus, the segment division result of the entire segment speech of full_text and the classification result of each segment speech are obtained, which are uniformly represented as {(text_n,tag_n), n=1,2,3,...,N}. Step 23: Determine the double-recording audit item list. Suppose the double-recording audit item list is checklist. For the n=1, 2, ..., Nth link script, perform the following processing in sequence: If tag_n is a salesperson's statement, add an audit item to the checklist: integrity of the link statement text_n; If tag_n is an action instruction, use the trained action information extraction model to extract information about the action type, the action executor, and the action execution object. Set the action type to ActionType, the action executor to actor, and the action execution object to object. Add audit items to the checklist: integrity of the link text_n, actor+ActionType+object; If tag_n is a speech for identity verification, add an audit item to the checklist: unity of person and evidence; If tag_n is a customer answer instruction script, add audit items to the checklist: completeness of the link script text_n, and the customer answers the salesperson's questions in accordance with the customer's compliant affirmative answer text customer_answers.

[0016] By adopting the above technical solution, the method can automatically determine the corresponding audit items according to the different types of scripts in different links, thereby greatly reducing the huge workload of financial institutions in setting which audit items to use for compliance testing of hundreds of financial products.

[0017] A device for automatically determining an audit item list for a dual-recording audit system includes a computer-readable storage medium and a processor. The computer-readable storage medium stores a program designed using a method for automatically determining an audit item list for a dual-recording audit system. The processor is communicatively connected to the computer-readable storage medium and runs the program to automatically determine the audit item list.

[0018] Optionally, a display is also included, which is communicatively connected to the processor and is used to display the results of determining the audit item list.

[0019] In summary, the present invention includes at least the following beneficial technical effects: The present invention can provide a method and device for automatically determining an audit item list for a double-recording audit system; 1) This method can perform more detailed compliance checks on each link in the dual recording process by segmenting the entire speech, ensuring that each link meets regulatory requirements.

[0020] 2) Automatically divide the speech into segments: Using the trained segment division model for the entire speech, the entire speech can be automatically divided into segments according to the different segments of the double recording, which improves work efficiency and reduces human errors.

[0021] 3) Intelligent classification of link speech: By training the link speech classification model, it can automatically identify and classify different types of speech such as sales staff statements, action instructions, identity verification and customer answer instructions; 4) Accurately extract action information: For the action instruction-type speech, the action information extraction model can accurately extract key information such as the action executor, action type and action execution object, further improving the accuracy and automation of the audit.

[0022] 5) Automatic determination of audit items: Based on the different types of scripts in different links, this method can adaptively determine the corresponding audit items, thereby greatly reducing the huge workload of financial institutions in setting which audit items to use for compliance testing of hundreds of financial products. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of a method for automatically determining an audit item list in a double-recording audit system according to the present invention; DETAILED DESCRIPTION

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings. The embodiment of the present invention discloses a method and device for automatically determining an audit item list for a dual-recording audit system.

[0025] Reference Figure 1 , a method for automatically determining an audit item list for a double-recording audit system, comprising a model training phase and a double-recording audit item list determination phase, wherein the model training phase comprises training a segment segmentation model, a segment segment segment classification model, and a training action information extraction model; Training the segment segmentation model for the entire segment includes the following sub-steps: Step 111, collecting the entire speech of the dual recording application scenario, and constructing a training set of the segment division model of the entire speech; Step 112, training a segment division model for the entire speech; The training phase of the speech classification model includes the following sub-steps: Step 121, obtaining a training data set for a speech classification model in a training phase; Step 122, training session speech classification model; Training the motion information extraction model consists of the following sub-steps: Step 131, collecting a training data set for the motion information extraction model; Step 132, training a motion information extraction model; The stage of determining the double-record audit item list includes the following steps: Step 21, using the whole-segment speech segment division model to divide the whole-segment speech segment; Step 22, using the link speech classification model to classify each link speech into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech; Step 23, determine the list of double-recording audit items for speech, and add corresponding audit items to the list of double-recording audit items for speech according to the segment speech classification results determined in step 22.

[0026] By segmenting the entire speech, we can conduct a more detailed compliance check on each link in the dual recording process to ensure that each link meets regulatory requirements; By using the trained segment division model of the entire speech, the entire speech can be automatically divided into segment speech according to different segments of the double recording, which improves work efficiency and reduces human errors; Through the training phase speech classification model, it can automatically identify and classify different types of speech, such as sales staff statements, action instructions, identity verification and customer answer instructions; For the action instruction-type speech, the action information extraction model can accurately extract key information such as the action performer, action type and action execution object, further improving the accuracy and automation of the audit; According to the different types of scripts in different links, this method can adaptively determine the corresponding audit items, thereby greatly reducing the huge workload of financial institutions in setting which audit items to use for compliance testing of hundreds of financial products.

[0027] The entire speech refers to all the words that sales personnel need to express to customers in the process of selling finance. The division of the entire speech into segments is to divide the entire speech into several segments according to the double recording links, and each double recording link corresponds to a segment of speech.

[0028] Optionally, the specific method of step 111 is: mark the training set as trainset={full_text_i, i=1,2,3,...,M}, where M is the number of training samples, full_text_i is the i-th whole paragraph; full_text_i={text_i_j, j=1,2,...,Ni}, text_i_j is the j-th link of the i-th whole paragraph full_text_i, and Ni is the number of link speeches of full_text_i.

[0029] The specific method of step 121 is: manually label the text_i_j obtained in step 111, i=1,2,3,...,M, j=1,2,...,Ni, to obtain tag_i_j, where the value of tag_i_j is sales staff statement speech, action instruction speech, identity verification speech or customer answer instruction speech.

[0030] In step 131, the link speech belonging to the action instruction type speech obtained in step 121 is manually labeled one by one, and three key information including action executor Actor, action type ActionType, and action execution object Object are given to obtain a training data set for the action information extraction model.

[0031] By adopting the above technical solutions, we have made comprehensive applications of sample data. Not only have we annotated the entire conversation process in detail, but the same set of data can also serve three different model trainings at the same time: the segment division model of the entire conversation, the classification model of the conversation within the segment, and the action information extraction model. This improves the utilization rate of data.

[0032] The training data set of the action information extraction model is represented as trainset_action={(text_k, actor_k, type_k, object_k), k=1, 2, ..., K}, where K is the number of samples, text_k is the link speech of the k-th action instruction type speech, actor_k, type_k, object_k are the corresponding action executors, action types, and action execution objects.

[0033] The stage of determining the double-record audit item list includes the following specific steps: Step 21: Use the trained segment segmentation model to divide the input full_text into several segment segments. The segmentation result can be expressed as {text_n, n=1,2,3,...,N}, where N is the number of segments and text_n is the segment segment of the nth segment. Step 22: Use the trained segment speech classification model to classify each segment speech text_n into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech. The speech classification result is represented as tag_n. Thus, the segment division result of the entire segment speech of full_text and the classification result of each segment speech are obtained, which are uniformly represented as {(text_n,tag_n), n=1,2,3,...,N}. Step 23: Determine the double-recording audit item list. Suppose the double-recording audit item list is checklist. For the n=1, 2, ..., Nth link script, perform the following processing in sequence: If tag_n is a salesperson's statement, add an audit item to the checklist: integrity of the link statement text_n; If tag_n is an action instruction, use the trained action information extraction model to extract information about the action type, the action executor, and the action execution object. Set the action type to ActionType, the action executor to actor, and the action execution object to object. Add audit items to the checklist: integrity of the link text_n, actor+ActionType+object; If tag_n is a speech for identity verification, add an audit item to the checklist: unity of person and evidence; If tag_n is a customer answer instruction script, add audit items to the checklist: completeness of the link script text_n, and the customer answers the salesperson's questions in accordance with the customer's compliant affirmative answer text customer_answers.

[0034] According to the different types of scripts in different links, this method can automatically determine the corresponding audit items, thereby greatly reducing the huge workload of financial institutions in setting which audit items to use for compliance testing of hundreds of financial products.

[0035] A device for automatically determining an audit item list for a dual-recording audit system includes a computer-readable storage medium and a processor. The computer-readable storage medium stores a program designed using a method for automatically determining an audit item list for a dual-recording audit system. The processor is communicatively connected to the computer-readable storage medium and runs the program to automatically determine the audit item list.

[0036] It also includes a display, which is communicatively connected to the processor and is used to display the results of determining the audit item list.

[0037] The following uses a specific implementation case to illustrate the implementation principle of the method and device for automatically determining the audit item list for the double-recording audit system: The method for automatically determining the audit item list for a double-recording audit system comprises the following steps: Step 1: Model training phase.

[0038] Step 11: Train the segment division model for the entire speech. The segment division of the entire speech is to divide the entire speech into several segments according to the double recording links, and each double recording link corresponds to a segment of the speech.

[0039] Step 111: Collect the entire speech of application scenarios such as financial management double recording, insurance double recording, trust double recording, and fund double recording, and construct a training set for the segment division model of the entire speech. Mark the training set as trainset={full_text_i, i=1,2,3,...,M}, where M is the number of training samples and full_text_i is the i-th entire speech. full_text_i={text_i_j, j=1,2,...,Ni}, text_i_j is the j-th segment speech of the i-th entire speech full_text_i, and Ni is the number of segment speeches of full_text_i.

[0040] Here is an example of a complete paragraph: full_text_1="[[customerName]][[honorificTitle]], Hello! I am [[realName]], employee number [[sellerNo]]. According to the regulations of the regulatory authorities and our bank, in order to protect your legal rights and interests, the following conversation will be recorded. Can I have a few minutes of your time? Please answer "yes" or "no". The recording begins. Please look up here for identity verification. Before signing the contract, you need to take a risk tolerance assessment test at our bank. Our bank will provide you with a refund based on your risk test results. You recommend a suitable financial product for investment. This suggestion is for reference only. You should still fully understand the investment risks and invest prudently. Are you clear? Please answer "clear" or "unclear". Are you clearly aware that investment and financial management are not savings deposits, and there are risks of income fluctuations and principal loss, and are you sure you can bear the corresponding investment risk results? Please answer "yes" or "no". Have you fully read and understood our bank's comprehensive financial management general agreement, customer rights and interests notice, application form and other documents? Please answer "yes" or "no". If you have no objection, please sign the sales contract, thank you. Thank you for your cooperation, this recording and video recording is over. " The segment division results for the entire speech full_text_1 are as follows: text_1_1="[[customerName]][[honorificTitle]], Hello! I am [[realName]], employee number [[sellerNo]]. According to the regulations of the regulatory authorities and our bank, in order to protect your legal rights and interests, the following conversation will be recorded. May I take a few minutes of your time? Please answer "yes" or "no"." text_1_2="Recording begins. Please look over here for identity verification." text_1_3="Before signing the contract, you need to take a risk tolerance assessment test at our bank. Our bank will recommend suitable financial products for you based on your risk test results. This recommendation is for reference only. You should still fully understand the investment risks and invest prudently. Do you understand this? Please answer "clear" or "unclear." text_1_4="Are you aware that investment and financial management are not savings deposits, and that there are risks of income fluctuations and principal loss, and are you sure you can bear the corresponding investment risk results? Please answer "yes" or "no". text_1_5="Have you fully read and understood our bank's comprehensive financial management agreement, customer rights notice, application form and other documents? Please answer "yes" or "no"." text_1_6="If you have no objection, please sign the sales contract. Thank you." text_1_7="Thank you for your cooperation. This recording is over." Step 112: Train the segmentation model for the entire speech. There is no restriction on the specific segmentation algorithm used, such as topic models (such as LDA, Latent Dirichlet Allocation), Text Tiling algorithm and other traditional algorithms. You can also use the segmentation of the entire speech based on BERT (Bidirectional Encoder Representations from Transformers) deep learning, or use a large language model to achieve segmentation of the entire speech.

[0041] In one embodiment, the open source pre-trained large language model is fine-tuned by instruction supervision to obtain the whole paragraph speech segment division model. The specific training method is as follows: first, the training set trainset={full_text_i, i=1,2,3,...,M} of the whole paragraph speech segment division model obtained in step 111 is converted into the data format required for instruction supervision fine-tuning of the large language model, and then the open source pre-trained large language model ChatGLM3-6B-32K is fine-tuned by instruction supervision to obtain a special large model for the whole paragraph speech segment division.

[0042] The format of the converted i-th full text_i={text_i_j,j=1,2,...,Ni} is as follows: {“instruction”: “You are an expert in segmenting the entire speech.” "input": "Divide the following entire speech into several segments according to the double recording stages, and each double recording stage corresponds to a segment of speech. full_text_i" "output": Session 1: text_i_1 Session 2: text_i_2 ...... Links Ni: text_i_Ni”} Step 12: Train the segment speech classification model. Segment speech classification is to classify segment speech into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech. It should be noted that each segment speech can belong to one of the categories or multiple categories at the same time.

[0043] Salesperson statements refer to the words used by sales personnel to introduce product information, explain terms, provide service details, etc. to customers. These statements are usually informational.

[0044] Action instruction-type speech refers to speech that requires sales personnel or customers to perform actions such as display, reading, signing, and stamping during the double recording process. Display action instructions are actions for sales personnel or customers to display their qualifications, identity documents, documents, and signature handwriting; reading instructions include actions for customers to read sales personnel's qualifications and documents and for sales personnel to read customers' signature handwriting and check customers' documents; signature action instructions are actions for customers to sign or transcribe specified sentences; Identity verification language refers to language that requires customers and sales staff to perform facial recognition during the double recording process.

[0045] Customer answer instruction-type speech refers to the speech that requires customers to answer questions asked by sales staff during the double recording process.

[0046] Step 121: Obtain a training data set for the speech classification model of the training phase. The specific method is to manually label the text_i_j, i=1,2,3,...,M, j=1,2,...,Ni obtained in step 111 to obtain tag_i_j, where the value of tag_i_j is one or more of the salesperson's statement speech, action instruction speech, identity verification speech, or customer answer instruction speech.

[0047] The classification results of the seven segments of speech divided by the above whole speech example full_text_1, text_1_1, text_1_2, text_1_3, text_1_4, text_1_5, text_1_6, and text_1_7 are as follows: tag_1_1=sales staff's statement-making skills, customers' answering instructions-making skills; tag_1_2=Action instruction speech, identity verification speech; tag_1_3=sales staff's statement-making techniques, customers' answering instructions-making techniques; tag_1_4=sales staff's statements and customers' instructions; tag_1_5=Customer response instruction type speech, action instruction type speech; tag_1_6=Action instruction type of speech; tag_1_7=Sales staff's presentation skills; Step 122: training the segment speech classification model. There is no restriction on the text classification algorithm used. For example, traditional statistical models such as Naive Bayes classification (NB), K-nearest neighbor (KNN), support vector machine (SVM), and tree model structure XGBoost and LightGBM can be used. Deep learning methods such as TextCNN (Convolutional Neural Networks for Sentence Classification, Yoon Kim, 2014 EMNLP) can also be used. A large language model can also be used to train the segment speech classification model.

[0048] In one embodiment, the link speech classification model is obtained by fine-tuning the open source pre-trained large language model with instruction supervision. The training data set of the link speech classification model obtained in step 121 is converted into a format, and then the open source pre-trained large language model ChatGLM3-6B-32K is fine-tuned with instruction supervision to obtain a dedicated large model for link speech classification. The converted format of the jth link speech text_i_j of the i-th whole speech is: {“instruction”: “You are an expert in segmentation categorization.” "input": "text_i_j" "output": "tag_i_j" } For example: {“instruction”: “You are an expert in segmentation categorization.” "input": "Are you aware that investment and financial management are not savings deposits, and there are risks of income fluctuations and principal loss, and are you sure you can bear the corresponding investment risk results? Please answer "yes" or "no". "output": "sales staff statement-type speech, customer response instruction-type speech"} Step 13: Train the action information extraction model. Extract three key information from the action instruction-based speech: the action performer Actor, the action type ActionType, and the action execution object Object.

[0049] Step 131: Collect the training data set of the action information extraction model. Manually label the link speech belonging to the action instruction type speech obtained in step 121 one by one, and assign three key information: the action executor Actor, the action type ActionType, and the action execution object Object, so as to obtain the training data set of the action information extraction model. We represent the training data set of the action information extraction model as trainset_action={(text_k, actor_k, type_k, object_k), k=1,2,...,K}, where K is the number of samples, text_k is the link speech of the kth action instruction type speech, actor_k, type_k, object_k are the corresponding action executor, action type, and action execution object.

[0050] For example, the manual marking result for the action instruction "Have you fully read and understood our bank's comprehensive financial management agreement, customer rights notice, application form and other documents? Please answer "yes" or "no"." is: (text_0="Have you fully read and understood our bank's comprehensive financial management agreement, customer rights notice, application form and other documents? Please answer "yes" or "no.", actor_0="customer", type_0="read", object_0="comprehensive financial management agreement, customer rights notice, application form") Step 132: training the action information extraction model. Similarly, it is not limited to which information extraction method is used. For example, the universal information extraction unified framework UIE algorithm (Universal Information Extraction, Yaojie Lu, ACL-2022) can be used, or a large language model can be used to train the action information extraction model.

[0051] It should be noted that if a large language model is used, steps 11, 12, and 13 can be combined to implement a model that simultaneously has the three functions of dividing the entire speech into segments, classifying the segment speech, and extracting action information.

[0052] Step 2: In the reasoning stage, determine the list of double-recording audit items based on the entire input speech.

[0053] Step 21: Divide the whole speech into segments: Use the whole speech segment division model obtained in step 11 to divide the input whole speech full_text into several segments. The division result can be expressed as {text_n, n=1,2,3,...,N}, N is the number of segments, and text_n is the segment speech of the nth segment.

[0054] Step 22: Segment Classification: Use the segment segment classification model obtained in step 12 to classify each segment segment text_n into one or more of the following categories: salesperson statement segment, action instruction segment, identity verification segment, and customer answer instruction segment. The results are represented as tag_n. The result of segmentation of the entire speech of full_text and the classification result of the speech in each segment are obtained by steps 21 and 22, which are uniformly expressed as {(text_n,tag_n), n=1,2,3,...,N} Step 23: Determine the double-recording audit item list. Assume that the double-recording audit item list is checklist = ["detect sales staff's illegal speech", "detect sales staff's idle chat with customers"], and for the speech of the n = 1, 2, ..., N links, perform the following processing in sequence: If tag_n is a statement made by the salesperson: add the audit item "Completeness of the text_n statement in the link" to the checklist.

[0055] If tag_n is an action instruction-type speech: use the action information extraction model trained in step 13 to extract the information of the action type, action executor and action execution object. Set the action type to ActionType, the action executor to actor, and the action execution object to object. Add audit items "integrity of link speech text_n" and "actor+ActionType+object" to the checklist.

[0056] If tag_n is an identity verification-related statement: add the audit item "Integration of person and evidence" to the checklist.

[0057] If tag_n is a customer answer instruction script: add audit items "Completeness of link script text_n" and "Customers answer salesperson's questions in accordance with the customer's compliant affirmative answer text customer_answers" to the checklist.

[0058] The customer Q&A instructions recorded in the double recording usually have an obvious question format, i.e. "Please answer yes or no?", "Please answer whether xx?", "May I ask if you xx?", "May I ask if you xx?", "Please answer xx or no xx?", "Whether xx?", where xx is "agree", "clear" or "ok". Therefore, the string search method can be used to determine the customer's compliant affirmative answer text customer_answers.

[0059] text=Customer answer instruction customer_answers=[] / / Customer answers initialize an empty list Let the client's affirmative answer instruction be text If text.find("yes")>=0 / / The customer's answer instruction contains "yes" customer_answers.append("yes") / / Add "yes" to the compliant affirmative answer keys_list = ["Agree", "Clear" or "OK"] forkeyinkeys_list: / / Convenient each element one by one If text.find(key)>=0: customer_answers.append(key) / / Add key to the compliant affirmative answer The audit item list for the above full text example full_text_1 is: checklist=[ Conduct sales staff detection of illegal speech techniques, Sales staff and customer chat detection, The completeness of the dialogue: "[[customerName]][[honorificTitle]], Hello! I am [[realName]], employee number [[sellerNo]]. According to the regulations of the regulatory authorities and our bank, in order to protect your legal rights and interests, the following conversation will be recorded. May I take a few minutes of your time? Please answer "yes" or "no". " The customer answers the salesperson's questions in the manner indicated in ["Yes"]. The completeness of the dialogue in the segment: "Recording begins. Please look over here for identity verification." The unity of person and evidence The completeness of the dialogue in this session is "Before signing the contract, you need to take a risk tolerance assessment test with our bank. Our bank will recommend suitable financial products for investment to you based on your risk test results. This suggestion is for reference only. You should still fully understand the investment risks and invest prudently. Are you clear? Please answer "clear" or "unclear."

[0060] The customer answers the salesperson's questions according to the answer method in ["clear"], The completeness of the dialogue in this section is as follows: "Are you aware that investment and financial management are not savings deposits, and there are risks of income fluctuations and principal loss, and are you sure you can bear the corresponding investment risk results? Please answer "yes" or "no". The customer answers the salesperson's question in the same way as ["Yes"]. The completeness of the dialogue in the session: "Have you fully read and understood our bank's comprehensive financial management master agreement, customer rights notice, application form and other documents? Please answer "yes" or "no"."

[0061] The customer answers the salesperson's question in the same way as ["Yes"]. The completeness of the link words "If you have no objection, please sign the sales contract, thank you." The customer signs the sales contract. The completeness of the segment script "Thank you for your cooperation. This recording and videotaping is over." ] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically determining an audit item list for a double-recording audit system, characterized in that: It includes the model training stage and the stage of determining the double-recording audit item list. The model training stage includes training the segment division model of the whole speech, training the segment speech classification model and training the action information extraction model; Training the segment segmentation model for the entire segment includes the following sub-steps: Step 111, collecting the entire speech of the dual recording application scenario, and constructing a training set of the segment division model of the entire speech; Step 112, training a segment division model for the entire speech; The training phase of the speech classification model includes the following sub-steps: Step 121, obtaining a training data set for a speech classification model in a training phase; Step 122, training session speech classification model; Training the motion information extraction model consists of the following sub-steps: Step 131, collecting a training data set for the motion information extraction model; Step 132, training a motion information extraction model; The stage of determining the double-record audit item list includes the following steps: Step 21, using the whole-segment speech segment division model to divide the whole-segment speech segment; Step 22, using the link speech classification model to classify each link speech into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech; Step 23, determine the list of double-recording audit items for speech, and add corresponding audit items to the list of double-recording audit items for speech according to the segment speech classification results determined in step 22.

2. The method for automatically determining an audit item list for a dual-recording audit system according to claim 1, characterized in that: The entire speech refers to all the words that sales personnel need to express to customers in the process of selling finance. The division of the entire speech into segments is to divide the entire speech into several segments according to the double recording links, and each double recording link corresponds to a segment of speech.

3. The method for automatically determining an audit item list for a dual-recording audit system according to claim 1, characterized in that: The specific method of step 111 is: mark the training set as trainset={full_text_i, i=1,2,3,...,M}, where M is the number of training samples, full_text_i is the i-th whole paragraph; full_text_i={text_i_j, j=1,2,...,Ni}, text_i_j is the j-th link of the i-th whole paragraph full_text_i, and Ni is the number of link statements of full_text_i.

4. The method for automatically determining the audit item list for a dual-recording audit system according to claim 3, characterized in that: The specific method of step 121 is: manually label the text_i_j obtained in step 111, i=1,2,3,...,M, j=1,2,...,Ni, to obtain tag_i_j, where the value of tag_i_j is sales staff statement speech, action instruction speech, identity verification speech or customer answer instruction speech.

5. The method for automatically determining an audit item list for a dual-recording audit system according to claim 1, characterized in that: In step 131, the link speech belonging to the action instruction type speech obtained in step 121 is manually labeled one by one, and three key information including action executor Actor, action type ActionType, and action execution object Object are given to obtain a training data set for the action information extraction model.

6. The method for automatically determining the audit item list for a dual-recording audit system according to claim 5, characterized in that: The training data set of the action information extraction model is represented as trainset_action={(text_k, actor_k, type_k, object_k), k=1, 2, ..., K}, where K is the number of samples, text_k is the link speech of the k-th action instruction type speech, actor_k, type_k, object_k are the corresponding action executors, action types, and action execution objects.

7. The method for automatically determining an audit item list for a dual-recording audit system according to claim 6, characterized in that: The stage of determining the double-record audit item list includes the following specific steps: Step 21: Use the trained segment segmentation model to divide the input full_text into several segment segments. The segmentation result is expressed as {text_n, n=1,2,3,...,N}, where N is the number of segments and text_n is the segment segment of the nth segment. Step 22: Use the trained segment speech classification model to classify each segment speech text_n into one or more of the following categories: salesperson statement speech, action instruction speech, identity verification speech, and customer answer instruction speech. The speech classification result is represented as tag_n. Thus, the segment division result of the entire segment speech of full_text and the classification result of each segment speech are obtained, which are uniformly represented as {(text_n,tag_n), n=1,2,3,...,N}. Step 23: Determine the double-recording audit item list. Suppose the double-recording audit item list is checklist. For the n=1, 2, ..., Nth link script, perform the following processing in sequence: If tag_n is a salesperson's statement, add an audit item to the checklist: integrity of the link statement text_n; If tag_n is an action instruction, use the trained action information extraction model to extract information about the action type, the action executor, and the action execution object. Set the action type to ActionType, the action executor to actor, and the action execution object to object. Add audit items to the checklist: integrity of the link text_n, actor+ActionType+object; If tag_n is a speech for identity verification, add an audit item to the checklist: unity of person and evidence; If tag_n is a customer answer instruction script, add audit items to the checklist: completeness of the link script text_n, and the customer answers the salesperson's questions in accordance with the customer's compliant affirmative answer text customer_answers.

8. A device for automatically determining an audit item list for a double-recording audit system, characterized in that: It comprises a computer-readable storage medium and a processor, wherein the computer-readable storage medium stores a program designed using the method for automatically determining an audit item list for a dual-recording audit system as described in any one of claims 1 to 7, and the processor is communicatively connected to the computer-readable storage medium to run the program to automatically determine the audit item list.

9. The device for automatically determining the audit item list for a dual-recording audit system according to claim 8, characterized in that: It also includes a display, which is communicatively connected to the processor and is used to display the results of determining the audit item list.

Images