A method and device for quality inspection of sensitive words in debt collection based on a large model

Abstract

The present invention proposes a method and device for quality inspection of sensitive words in debt collection based on a large model, the method comprising: obtaining a debt collection recording generated online; calling a translation model API interface to convert the debt collection recording into a recording text; preprocessing the recording text, and segmenting the long text to obtain the input text; importing the input text into the original model for quality inspection, and outputting a first quality inspection result; constructing a local debt collection compliance vector knowledge base, calling the vector knowledge base based on a large language model, and importing the input text into the large language model for quality inspection, and outputting a second quality inspection result; according to historical debt collection recording data, and training based on a large language model, obtaining a customized large language model; importing the input text into the customized large language model for quality inspection, and obtaining a third quality inspection result; if there is one non-compliance, the quality inspection result is non-compliance. The present invention can ensure compliance collection, improve efficiency, reduce costs, reduce disputes, and promote the development of large language models in the financial field.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a method and device for quality inspection of sensitive words in debt collection based on a large model. Background Art

[0002] As the supervision of the financial industry intensifies, the compliance requirements for debt collection are getting higher and higher. Enterprises need to identify and filter sensitive words in the debt collection process to ensure that the debt collection behavior complies with relevant laws, regulations and industry standards and reduce potential legal risks. Post-loan debt collection is an important part of risk management for financial institutions and is also the part with the most manual intervention. In particular, in the field of traditional debt collection quality inspection, it mainly relies on manual work, which is costly and inefficient, and cannot meet the development needs of the financial debt collection industry. Summary of the invention

[0003] In view of the above problems, the present invention provides a method and device for quality inspection of sensitive words in debt collection based on a large model, which solves the problem that the traditional debt collection quality inspection field mainly relies on manual work, which is costly and inefficient.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for quality inspection of sensitive words in debt collection based on a large model, comprising the following steps: obtaining a debt collection recording generated online; calling a translation model API interface to convert the debt collection recording into a recorded text; preprocessing the recorded text, and segmenting the long text in the recorded text to obtain an input text; importing the input text into the original model for quality inspection, and outputting a first quality inspection result; building a local debt collection compliance vector knowledge base, calling the vector knowledge base based on a large language model, and importing the input text into the large language model for quality inspection, and outputting a second quality inspection result; according to historical debt collection recording data, and based on the large language model, P-tuning training is performed to obtain a customized large language model; importing the input text into the customized large language model for quality inspection, and obtaining a third quality inspection result; if there is at least one non-compliant item among the first quality inspection result, the second quality inspection result and the third quality inspection result, the final quality inspection result is non-compliant.

[0005] As a preferred solution, the construction of a local collection compliance vector knowledge base includes: collecting collection recordings based on expert experience or historical customer complaint cases; after converting the collection recordings into recording texts, screening out the speech part of the collector; using encoding software to convert the recording texts into 512-dimensional vectors, and storing the converted vectors in a database; after converting the recording text to be evaluated into a 512-dimensional vector, performing inner product calculation with all vectors in the database, the larger the inner product, the higher the similarity; if the similarity exceeds a set threshold, it means that the corresponding recording has a historical non-compliance problem.

[0006] As a preferred solution, before the input text is imported into the large language model for quality inspection, it also includes: reading the content to obtain the context related to the user request; filling the template with the request content and the context content to obtain the prompt word; and inputting the prompt word into the large language model.

[0007] As a preferred solution, the recorded text is preprocessed, including: removing the recorded text less than 30 seconds, and adding target label information based on expert experience and historical complaint information.

[0008] As a preferred solution, the method of obtaining a customized large language model based on historical collection recording data and P-tuning training based on a large language model includes: collecting recordings and text data in the collection field and preprocessing the data; using ASR technology to recognize recording data, distinguishing between collection personnel and overdue users, and converting the recording data into text data; using experts to label the text data, and labeling positive and negative labels based on whether they are compliant to generate training samples; dividing the training samples into a training set and a test set, the training set is used for p-tuning training, and the test set is used to evaluate the model effect; configuring p-tuning model parameters, and when the model effect reaches a set threshold, the customized large language model training is completed; and deploying the customized large language model in a production environment so that it can be called by the collection system through an API.

[0009] As a preferred solution, the ASR technology is used to identify the recorded data, distinguish between debt collectors and overdue users, and convert the recorded data into text data, including: using the whisperX model, specifying the language as Chinese and the speakers as two people; inputting the recorded file into the whisperX model, outputting the speaker and the speech content text; and filtering out the debt collector's speech text data based on the debt collector's fixed opening remarks.

[0010] As a preferred solution, the customized large language model is deployed in a production environment, including: importing the customized large language model into the production environment, adjusting the model state to eval mode; using the fastapi interface to provide API services to the outside world; providing the recorded text to be evaluated with prompt words on the API, and returning the evaluation result of the recorded text.

[0011] The present invention also provides a debt collection sensitive word quality inspection device based on a large model, comprising: an acquisition module, used to acquire a debt collection recording generated online; a recording conversion module, used to call a translation model API interface to convert the debt collection recording into a recording text; a preprocessing module, used to preprocess the recording text and segment the long text in the recording text to obtain an input text; a first quality inspection module, used to import the input text into the original model for quality inspection and output a first quality inspection result; a second quality inspection module, used to build a local debt collection compliance vector knowledge base, call the vector knowledge base based on a large language model, and import the input text into the large language model for quality inspection, and output a second quality inspection result; a model training module, used to perform P-tuning training based on historical debt collection recording data and a large language model to obtain a customized large language model; a third quality inspection module, used to import the input text into a customized large language model for quality inspection and obtain a third quality inspection result; a quality inspection result module, if there is at least one non-compliant item among the first quality inspection result, the second quality inspection result and the third quality inspection result, then the final quality inspection result is non-compliant.

[0012] Compared with the prior art, the beneficial effects of the present invention include: by mining the unstructured collection recording data accumulated in the collection business of financial institutions, after pre-processing operations such as data cleaning, the speaking object is identified, and then a model of collection sensitive words is generated, the sensitive content of the speech content is more accurately identified, and finally the quality inspection standard verification is completed, and the sensitive word risk situation that may be involved in the collection speech is output. Using the collection sensitive word quality inspection technology, the collection speech and text can be analyzed in an automated manner to reduce labor costs. Using the large language model technology, sensitive words can be discovered through the analysis of collection speech and text data, which helps to improve collection efficiency. Quality inspectors can conduct more targeted spot checks, reduce the workload of manual quality inspections, and improve work efficiency. The collection sensitive word quality inspection method provided by the present invention can ensure compliance, improve efficiency, reduce costs, reduce disputes, and promote the development of large language model technology in the financial field. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the accompanying drawings, the same reference numerals are used to refer to the same components. Among them:

[0014] Figure 1 This is a flow chart of a method for quality inspection of sensitive words in debt collection according to an embodiment of the present invention;

[0015] Figure 2 It is another flow chart of the method for quality inspection of sensitive words in debt collection according to an embodiment of the present invention;

[0016] Figure 3Schematic diagram of the structure of a sensitive word quality inspection device for debt collection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific implementation modes and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or as a limitation or restriction to the technical solution of the present invention.

[0018] According to one embodiment of the present invention, Figure 1 and 2 A collection sensitive word quality inspection method based on a large model includes the following steps:

[0019] S101, obtaining the collection recording generated online.

[0020] S102, calling the translation model API interface to convert the collection recording into recording text.

[0021] S103, pre-processing the recorded text, and segmenting the long text in the recorded text to obtain the input text. Performing segmented quality inspection and prediction on the segmented text can solve the bottleneck of input limitation of the large language model.

[0022] Among them, the recorded text is preprocessed, including: removing the recorded text less than 30 seconds, and adding target label information based on expert experience and historical complaint information.

[0023] S104, importing the input text into the original model for quality inspection, and outputting a first quality inspection result. The original model can be called to identify some obvious NSFW (uncivilized or inappropriate) words.

[0024] S105, build a local collection compliance vector knowledge base, call the vector knowledge base based on the large language model, import the input text into the large language model for quality inspection, and output the second quality inspection result. The large prediction model calls the vector knowledge base of local sensitive words to identify non-compliant business content.

[0025] Specifically, the above-mentioned construction of a local collection compliance vector knowledge base includes:

[0026] (1) Based on expert experience or historical customer complaint cases, collect collection recordings and expand the collection compliance field information available to the pre-trained large language model.

[0027] (2) After converting the collection recording into a transcript, filter out the collector’s speech;

[0028] (3) Use encoding software to convert the recorded text into a 512-dimensional vector and store the converted vector in the database. The encoding software is google-universal-encode.

[0029] (4) After converting the audio recording to be evaluated into a 512-dimensional vector, the inner product is calculated with all the vectors in the database. The larger the inner product, the higher the similarity.

[0030] (5) If the similarity exceeds the set threshold, it means that the corresponding recording has a history of non-compliance issues. For example, if the similarity threshold is set to 0.8, if it exceeds the similarity threshold, it means that the recording has a history of non-compliance issues.

[0031] In an embodiment of the present invention, before the input text is imported into the large language model for quality inspection, it also includes: reading the content of the local vector knowledge base, obtaining the context related to the user request, and then using the request content and the context content to fill in the template to obtain the prompt word, and then inputting the prompt word into the large language model.

[0032] S106, performing P-tuning training based on the historical collection recording data and the large language model to obtain a customized large language model.

[0033] Specifically, the steps include:

[0034] 1) Collect audio and text data in the field of debt collection and pre-process the data. Pre-processing includes data cleaning, denoising, labeling and other operations. For example: remove audio texts less than 30 seconds and add target label information based on expert experience and historical complaint information.

[0035] 2) Use ASR technology to identify recorded data, distinguish between debt collectors and overdue users, and convert the recorded data into text data locally in batches.

[0036] The ASR technology is specifically the DiarizationPipeline module of the whisperX model. When recognizing recordings, specify the language as Chinese and the speakers as two people; input the recording file (wav or MP3 format) to the whisperX model, and the model directly outputs the speaker and the speech content text. The speech text data of the debt collector can be filtered out based on the fixed opening remarks of the debt collector. The fixed opening remarks can be "I am from XX Bank...".

[0037] 3) Use expert labeling to distinguish text data, assign positive and negative labels based on compliance, and generate training samples.

[0038] For example: The training sample format of the model is:

[0039] {“input”: “Please judge whether the following reminder recording is compliant: [Reminder recording text]”,

[0040] "output": "noncompliant"}

[0041] 4) Divide the training samples into training set and test set. The training set is used for p-tuning training, and the test set is used to evaluate the model effect. Generally, thousands of labeled samples can be used for pre-training.

[0042] 5) Configure the p-tuning model parameters. When the model effect reaches the set threshold, the customized large language model training is completed. The model parameter that has a greater impact on the result is the learning rate. The training data is read in and trained on the GPU. After the training is completed, it is evaluated on the test set.

[0043] The P-tuning technology is used to fine-tune the large language model. The basic parameter values ​​in the pre-trained large language model are not changed. Only the embedding layer of the prompt word in the large language model is fine-tuned. Because there are fewer trainable parameters, it can be completed on a single GPU card. The customized model after P-tuning can output sensitive word quality inspection results, and the model output stability and accuracy are greatly improved compared to using only the pre-trained model.

[0044] Based on historical collection recordings, the P-tuning technology is used to fine-tune the large language model. The fine-tuned customized model can identify the quality inspection risks of users in the recordings end-to-end.

[0045] 6) Deploy the customized large language model in the production environment, which can be called by the collection system through the API. The sensitive word detection method based on the deep learning algorithm can efficiently detect sensitive words in the collection field and reduce missed detection and false detection.

[0046] Deploy the customized large language model in the production environment, including: importing the customized large language model into the production environment, adjusting the model status to eval mode; using the fastapi interface to provide API services to the outside world; providing the recorded text with prompt words to be evaluated on the API, and returning the evaluation result of the recorded text.

[0047] S107, importing the input text into the customized large language model for quality inspection to obtain a third quality inspection result. By calling the customized large language model, speech that is difficult to recognize can be recognized.

[0048] S108, if at least one of the first quality inspection result, the second quality inspection result and the third quality inspection result is non-compliant, the final quality inspection result is non-compliant.

[0049] Specifically, if any one of the first, second, and third quality inspection results is non-compliant, the final quality inspection result is non-compliant. The three quality inspections are evaluated from three different perspectives with different focuses: the first quality inspection is to simply identify obvious swear words through a large language model; the second quality inspection is to find non-compliant cases similar to those in history; the third quality inspection is to use the model to expand and predict potential non-compliant situations that have not occurred before. Through the three quality inspection processes, accurate quality inspections can be effectively performed on different objects, improving the efficiency and accuracy of quality inspections.

[0050] For example, if a debt collector curses someone on the phone, it can be directly identified by the first quality inspection. If a debt collector asks a customer to pay back the money to the debt collector's personal account (a non-compliance case that has frequently occurred in history), the user can repay the money, which can be identified by the second quality inspection. Some suggestive and inducing behaviors that may not be so easy to clearly define may be identified by the third quality inspection.

[0051] Optionally, the detected sensitive words can be compared with the quality inspection standards to optimize the quality inspection process and improve the efficiency and accuracy of quality inspection. By adjusting and optimizing the corpus, the corpus content is continuously optimized according to the latest industry requirements and the specifications of financial institutions.

[0052] See also Figure 3 The present invention also provides a collection sensitive word quality inspection device based on a large model, comprising:

[0053] An acquisition module 101 is used to acquire a collection recording generated online;

[0054] The recording conversion module 102 is used to call the translation model API interface to convert the collection recording into a recording text;

[0055] The preprocessing module 103 is used to preprocess the recorded text and segment the long text in the recorded text to obtain the input text;

[0056] A first quality inspection module 104, used to import the input text into the original model for quality inspection and output a first quality inspection result;

[0057] The second quality inspection module 105 is used to build a local debt collection compliance vector knowledge base, call the vector knowledge base based on the large language model, import the input text into the large language model for quality inspection, and output a second quality inspection result;

[0058] The model training module 106 is used to perform P-tuning training based on the historical collection recording data and the large language model to obtain a customized large language model;

[0059] A third quality inspection module 107, used to import the input text into the customized large language model for quality inspection to obtain a third quality inspection result;

[0060] The quality inspection result module 108 determines that if at least one of the first quality inspection result, the second quality inspection result and the third quality inspection result is non-compliant, the final quality inspection result is non-compliant.

[0061] In summary, the beneficial effects of the present invention include: by mining the unstructured collection recording data accumulated in the collection business of financial institutions, after pre-processing operations such as data cleaning, the speaking object is identified, and then a model of collection sensitive words is generated, the sensitive content of the speech content is more accurately identified, and finally the quality inspection standard verification is completed, and the risk of sensitive words that may be involved in the collection speech is output. Using the collection sensitive word quality inspection technology, the collection speech and text can be analyzed in an automated manner to reduce labor costs. Using the large language model technology, sensitive words can be discovered through the analysis of collection speech and text data, which helps to improve collection efficiency. Quality inspectors can conduct more targeted spot checks, reduce the workload of manual quality inspections, and improve work efficiency. The collection sensitive word quality inspection method provided by the present invention can ensure compliance, improve efficiency, reduce costs, reduce disputes, and promote the development of large language model technology in the financial field.

[0062] The present invention is a targeted application optimization of intelligent speech recognition and large language model projects in the financial field, and is an innovative attempt to use large models in the financial field in the field of debt collection quality inspection and compliance. The application scenarios are wide, and this technology can be applied to the quality inspection of voice, text and other data in the debt collection field, effectively identify sensitive words, improve quality inspection efficiency and accuracy, and help protect consumer rights and enhance the industry image. At the same time, this technical method can also be applied to other fields that require compliance control, such as financial product recommendations, live broadcasts and other emerging industries. In addition to the output results of large language model training, the technical method in this patent retains knowledge base rule judgment schemes such as expert labeling, and comprehensively gives quality inspection results, effectively combining the advantages of manual and large models.

[0063] The present invention proposes a method and device for quality inspection of sensitive words in debt collection based on a large model, which uses a machine learning algorithm to detect voice, text and other data in the debt collection field, and has high technical innovation. In addition, this patent discloses the specific implementation process of the quality inspection technology and application method of sensitive words in debt collection based on a large model, which is helpful to promote technical exchanges and cooperation and promote the development of related technologies. This patent can help reduce costs and increase efficiency in the debt collection field, improve quality inspection efficiency, and reduce labor costs; by using a large model for quality inspection of sensitive words and compliance in debt collection, the workload of manual quality inspection can be greatly reduced and the quality inspection efficiency can be improved. The large language model has a high detection accuracy rate, which can effectively discover sensitive words and compliance issues in the debt collection field and improve the accuracy of quality inspection. This patent can better supervise the compliance of the debt collection industry and protect the rights and interests of consumers through the quality inspection technology of sensitive words and compliance in debt collection based on a large language model. For the debt collection industry, the quality inspection technology of sensitive words and compliance in debt collection in this patent can improve the image of the industry and enhance the trust of the society in the debt collection industry. After the technical method of this patent is promoted, it can standardize the operation of the debt collection industry, prevent some improper debt collection behaviors, and promote the healthy development of the industry.

[0064] It should be understood that if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0065] The technical scope of the present invention is not limited to the contents in the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A collection sensitive word quality inspection method based on a large model, characterized in that: The steps include: Obtain collection recordings generated online; Calling the translation model API interface to convert the collection recording into recording text; Preprocessing the recorded text and segmenting the long text in the recorded text to obtain input text; Importing the input text into the original model for quality inspection, and outputting a first quality inspection result; Building a local debt collection compliance vector knowledge base, calling the vector knowledge base based on the large language model, importing the input text into the large language model for quality inspection, and outputting a second quality inspection result; Based on historical collection recording data and P-tuning training based on the large language model, a customized large language model is obtained; The method of obtaining a customized large language model by performing P-tuning training based on the historical collection recording data and the large language model includes: Collect audio and text data in the field of debt collection and pre-process the data; Use ASR technology to identify recording data, distinguish between debt collectors and overdue users, and convert recording data into text data; Using experts to label the text data, assigning positive and negative labels based on compliance, and generating training samples; Dividing the training samples into a training set and a test set, wherein the training set is used for p-tuning training and the test set is used for evaluating the model effect; Configure the p-tuning model parameters. When the model effect reaches the set threshold, the customized large language model training is completed. Deploy the customized large language model in a production environment so that it can be called by the collection system through an API; Importing the input text into the customized large language model for quality inspection to obtain a third quality inspection result; If at least one of the first quality inspection result, the second quality inspection result and the third quality inspection result is non-compliant, the final quality inspection result is non-compliant.

2. The method for quality inspection of sensitive words in debt collection based on a large model according to claim 1 is characterized in that: The construction of a local collection compliance vector knowledge base includes: Collect collection recordings based on expert experience or historical customer complaint cases; After converting the debt collection recording into a text recording, the speech portion of the debt collector is screened out; Using encoding software to convert the recorded text into a 512-dimensional vector, and storing the converted vector in a database; After converting the recorded text to be evaluated into a 512-dimensional vector, the inner product is calculated with all the vectors in the database. The larger the inner product, the higher the similarity; If the similarity exceeds the set threshold, it means that the corresponding recording has historical non-compliance issues.

3. The method for quality inspection of sensitive words in debt collection based on a large model according to claim 1 is characterized in that: Before the input text is imported into the large language model for quality inspection, it also includes: Read the content and obtain the context related to the user request; Fill the template with the request content and context content to obtain the prompt words; The cue words are input into a large language model.

4. The method for quality inspection of sensitive words in debt collection based on a large model according to claim 1 is characterized in that: The recorded text is preprocessed, including: removing the recorded text less than 30 seconds, and adding target label information based on expert experience and historical complaint information.

5. The method for quality inspection of sensitive words in debt collection based on a large model according to claim 1 is characterized in that: The method of using ASR technology to identify recording data, distinguish between debt collectors and overdue users, and convert the recording data into text data includes: Using the whisperX model, specify the language as Chinese and the speakers as 2 people; Input the recording file to the whisperX model and output the speaker and speech content text; The speech text data of the debt collectors are filtered out according to their fixed opening remarks.

6. The method for quality inspection of sensitive words in debt collection based on a large model according to claim 1 is characterized in that: Deploying the customized large language model in a production environment includes: Import the customized large language model into the production environment, and adjust the model state to eval mode; Use fastapi interface to provide API services to the outside world; Provide the recording text to be evaluated with the prompt words on the API, and the evaluation result of the recording text will be returned.

7. A collection sensitive word quality inspection device based on a large model, characterized in that: include: The acquisition module is used to obtain the collection recordings generated online; A recording conversion module, used to call the translation model API interface to convert the collection recording into a recording text; A preprocessing module, used to preprocess the recorded text and segment the long text in the recorded text to obtain input text; A first quality inspection module, used for importing the input text into the original model for quality inspection and outputting a first quality inspection result; A second quality inspection module is used to build a local debt collection compliance vector knowledge base, call the vector knowledge base based on the large language model, import the input text into the large language model for quality inspection, and output a second quality inspection result; The model training module is used to perform P-tuning training based on the large language model according to the historical collection recording data to obtain a customized large language model; the P-tuning training based on the large language model according to the historical collection recording data to obtain a customized large language model includes: Collect audio and text data in the field of debt collection and pre-process the data; Use ASR technology to identify recording data, distinguish between debt collectors and overdue users, and convert recording data into text data; Using experts to label the text data, assigning positive and negative labels based on compliance, and generating training samples; Dividing the training samples into a training set and a test set, wherein the training set is used for p-tuning training and the test set is used for evaluating the model effect; Configure the p-tuning model parameters. When the model effect reaches the set threshold, the customized large language model training is completed. Deploy the customized large language model in a production environment so that it can be called by the collection system through an API; A third quality inspection module, used for importing the input text into the customized large language model for quality inspection to obtain a third quality inspection result; Quality inspection result module: if at least one of the first quality inspection result, the second quality inspection result and the third quality inspection result is non-compliant, the final quality inspection result is non-compliant.

Images