Debt conciliation verification method based on multi-modal characteristics and electronic equipment

Abstract

The embodiment of the invention relates to a debt mediation verification method based on multi-modal features and electronic equipment, and belongs to the technical field of financial credit risk assessment or fraud detection.The method comprises the steps that first and second voice subjects in voice information are recognized, debtor declaration income of second voice content is extracted, and the debtor declaration income of the second voice content is obtained; and comparing the actual income of the debtor with the declared income of the debtor to obtain a flow matching deviation, analyzing an acoustic index, carrying out matching degree detection on the current voiceprint, and obtaining a credibility score of the debtor according to a comprehensive evaluation formula. According to the embodiment of the invention, the voice interaction content of the first voice main body and the second voice main body is split into multi-modal information to obtain the corresponding voiceprint and acoustic index, and the credibility score is calculated by using the comprehensive evaluation formula in combination with the multi-modal characteristics such as the voice content credibility, the flow matching deviation, the voiceprint matching degree, the environmental noise index and the voice speed fluctuation coefficient. And finally, the real repayment capability of the debtor is restored.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to a debt mediation verification method based on multi-modal features, and an electronic device, and belongs to the technical field of financial credit risk assessment or fraud detection. BACKGROUND

[0002] With the rapid development of consumer finance, Internet lending and small and micro loans, the debt disputes between borrowers and financial institutions are showing explosive growth, and it is particularly important to master the real financial situation of the debtor. The traditional debt mediation in the prior art mainly relies on the manual interview of mediators, paper document review or telephone follow-up, and the following problems exist: It is difficult to master the real financial status of the debtor at the first time, because the debtor often exaggerates the expenditure, hides the income or forges the bank flow, and the mediator can hardly distinguish it by naked eye.

[0003] The mediation efficiency is low, and the labor hours are consumed: the artificial comparison of bank flow, call recording, social media, etc. is time-consuming and labor-intensive, and it is difficult to improve the efficiency, and the experience of the mediator is highly dependent.

[0004] Incomplete evidence chain: the prior art can usually only process single modal data (such as bank flow account documents), and cannot fuse multi-dimensional information such as the voice, behavior of the debtor and the transaction records involving the bank flow, which is easy to cause secondary disputes.

[0005] High subjective risk: the result of debt mediation is easily affected by the subjective judgment of the mediator, lacks a quantitative and reliable scoring system, and it is difficult to form a unified standard, and the misjudgment rate and error rate are high.

[0006] In summary, the prior art cannot meet people's requirements and needs to be improved. SUMMARY

[0007] The main purpose of the embodiment of the present application is to provide a debt mediation verification method based on multi-modal features and an electronic device, so as to solve the technical problems in the prior art that only single modal debt mediation data can be processed, the result of debt mediation is easily affected by the subjective judgment of the mediator, and a unified debt mediation standard is lacked.

[0008] The embodiment of the present application adopts the following technical solutions: According to an aspect of the embodiments of the present application, a debt mediation verification method based on multi-modal features is provided, including: obtaining voice information, identifying a first voice subject and a second voice subject in the voice information, the first voice subject being a debt mediator, and the second voice subject being a debtor; cutting voice interaction content of the first voice subject and the second voice subject, filtering first voice content of the first voice subject in the voice interaction content, performing voice recognition on second voice content of the second voice subject, and extracting a debtor declared income of the second voice content; querying actual income of the debtor of the second voice subject according to the debtor declared income, comparing the actual income of the debtor with the debtor declared income, and obtaining a flow matching deviation; obtaining an acoustic index of the second voice content, performing authenticity analysis according to a change of the acoustic index, and obtaining a second voice content credibility; obtaining a voiceprint of the second voice subject, performing matching degree detection on the current voiceprint and a historical voiceprint, and obtaining a voiceprint consistency of the second voice subject; providing a comprehensive evaluation formula, evaluation factors in the comprehensive evaluation formula including at least the flow matching deviation, the second voice content credibility, and the voiceprint consistency, assigning corresponding weight values to the evaluation factors, evaluating a repayment ability of the second voice subject through a calculation result of the comprehensive evaluation formula, and obtaining a second voice subject credibility score.

[0009] According to at least one of the embodiments of the present application, in the process of separating the first voice subject and the second voice subject according to the subject of the voice information, the first voice content of the first voice subject and the acoustic index of the second voice subject are obtained respectively, the first voice content is a legal term related to debt mediation, and the acoustic index includes a fundamental frequency change and an amplitude change.

[0010] According to at least one of the embodiments of the present application, the voice recognition on the second voice content of the second voice subject and the extraction of the debtor declared income of the second voice content further include: identifying voice content related to the income situation of the second voice subject, and connecting the voice recognition result to a bank system interface to query the actual income of the debtor of the second voice subject.

[0011] According to at least one of the embodiments of the present application, the connection of the voice recognition result to the bank system interface and the query of the actual income of the debtor of the second voice subject specifically include: extracting the debtor declared income from the second voice content by using voice recognition technology, positioning a time window of the debtor declared income through a time stamp; inputting the extracted debtor declared income to the bank system through the interface, and returning a corresponding query result by the bank system, the query result being a deviation percentage of the debtor declared income and the actual income of the debtor.

[0012] According to at least one of the embodiments of the present application, the acoustic index of the second voice content is obtained, authenticity analysis is performed according to the change of the acoustic index, and the second voice content credibility is obtained. Further, the acoustic index of the second voice content is obtained, and the acoustic index includes a fundamental frequency change and an amplitude change. According to the current voiceprint vector and the historical voiceprint vector of the second voice subject, a voiceprint matching degree is obtained, and the voiceprint matching degree satisfies the following formula: voiceprint matching degree = 1 - (cosine distance of current voiceprint vector - historical voiceprint vector).

[0013] According to at least one of the embodiments of the present application, a comprehensive evaluation formula is provided, and evaluation factors in the comprehensive evaluation formula at least include a flow matching deviation, a second voice content credibility, and a voiceprint consistency. Further, an environmental noise index is included, the environmental noise index is an abnormal sound in voice interaction, and a speech speed fluctuation coefficient satisfies the following formula: speech speed fluctuation coefficient = min (0.5, second voice subject's speech speed standard deviation / second voice subject's basic speech speed).

[0014] According to at least one of the embodiments of the present application, the comprehensive evaluation formula is specifically: credibility score = [0.3 x second voice content credibility] + [0.25 x (1 - min (1, |flow matching deviation| / 0.5))] + [0.2 x voiceprint matching degree] - [0.15 x environmental noise index] - [0.1 x speech speed fluctuation coefficient].

[0015] According to another aspect of the embodiments of the present application, a debt mediation verification system based on multi-modal features is provided for implementing the debt mediation verification method based on multi-modal features. The system comprises: a voice information and subject acquisition module, which acquires voice information, identifies a first voice subject and a second voice subject in the voice information, the first voice subject being a debt mediator, and the second voice subject being a debtor; a voice interaction content cutting and filtering module, which cuts voice interaction content of the first voice subject and the second voice subject, filters first voice content of the first voice subject in the voice interaction content, performs voice recognition on second voice content of the second voice subject, and extracts a debtor income declaration of the second voice content; a flow matching deviation calculation module, which queries actual income of the debtor of the second voice subject according to the debtor income declaration, compares the actual income of the debtor with the debtor income declaration, and obtains a flow matching deviation; a voice content credibility analysis module, which acquires an acoustic index of the second voice content, performs authenticity analysis according to a change of the acoustic index, and obtains a credibility of the second voice content; a voiceprint consistency acquisition module, which acquires a voiceprint of the second voice subject, performs matching degree detection on the current voiceprint and a historical voiceprint, and obtains a voiceprint consistency of the second voice subject; and a credibility scoring module, which provides a comprehensive evaluation formula, evaluation factors in the comprehensive evaluation formula at least including the flow matching deviation, the credibility of the second voice content, and the voiceprint consistency, assigns corresponding weight values to the evaluation factors, evaluates a repayment ability of the second voice subject through a calculation result of the comprehensive evaluation formula, and obtains a credibility score of the second voice subject.

[0016] According to another aspect of the embodiments of the present application, an electronic device is provided, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes steps of the method.

[0017] According to other aspects of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes steps of the method.

[0018] The embodiments of the present application have the following beneficial technical effects: The embodiment of the application verifies debt mediation based on multi-modal features, splits the voice interaction content of the first voice subject and the second voice subject into multi-modal information, obtains the first and second voice content and corresponding voiceprints and acoustic indexes, extracts the debtor's declared income and corresponding timestamps in the mediation process through the sound source separation technology, queries the bank flow information of the time window corresponding to the timestamp through the timestamp and access to the bank system interface, combines multi-modal features such as voice content credibility, flow matching deviation, voiceprint matching degree, environmental noise index and speech speed fluctuation coefficient, calculates the credibility score by using a comprehensive evaluation formula, and finally restores the real repayment ability of the debtor.

[0019] The embodiment of the application cuts the subject of the voice interaction content, analyzes the voice content related to the debtor, prevents confusion of the subject of the voice, ensures the consistency of the voice subject in the subsequent data processing process, obtains the flow matching deviation by comparing the debtor's declared income and the debtor's actual income, restores the objective financial status of the debtor, and provides real and reliable original financial data for restoring the real repayment ability of the debtor.

[0020] The embodiment of the application can also perform dual verification of semantics and subjects through the acoustic indexes and voiceprint matching degree of the debtor, compare the voiceprint of this call with historical records to ensure that the current call and previous calls are the same person, form a traceable evidence chain, and additionally confirm the authenticity of the current call through the environmental noise index and the speech speed fluctuation coefficient. Based on multi-modal features, the debtor is verified from multiple dimensions, a comprehensive evaluation formula reflecting the real financial status of the debtor is formed, the debtor repayment ability index is output, the real status of the debtor is restored, and fraud risks are identified. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0022] Figure 1 is the framework diagram of the main application scenario of the embodiment of the application.

[0023] Figure 2 is the timing diagram of the main application scenario of the embodiment of the application.

[0024] Figure 2A is the flowchart of voice subject role separation and bank flow verification in the main application scenario.

[0025] Figure 2Bis a flowchart of a comprehensive evaluation formula for calculating the reliability score in the main application scenario.

[0026] Figure 3 is a flowchart of steps S1 to S5 of the embodiment of the present application.

[0027] Figure 4 is a flowchart of steps S21 to S23 of the embodiment of the present application.

[0028] Figure 5 is a flowchart of steps S41 to S43 of the embodiment of the present application.

[0029] Figure 6 is an architectural diagram of the debt mediation verification system based on multi-modal features.

[0030] Figure 7 is a structural schematic diagram of an electronic device. DETAILED DESCRIPTION

[0031] The technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the specific embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0032] As shown in the main application scenario of the embodiments of the present application shown in Figure 1 and Figure 2 , the data flow and the order of calling data between different subjects in a mediation call are described by a timing chart. The subjects in the main application scenario include: a user (a debtor), a mediator, a mediation system (which applies the debt mediation verification method of the present application), a bank external API (a bank gateway), and a bank flow database (a bank internal database).

[0033] Stage 1: Mediation Preparation The user (the debtor) and the mediator negotiate, both parties have the willingness to mediate, and have the basis for debt mediation and the necessity of debt mediation verification.

[0034] The mediator configures the mediation system: selects a mediation template to prepare a conversation in the mediation system, the template has pre-installed scripts, for example, requiring the debtor to declare his income situation and show his true identity, etc.

[0035] Stage 2: Mediation Call The mediator conducts mediation dialogue according to a fixed template and words, the mediation system records in real time, and the mediator and the user communicate in real time. After mediation, the mediator submits the mediation recording and template information according to the record of the mediation system.

[0036] Stage 3, the mediation system performs technical processing: Implementing the technical solutions provided by the embodiments of the application, the subject of voice sending is identified, the voice of the mediator in the recording is filtered out, only the voice segment of the debtor is retained, the voice of the debtor is converted into text, and is mapped to the template field. Among them, at the template field "income declaration", claim.amount, that is, the monthly income amount claimed by the debtor, is extracted, and through the voice segment, the declaration time point (t) is located. The timestamp positioning can use the ASR technology of intelligent speech recognition, and the starting time point (t) corresponding to the income declaration of the debtor is obtained Stage 4, bank flow query: The mediation system accesses the bank's external API (interface), constructs query parameters, for example: start_time = t - 30 d end_time= t type = "wage entry" The bank's external API obtains the query parameters constructed and improved by the mediation system, sends instructions to the bank's internal bank flow database, and executes the query task. The bank flow database obtains the query task, returns the actual wage flow record (which has been desensitized) to the bank's external API. The bank's external API calculates the flow matching deviation using the formula: deviation=(claim-actual_avg) / actual_avg), and after the calculation is completed, only the flow matching deviation is returned, and no specific amount or other sensitive information is returned.

[0037] Stage 5, risk level calculation: After the mediation system receives the flow matching deviation, it calculates the risk level. If the risk_level=(deviation>0.3) is true, it is judged as "high risk", that is, the corresponding debtor is a high-risk debtor, or: deviation<0.3, it is judged that the corresponding debtor is a low-risk debtor. Deviation exceeding ±30% is considered to be obviously false or concealed, triggering a high-risk flag, and different processing strategies and solutions are used for high-risk debtors than low-risk debtors.

[0038] The local database or cloud system stores the verification results of stages 1 to 5, including but not limited to: claim_amount, actual_avg, deviation, risk_level, timestamp, etc. Storing the verification results can provide the mediator or other risk control personnel with a review of the verification conclusion, and can also serve as electronic evidence for subsequent arbitration and litigation.

[0039] As shown in Figure 2A and Figure 2B , in the main application scenario of the embodiments of the present application, the debt mediation verification process can be summarized as: In Figure 2A , the mediation recording is performed and the acoustic feature hierarchical extraction is performed to obtain the surface features (speech rate / pause frequency) of the voice interaction content, the deep features (base frequency jitter (concealment tendency)) and the scene features (double role dialogue gap). When verifying the bank flow, the bank flow is dynamically parsed to obtain the explicit indicators (income stability) and the implicit indicators (abnormal transaction density), and finally the content obtained by the mediation recording and the content obtained by the bank flow are subjected to cross-modal alignment matrix operation, a comprehensive evaluation formula is generated and the credibility score is calculated. The function of the cross-modal alignment matrix is to record the mapping relationship between different signal sources.

[0040] In Figure 2B , the debt mediation verification and evaluation is started, multi-factor data collection is performed, the integrity of the collected data is detected, and whether there is data missing is detected. If there is data missing, the process of supplementary verification is triggered, or: if there is no data missing, the corresponding evaluation factors are preprocessed, the credibility score is calculated through the comprehensive evaluation formula, the repayment ability of the debtor is judged according to the score threshold, if the score ≥ 0.75, it is judged as a low-risk debtor, and a green channel is provided for it; if the score is between 0.5-0.75, it is judged as a medium-risk debtor, and artificial review is needed; if the score ≤ 0.5, it is judged as a high-risk debtor, the mediation scheme is rejected and fraud warning information is triggered.

[0041] As shown in Figure 3 , the debt mediation verification method based on multi-modal features comprises: Step S1, obtaining voice information, identifying the first voice subject and the second voice subject in the voice information, the first voice subject being the debt mediator, and the second voice subject being the debtor. In the example, in the process of separating the first voice subject and the second voice subject according to the subject of the voice information in step S1, the first voice content of the first voice subject and the acoustic index of the second voice subject are obtained respectively, the first voice content being legal terms related to debt mediation, and the acoustic index including fundamental frequency variation and amplitude variation. The legal terms related to debt mediation can include: mediation agreement, subrogation, joint and several liability, enforcement, penalty interest, etc.

[0042] Step S2, cutting the voice interaction content of the first voice subject and the second voice subject, filtering the first voice content of the first voice subject in the voice interaction content, performing voice recognition on the second voice content of the second voice subject, and extracting the debtor's declared income of the second voice content.

[0043] Step S3, querying the actual income of the debtor according to the debtor's declared income of the second voice subject, comparing the actual income of the debtor with the declared income of the debtor, and obtaining the water matching deviation. As a supplement to step S3, the actual income of the debtor can also include: bank water, WeChat / PayPal water, water, electricity and coal expenses, etc.

[0044] Step S4, obtaining the acoustic index of the second voice content, performing authenticity analysis according to the change of the acoustic index, obtaining the credibility of the second voice content; obtaining the voiceprint of the second voice subject, and performing matching degree detection on the current voiceprint and the historical record of the voiceprint, obtaining the voiceprint consistency of the second voice subject.

[0045] Step S5, providing a comprehensive evaluation formula, the evaluation factors in the comprehensive evaluation formula including at least the water matching deviation, the credibility of the second voice content and the voiceprint consistency, assigning corresponding weight values to the evaluation factors, evaluating the repayment ability of the second voice subject through the calculation result of the comprehensive evaluation formula, and obtaining the credibility score of the second voice subject.

[0046] The technical solutions provided by steps S1 to S5 are based on multi-modal feature comprehensive evaluation of the repayment ability of the debtor, involving the collaborative mode of voice processing, recognition, voice feature recognition, personal income cross verification and intelligent decision output. Step S2 involves voice content extraction and voice subject filtering. The debtor's declared income of the second voice subject is obtained according to the voice recognition result. The debtor's declared income process excludes the speech interference of the first voice subject and the content unrelated to the declared income in the second voice subject. Step S3 involves multi-source data comparison. The debtor's declared income of the second voice subject is compared with the actual income data extracted from the bank, third-party payment platform (WeChat / Alipay) and even water, electricity and coal consumption bills in multiple dimensions. The subjective statement of the second voice subject is compared with the objective real income in real time. A relatively reliable flow matching deviation is obtained, which avoids the influence of the false statement of the second voice subject on the judgment of the financial status. Step S4 is based on acoustic features to analyze the authenticity of the statement of the second voice subject. The voiceprint vector is used to judge whether the second voice subject is consistent with the subject in the historical record. The authenticity of the statement of the second voice subject is judged by acoustic features. Different acoustic features correspond to different physiological responses, such as tension, hesitation or subconscious physiological responses to fraud. Different acoustic features can be used to assist in verifying the authenticity and credibility of the statement of the second voice subject. Finally, step S5 implements comprehensive evaluation. Different weights are assigned to different evaluation factors generated in steps S1 to S4 to form a comprehensive evaluation formula. The multiple key evaluation factors are fused and weighted calculated in the comprehensive evaluation formula. The credibility score with quantitative value is output for auxiliary decision-making, so as to achieve the purpose of automatic, intelligent and multi-dimensional comprehensive evaluation of the repayment ability of the debtor.

[0047] The technical solutions provided by steps S1 to S5 realize the accuracy of debt risk assessment, have very high efficiency, and realize the real-time fraud prevention function of the call process of different voice subjects (debt mediators and debtors). Compared with the prior art, the prior art relies on paper documents and manual experience. The subjective factors of the debt mediator greatly affect the evaluation results. After implementing the technical solutions of the embodiments of the application, the verification process in the debt mediation process becomes a multi-dimensional and intelligent process based on objective data, overcoming the defects of single data source, great subjective factors and lack of authenticity verification method in the prior art, and providing intelligent auxiliary decision-making function for financial institutions and debt subjects for risk control decision-making.

[0048] As shown in Figure 4 , as a further improvement of the embodiments of the application, in step S2, the second voice content of the second voice subject is recognized, and the debtor's declared income of the second voice content is further extracted, which further includes: Step S21, identify the voice content related to the second voice subject's income, and input the voice recognition result into the bank system interface to query the actual income of the second voice subject.

[0049] Step S22, use voice recognition technology to extract the debtor's declared income from the second voice content, and locate the time window of the debtor's declared income through timestamp.

[0050] Step S23, input the extracted debtor's declared income into the bank system through the interface, and the bank system returns the corresponding query result, which is the deviation percentage of the debtor's declared income from the actual income, rather than the specific amount of the debtor's actual income.

[0051] The optimization technical solution provided by steps S21 to S23 identifies the voice content, verifies the actual income of the debtor, and compares the actual income of the debtor with the declared income of the debtor, which realizes real-time income verification and prevents the leakage of the debtor's privacy. Step S21 identifies the voice content related to the second voice subject's income and inputs it into the bank system, step S22 locates the time window of the voice content of the debtor's declared income through timestamp, which can query the deviation percentage of the debtor's declared income from the actual income in real time. Through the deviation percentage, the risk level of the current debtor can be directly reflected. Under the premise of complying with privacy regulations, the complex manual operation that needs to cross departments and systems is compressed into a call to API, which improves the mediation efficiency and avoids the legal risk of privacy compliance, and can be applied to various types of debt dispute mediation scenarios.

[0052] It is worth noting that in the optimization technical solution provided by steps S21 to S23, the bank only returns the deviation percentage, not the specific amount of the debtor's income, which meets the Personal Information Protection Law. For example, the debtor's declared income is 8800 yuan / month, and the actual income of the debtor is 12000 yuan / month. The bank system only returns "deviation XX%" through the interface, and does not return any information about the debtor's monthly income of 12000 yuan. Since it fully complies with privacy protection laws, the embodiments of the present application can be used for subsequent litigation or arbitration after implementation. By calling the voice interaction content of the first and second voice subjects and the deviation percentage as electronic evidence, the authenticity and legality of the evidence chain can be ensured.

[0053] For example, when applying the embodiments of the present application, the process of requesting access to the interface from the bank is as follows: Request path: / api / salary / validation Request method: HTTPS GET request, data submitted in application / json format.

[0054] Call direction: Access to XX platform Request parameters, Header: Parameter name, type, required, description, example value apikey String Y Identifier assigned to the platform cdf78ce2837aac1e90738ebea194efea sign String Y Signature ddccd985e340677725f717ae6d180451 Body Parameter name, type, required, description, example value `start_time` (String Y): The query start time (30 days before the declared time point `t`), formatted as: `YYYY-MM-DDTHH:mm:ss` end_time String Y Query end time (declaring the time point t), format is the same as start_time type String Y Fixed value: Salary deposit (indicates that only salary-related transactions are queried) Claim Float Y: User-declared revenue amount (unit: yuan) Request example: { "start_time": "2025-06-07T10:00:00Z", "end_time": "2025-07-07T10:00:00Z", "type": "Salary Receipt", "claim": 15000.0 } Return parameters: Parameter Name Type Description Example Value actual_avg float: Actual average wage (arithmetic average of wages recorded in the last 30 days). The `deviation_float` parameter declares the deviation between the claim and the actual average wage. The calculation formula is: (claim - actual_avg) / actual_avg `record_count` (int) represents the number of matching payroll records. Status code 1006 msg String Error message: Signature verification failed. Response example: { "code": 1000, "msg": "", "actual_avg": 12000.0, "deviation": 0.25, "record_count": 4 } As Figure 5 shown, as a further improvement of the embodiments of the present application, in step S4, the acoustic indicators of the second voice content are acquired, and the authenticity analysis is performed according to the change of the acoustic indicators, to obtain the second voice content credibility (test credibility), further comprising: Step S41, the acoustic indicators of the second voice content are acquired, and the acoustic indicators include the fundamental frequency (pitch) change and the amplitude (volume) change.

[0055] Step S42, according to the current voiceprint vector and the historical voiceprint vector of the second voice subject, the voiceprint matching degree is obtained, and the voiceprint matching degree satisfies the following formula.

[0056] Step S43, the voiceprint matching degree = 1 - (the cosine distance of the current voiceprint vector - the historical voiceprint vector).

[0057] Wherein, the cosine distance = 1 - cos(θ), wherein cos(θ) is the cosine similarity between the current voiceprint vector and the historical voiceprint vector.

[0058] The optimization technical scheme of steps S41 to S43 provides a sub-process of identity consistency verification based on the acoustic indicators of the second voice content. First, the current acoustic characteristics of the second voice subject are captured, and then the voiceprint vector converted from the current acoustic characteristics is compared with the historical voiceprint vector of the second voice subject. According to the voiceprint matching degree, it is determined whether the current voice subject and the historical voice subject are the same subject. The voiceprint matching degree ranges from 0 to 1. According to the score of the voiceprint matching degree, the consistency of the voice subject can be known, and it is determined whether the current voice subject and the historical voice subject are the same person, to prevent fraud by using a false identity. The optimization technical scheme provided by steps S41 to S43 not only confirms the uniqueness of the identity of the debtor when performing voice content authenticity analysis based on acoustic indicators, but also realizes the integrated operation of identity verification and voice content authenticity analysis, to prevent the technical problems of complicated process, low efficiency and easy misjudgment caused by the separation of traditional debtor subject identity authentication and voice content authenticity analysis.

[0059] As a further improvement of the embodiments of the present application, in step S5, a comprehensive evaluation formula is provided, and the evaluation factors in the comprehensive evaluation formula at least include the flow matching deviation, the second speech content credibility, and the voiceprint consistency, and further include: The environmental noise index is an abnormal sound in the voice interaction. For example, the abnormal sound can be multiple people's voice / keyboard sound, etc. which do not contain an effective amount of information in the voice interaction or the background process of the recording.

[0060] The speech speed fluctuation coefficient satisfies the following formula: The speech speed fluctuation coefficient = min (0.5, the standard deviation of the speech speed of the second speech subject / the basic speech speed of the second speech subject).

[0061] The above optimization technical solution adds the environmental noise index and the speech speed fluctuation coefficient on the basis of the comprehensive evaluation formula provided in steps S1 to S5. The environmental noise index is not simply a measure of the noise decibel size, but can intelligently identify abnormal sounds such as multiple people's voice and keyboard sound. The above "noise" is not only the simple decibel level, and does not exclude the possibility that the current debtor performs voice communication under the inducement or collusion of others, or makes false statements in an involuntary environment. The above situations will reduce the credibility of the debtor's statement content and increase the risk control risk. Through the further identification of the complexity of the recording environment in the present specific embodiment, and the comprehensive consideration of the credibility of the debtor's statement content in combination with the environmental information, the anti-fraud and anti-environmental interference capabilities of the comprehensive evaluation can be enhanced. Therefore, it is necessary to add the speech speed fluctuation coefficient and the environmental noise index in the evaluation factors to improve the risk control capability.

[0062] The speech speed fluctuation coefficient provided by the present specific embodiment quantitatively processes the psychological state and statement content of the debtor when communicating, and can measure the abnormal situation of the debtor relative to the normal speech speed. Abnormally high or abnormally low speech speed may reflect the debtor's emotional fluctuations, anxiety, inexperience in reciting, being directed by others on the spot, involuntary statement, etc. These abnormal situations will all lead to a decrease in the credibility score. For example, the debtor makes a statement that seems reasonable in content, but the speech speed has abnormal fluctuations, and the authenticity should be greatly discounted. Therefore, the consideration of the speech speed fluctuation of the debtor can be verified with other evaluation factors, which embodies the advantages of multi-modal characteristics, forms an organic whole for the comprehensive evaluation formula, enhances the recognition ability of some carefully planned fraudulent behaviors, and improves the voice analysis of the debtor mediation from automation to intelligence. The coordinated combination of multiple evaluation factors and dynamic weight distribution can output a more accurate credibility score.

[0063] As a further improvement of the embodiments of the present application, in some embodiments, the evaluation factors and weight distribution of the comprehensive evaluation formula are set according to the following content: Credibility score = [0.3 × second speech content credibility] + [0.25 × (1 - min(1,|flow matching deviation| / 0.5))] + [0.2 × voiceprint matching degree] - [0.15 × environmental noise index] - [0.1 × speech rate fluctuation coefficient].

[0064] In the aforementioned comprehensive evaluation formula, the credibility score employs a multi-factor weighted fusion approach. Differential weights are assigned to five key factors: credibility of the second voice content, flow matching deviation, voiceprint matching accuracy, environmental noise index, and speech rate fluctuation coefficient. This unified quantification of evaluation factors of different dimensions and natures before outputting a credibility score enhances the objectivity and quantifiable feasibility of debt mediation verification. In this comprehensive evaluation formula, the credibility of the second voice content and voiceprint matching accuracy respectively reflect the debtor's linguistic logic, psychological state, and identity authenticity, and these two, as core elements, carry high weights. Flow matching deviation also carries a relatively heavy weight. The high weighting of these evaluation factors reflects the emphasis on objective factual data, indicating that the debt mediation verification process should be based on facts and guided by law. Environmental noise index and speech rate fluctuation coefficient are corrections and deductions to objective factual data, serving as auxiliary and early warning indicators. Although their weights are relatively low, they still play a certain regulatory and corrective role in the comprehensive evaluation formula, avoiding the adverse impact of single environmental or behavioral fluctuations on the overall score. This ensures that the comprehensive evaluation formula maintains a balance between sensitivity and stability, giving it a hierarchy of importance. The final result is based on a comprehensive assessment of the debtor's repayment ability, rather than being limited to a few evaluation factors. The comprehensive evaluation formula provided in this application uses estimated factors from multiple dimensions for comprehensive evaluation. Each estimated factor contributes to characterizing the debtor's repayment ability in debt mediation verification. Relying on human experience cannot achieve the balance and integration of the above-mentioned multi-dimensional comprehensive information. Therefore, the comprehensive evaluation formula provided in this application provides a reliable and intelligent evaluation method for the intelligent decision-making output of debt mediation verification.

[0065] For example, the embodiments of this application are applied to a comprehensive repayment ability assessment model. The assessment objective of this comprehensive repayment ability assessment model is to quantify the credibility of a user's repayment ability based on multi-dimensional data and identify fraud risks.

[0066] Evaluation factors and weighting: Factor weight explanation The credibility of a lie detector is 30%. Physiological indicators during user statements (fundamental frequency (pitch) change / amplitude (volume) change) Authenticity analysis (e.g., chemical analysis) A 25% discrepancy in payroll records indicates the degree of difference between declared income and actual payroll records. Voiceprint consistency 20% - Current voiceprint matches historical records. Ambient Noise Index 15% Disturbance intensity of abnormal sound (multiple voices / keyboard sounds, etc.) in the recording background Speech Speed Fluctuation Coefficient 10% Standard deviation of speech speed when answering key questions Comprehensive Evaluation Formula: Credibility Score = [0.3 × Polygraph Score] + [0.25 × (1 - min(1, |Deviation| / 0.5))] + [0.2 × Voiceprint Matching Degree] - [0.15 × Ambient Disturbance Index] - [0.1 × Speech Speed Fluctuation Coefficient] Parameter Description: Polygraph Score [0-1]: Composite indicator based on fundamental frequency (pitch) change + amplitude (volume) change score = 0.6* (fundamental frequency (pitch) change) + 0.4* (1 - amplitude (volume) change) Stream Deviation Penalty, using piecewise function to suppress high deviation: if |deviation| ≤ 0.3: weight = 1 - |deviation| else: weight = 0.7 * (1 - min(1, |deviation|)) # High deviation penalty Voiceprint Matching Degree [0-1]: Matching Degree = 1 - (cosine distance of current voiceprint vector - historical voiceprint vector) Ambient Disturbance Index [0-1]: Disturbance Index = min(1, number of detected abnormal sound sources / 3 + average noise decibels / 60) Speech Speed Fluctuation Coefficient [0-0.5]: Coefficient = min(0.5, standard deviation of speech speed when answering key questions / basic speech speed) For the method steps disclosed in the above embodiments, the method steps are expressed as a combination of a series of actions for the purpose of simple description, but those skilled in the art should know that the application embodiments are not limited by the order of the described actions, because according to the application embodiments, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily necessary for the application embodiments.

[0067] Any process or method described in a flowchart or otherwise described herein can be understood as representing a module, segment, or portion of code that includes one or more executable instructions for implementing specific logical functions or steps in the process, and the application embodiments include the possibility that the described processes can be implemented in alternative order, including substantially simultaneously, in reverse order (where possible), or in other sequences, that the various blocks or steps can be implemented in one or more alternative orders or in parallel, or that described functionality can be implemented at least in part outside the order shown or discussed, including in processes running at substantially the same time, that computer instructions can be implemented in hardware, software, or a combination of the two, and that the described embodiments of the application are not limited to any particular computer or machine platform, unless otherwise specified.

[0068] As shown in Figure 6 the application embodiments provide a multi-modal feature-based debt mediation verification method, and also provide a multi-modal feature-based debt mediation verification system corresponding thereto, for implementing the multi-modal feature-based debt mediation verification method described in any specific embodiment of the application, including: a voice information and subject acquisition module, which acquires voice information, identifies a first voice subject and a second voice subject in the voice information, the first voice subject being a debt mediator, and the second voice subject being a debtor; a voice interaction content cutting and filtering module, which cuts voice interaction content of the first voice subject and the second voice subject, filters first voice content of the first voice subject in the voice interaction content, performs voice recognition on second voice content of the second voice subject, and extracts a debtor income declaration of the second voice content; a flow matching deviation calculation module, which queries actual income of the debtor of the second voice subject according to the debtor income declaration, compares the actual income of the debtor with the debtor income declaration, and obtains a flow matching deviation; a voice content credibility analysis module, which acquires an acoustic index of the second voice content, and obtains second voice content credibility according to a change of the acoustic index; a voiceprint consistency acquisition module, which acquires a voiceprint of the second voice subject, performs matching degree detection on the current voiceprint and a historical record voiceprint, and obtains voiceprint consistency of the second voice subject; a credibility scoring module, which provides a comprehensive evaluation formula, evaluation factors in the comprehensive evaluation formula at least including the flow matching deviation, the second voice content credibility, and the voiceprint consistency, assigns corresponding weight values to the evaluation factors, evaluates the repayment ability of the second voice subject through a calculation result of the comprehensive evaluation formula, and obtains a credibility score of the second voice subject.

[0069] It is worth noting that although only some basic function modules are disclosed in the embodiments of the present application, it does not mean that the composition of the system is limited to the above basic function modules. On the contrary, the meaning expressed by the embodiments of the present application is that on the basis of the above basic function modules, those skilled in the art can add one or more function modules to form infinite embodiments or technical solutions in combination with the prior art. That is, the system is open rather than closed, and the protection scope of the claims of the embodiments of the present application cannot be limited to the disclosed basic function modules. At the same time, in order to describe conveniently, the above device is described as various units, modules. Of course, in the implementation of the embodiments of the present application, the functions of the units and modules can be realized in the same software and / or hardware.

[0070] The embodiments of the system described above are only illustrative, for example: the various function modules, units or subsystems in the system can or can not be physically separated, or can or can not be physical units, that is, they can be located in the same place or distributed to multiple different systems and their subsystems or modules. Those skilled in the art can select part or all of the function modules, units or subsystems to achieve the purpose of the embodiments of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0071] As shown in Figure 7 On the basis of providing the debt mediation verification method and system based on multi-modal features, the embodiments of the present application also provide corresponding electronic devices and storage media: An electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; the memory stores a computer program, when the computer program is executed by the processor, the processor executes the steps of the debt mediation verification method based on multi-modal features.

[0072] A computer-readable storage medium stores a computer program executable by an electronic device, when the computer program runs on the electronic device, the electronic device executes the steps of the debt mediation verification method based on multi-modal features.

[0073] Legend: electronic device 500, external device 514, processor 516, bus 518, network adapter 520, I / O interface 522, display device 524, memory 528, RAM 530, cache 532, storage system 534, program / utility 540, program module 542.

[0074] Figure 7The electronic device shown is merely one example and should not be taken as limiting the scope of the present embodiments. The electronic device can be a device of the electronic products described above that use the debt mediation verification method as a working basis. The electronic device 500 is in the form of a general computing device. The components of the electronic device 500 can include, but are not limited to, one or more processing units or processors 516, a memory 528, and a bus 518 that connects the various system components, including the memory 528 and the processor 516. The bus 518 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or local bus using any of a variety of bus structures. Examples of these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus. The electronic device 500 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 500 and include both volatile and nonvolatile media, removable and non-removable media. The memory 528 can include computer system readable media in the form of volatile memory, such as RAM 530 and / or cache 532. The electronic device 500 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 534 can be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown and typically called a "hard drive"). Although not shown, a storage system 534 can include a magnetic disk drive for reading from and writing to a magnetic disk, and an optical disk drive for reading from or writing to a removable non-volatile media such as an optical disk (e.g., a CD-ROM, DVD-ROM or other optical media). In these instances, each can be connected to the bus 518 by one or more data media interfaces. The memory 528 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of various implementations of the present embodiments. Program / utility 540, having a set (at least one) of program modules 542, can be stored in, for example, memory 528 by way of example, such program modules 542 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or a combination can include implementation of a network environment. The program modules 542 generally carry out the functions and / or methodologies of the embodiments described herein.The electronic device 500 can also communicate with one or more external devices 514 such as a keyboard or pointing device, a display 524, etc.; other devices such as are necessary to enable operation of the electronic device 500; and / or a network adaptation 520 that enables the electronic device 500 to communicate with one or more other computing devices. Such communication can occur via an I / O interface 522. Still yet, the electronic device 500 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adaptation 520. The network adaptation 520 communicates with the other modules of the electronic device 500 through the bus 518. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the electronic device 500. Such hardware would include, but is not limited to, a microcode, a device driver, a redundant processing unit, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc. The processor 516 performs a variety of functions including providing an interface between the various modules of the electronic device 500 and the memory 528, and executing program code stored in the memory 528 to perform various functions, such as implementing the methods provided by any one or more of the embodiments described herein.

[0075] Any of the technical features of the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described in the above description. However, any combination of the technical features of the above embodiments is considered to be within the scope of the disclosure.

[0076] In the description of the present application embodiments, reference has been made to descriptive terms such as "one embodiment", "an example", "a specific example" etc. Such terminology means that a particular feature, structure, material or characteristic being described is included in at least one embodiment or example of the present application. The illustrative examples given are not necessarily mutually exclusive, and the descriptive terms are not necessarily mutually exclusive. Moreover, the illustrative examples given can be combined in any suitable manner.

[0077] In addition, the technical solutions of the various embodiments of the present application can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can implement it. When the combination of technical solutions appears contradictory or unimplementable, it should be considered that the combination of technical solutions does not exist and is not within the protection scope claimed by the present application.

[0078] All the features disclosed in this application, or all the steps of any method or process specified in this application may be combined in any combinations, except combinations where at least some of the features and / or steps are mutually exclusive. Any one feature disclosed in this application may be replaced by an alternative feature or second equivalent feature unless expressly stated otherwise. The combination of features disclosed in any specific embodiment of the application is intended to be included within the scope of the application, unless specifically stated otherwise. Throughout this specification the word "comprise" or variations such as "comprises" or "comprising" will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

[0079] It should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application. Although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some or all of the technical features, and these modifications or replacements do not cause the corresponding technical solutions to deviate from the scope of the embodiments of the present application.

Claims

1. A method for debt mediation verification based on multi-modal features, characterized in that, The method comprises the following steps: acquiring voice information, identifying the first voice subject and the second voice subject in the voice information, the first voice subject being a debt mediator, and the second voice subject being a debtor; cutting the voice interaction content of the first voice subject and the second voice subject, filtering the first voice content of the first voice subject in the voice interaction content, performing voice recognition on the second voice content of the second voice subject, and extracting the debtor's declared income of the second voice content; querying the actual income of the debtor according to the debtor's declared income, comparing the actual income of the debtor with the debtor's declared income, and obtaining the flow matching deviation; acquiring the acoustic index of the second voice content, performing authenticity analysis according to the change of the acoustic index, and obtaining the credibility of the second voice content; acquiring the voiceprint of the second voice subject, performing matching degree detection on the current voiceprint and the historical voiceprint, and obtaining the voiceprint consistency of the second voice subject; providing a comprehensive evaluation formula, the evaluation factors in the comprehensive evaluation formula including at least the flow matching deviation, the credibility of the second voice content, and the voiceprint consistency, assigning corresponding weight values to the evaluation factors, evaluating the repayment ability of the second voice subject through the calculation result of the comprehensive evaluation formula, and obtaining the credibility score of the second voice subject. 2.The method of claim 1, wherein, In the process of separating the first voice subject and the second voice subject according to the subject of the voice information, the first voice content of the first voice subject and the acoustic index of the second voice subject are acquired respectively, the first voice content is a legal term related to debt mediation, and the acoustic index includes fundamental frequency change and amplitude change. 3.The multi-modal feature based debt mediation verification method of claim 1, wherein, The voice recognition on the second voice content of the second voice subject and the extraction of the debtor's declared income of the second voice content further comprise: identifying the voice content related to the income of the second voice subject, connecting the voice recognition result to the bank system interface, and querying the actual income of the debtor of the second voice subject.

4. The method of claim 3, wherein, The voice recognition result is connected to the bank system interface to query the actual income of the debtor of the second voice subject, specifically: extracting the debtor's declared income from the second voice content by using voice recognition technology, and positioning the time window of the debtor's declared income by time stamp; inputting the extracted debtor's declared income into the bank system through the interface, and returning the corresponding query result by the bank system, the query result being the deviation percentage of the debtor's declared income and the actual income of the debtor. 5.The multi-modal feature based debt mediation verification method of claim 1, wherein, The acoustic index of the second voice content is acquired, authenticity analysis is performed according to the change of the acoustic index, and the credibility of the second voice content is obtained, further comprising: acquiring the acoustic index of the second voice content, the acoustic index including fundamental frequency change and amplitude change; obtaining the voiceprint matching degree according to the current voiceprint vector and the historical voiceprint vector of the second voice subject, the voiceprint matching degree satisfying the following formula: voiceprint matching degree = 1 - (cosine distance of current voiceprint vector - historical voiceprint vector).

6. The method for debt mediation verification based on multi-modal features according to claim 1, wherein, The comprehensive evaluation formula further comprises: An environmental noise index, the environmental noise index being an abnormal sound in the voice interaction; A speech speed fluctuation coefficient, the speech speed fluctuation coefficient satisfying the following formula: The speech speed fluctuation coefficient = min(0.5, the second speech subject's speech speed standard deviation / the second speech subject's basic speech speed).

7. The method of claim 6, wherein, The comprehensive evaluation formula is specifically: The credibility score = [0.3 x the second speech content credibility] + [0.25 x (1-min(1, |stream matching deviation| / 0.5))] + [0.2 x the voiceprint matching degree] - [0.15 x the environmental noise index] - [0.1 x the speech speed fluctuation coefficient].

8. A multi-modal feature based debt mediation verification system, characterized in that, A method for implementing the multi-modal feature-based debt mediation verification method of any one of claims 1 to 7, comprising: A speech information and subject acquisition module acquires speech information, identifies a first speech subject and a second speech subject in the speech information, the first speech subject being a debt mediator, and the second speech subject being a debtor; A voice interaction content cutting and filtering module cuts the voice interaction content of the first speech subject and the second speech subject, filters the first speech content of the first speech subject in the voice interaction content, performs speech recognition on the second speech content of the second speech subject, and extracts the debtor's declared income of the second speech content; A stream matching deviation calculation module queries the debtor's actual income of the second speech subject according to the debtor's declared income, compares the debtor's actual income with the debtor's declared income, and obtains the stream matching deviation; A speech content credibility analysis module acquires the acoustic index of the second speech content, analyzes the authenticity according to the change of the acoustic index, and obtains the second speech content credibility; A voiceprint consistency acquisition module acquires the voiceprint of the second speech subject, performs matching degree detection on the current voiceprint and the historical voiceprint, and obtains the voiceprint consistency of the second speech subject; A credibility score module provides a comprehensive evaluation formula, the evaluation factors in the comprehensive evaluation formula at least including the stream matching deviation, the second speech content credibility, and the voiceprint consistency, the evaluation factors are assigned corresponding weight values, the repayment ability of the second speech subject is evaluated through the calculation result of the comprehensive evaluation formula, and the second speech subject credibility score is obtained.

9. An electronic device, comprising: It comprises: A processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the method of any one of claims 1 to 7.

Images