Call risk assessment method, apparatus, device, and medium

Abstract

The present application relates to the technical field of communication, and discloses a call risk assessment method, device, equipment and medium, comprising: acquiring call audio data; converting the call audio data into call text data based on voice recognition technology; generating semantic feature data based on the call audio data and the call text data; and generating a risk assessment result of the call audio data based on the call text data, the semantic feature data, preset business rule data and a preset risk assessment model. By integrating voice recognition, multi-modal semantic analysis, dynamic rule matching and machine learning inference, the obvious and hidden risks in the call content are comprehensively and automatically assessed, providing quantifiable, traceable and executable risk control capabilities for high-compliance industries such as finance and medicine, and effectively ensuring business compliance and operation safety.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method, apparatus, device, and medium for assessing call risks. Background Technology

[0002] Currently, risk monitoring of supplier-customer calls typically involves manual sampling of call recordings to assess potential risks. However, this approach has significant drawbacks: firstly, manual sampling is inefficient, limited by manpower and time costs, resulting in limited actual coverage and a large number of call records going unchecked, leading to a high rate of missed risk information and hindering comprehensive protection of business operations; secondly, manual review struggles to conduct in-depth semantic and sentiment analysis of call content, failing to effectively address complex and ever-changing business scenarios. Particularly in the financial sector, such as bank customer service, insurance sales, and loan management, traditional manual sampling methods cannot identify high-risk behaviors like financial fraud inducements, illegal promises, and customer information leaks in real time, leading to the continuous accumulation of compliance risks and financial risks. Similarly, in the healthcare sector, manual sampling struggles to systematically monitor key issues such as medical misrepresentation, unauthorized disclosure of privacy data, and medication safety risks in calls involving medical consultations, drug after-sales service, and patient follow-ups, failing to meet the refined requirements of medical compliance and service quality control. Summary of the Invention

[0003] This invention provides a method, apparatus, computer equipment, and medium for call risk assessment to solve the technical problems of low efficiency, limited coverage, and inability to deeply analyze semantics and sentiment in manual sampling.

[0004] Firstly, a method for assessing call risks is provided, including: Obtain call audio data; Based on speech recognition technology, call audio data is converted into call text data; Semantic feature data is generated based on call audio and text data. Based on call text data, semantic feature data, preset business rule data, and preset risk assessment models, risk assessment results are generated for call audio data.

[0005] Secondly, a call risk assessment device is provided, comprising: The acquisition module is used to acquire call audio data; The data conversion module is used to convert call audio data into call text data based on speech recognition technology; The first generation module is used to generate semantic feature data based on call audio data and call text data; The second generation module is used to generate risk assessment results for call audio data based on call text data, semantic feature data, preset business rule data, and preset risk assessment models.

[0006] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described call risk assessment method.

[0007] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described call risk assessment method.

[0008] The aforementioned call risk assessment method, device, computer equipment, and storage medium convert call audio into structured text, while simultaneously extracting multi-dimensional features by integrating sound features and semantic information. Based on this, and combined with a constantly updated business rule base and a continuously learning and optimizing risk assessment model, a highly efficient and accurate risk assessment mechanism is constructed by integrating speech recognition, multimodal semantic analysis, dynamic rule matching, and machine learning inference. This enables a comprehensive and automated assessment of both explicit and implicit risks in call content, significantly expanding the coverage of risk identification and accelerating response speed. It transforms the lag and low coverage of traditional manual spot checks into full-scale real-time monitoring. Furthermore, by leveraging multi-dimensional feature fusion, it greatly reduces false positives and false negatives, providing quantifiable, traceable, and executable risk management capabilities for industries with high compliance requirements such as finance and healthcare, effectively ensuring business compliance and operational security. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of an application environment for a call risk assessment method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a call risk assessment method according to an embodiment of the present invention; Figure 3 yes Figure 2 A schematic diagram of a specific implementation method for step S20; Figure 4 yes Figure 2 A schematic diagram of a specific implementation method for step S30; Figure 5 yes Figure 4 A flowchart illustrating a specific implementation of step S33; Figure 6 This is a schematic diagram of a call risk assessment device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 8 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] The call risk assessment method provided in this embodiment of the invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can acquire call audio data; based on speech recognition technology, it converts the call audio data into call text data; based on the call audio data and call text data, it generates semantic feature data; based on the call text data, semantic feature data, preset business rule data, and preset risk assessment model, it generates a risk assessment result for the call audio data. In this invention, call audio is converted into structured text, while simultaneously fusing sound features and semantic information to extract multi-dimensional features. Based on this, combined with a business rule base that can be updated at any time and a continuously learning and optimized risk assessment model, an efficient and accurate risk assessment mechanism is constructed by integrating speech recognition, multimodal semantic analysis, dynamic rule matching, and machine learning inference. This allows for a comprehensive and automated assessment of both explicit and implicit risks in call content, significantly expanding the coverage of risk identification and accelerating response speed. It transforms the lag and low coverage of traditional manual spot checks into full-scale real-time monitoring. Simultaneously, through multi-dimensional feature fusion, it greatly reduces false positives and false negatives, providing quantifiable, traceable, and executable risk management capabilities for industries with high compliance requirements such as finance and healthcare, effectively ensuring business compliance and operational security. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0013] Please see Figure 2 As shown, Figure 2A flowchart illustrating the call risk assessment method provided in this embodiment of the invention includes the following steps: S10: Obtain call audio data.

[0014] The call risk assessment method provided by this invention can be applied to intelligent interaction engines such as intelligent customer service or customer relationship management in various application scenarios. Intelligent interaction engines are typically implemented through a server-side component, which can acquire real-time audio data streams during calls between suppliers and customers. For example, in the insurance application field, it can acquire real-time audio of customer-agent conversations. In the medical scenario, it can acquire audio of doctor-patient telephone consultations.

[0015] For example, in a credit card installment sales call within a financial scenario, the real-time audio data includes the following dialogue segment: "Hello Mr. Wang, we noticed your last bill was high, so we're offering you a three-month interest-free installment plan. This offer is limited, and if you apply today, you'll receive an additional 200 yuan discount on the handling fee. Don't worry about repayment pressure; the system will automatically deduct the amount from your savings card ending in 6688. Rest assured, this installment plan will not affect your future credit limit increases."

[0016] S20: Based on speech recognition technology, it converts call audio data into call text data.

[0017] In this step, the real-time acquired call audio data is processed by a speech recognition engine deployed on the server. This engine uses a deep neural network model to convert the continuous audio signal stream into a timestamped text sequence in real time, providing structured text data input for subsequent risk analysis.

[0018] In one embodiment of this application, such as Figure 3 As shown, a specific text conversion scheme is provided. In step S20, which is based on speech recognition technology, the call audio data is converted into call text data, specifically including the following steps S21-S23: S21: Segment the call audio data to generate an audio segment sequence.

[0019] In this step, based on the actual application scenario and performance requirements, an appropriate segmentation strategy is selected to divide the continuous call audio data, generating a sequence of audio segments with millisecond-level timestamps. Each audio segment is appended with the speaker's voiceprint identifier and background noise feature metadata, forming a standardized set of audio data units that can be processed in a distributed parallel manner.

[0020] Optionally, a fixed-duration sliding window can be used for segmentation. The acquired call audio data is segmented in real time using a preset fixed-duration sliding window (e.g., a 2-second window with a 1-second step) to generate a sequence of audio segments with millisecond-level timestamps. Each audio segment contains its start and end time information in the original audio stream, forming a set of standardized audio data units that can be processed by subsequent speech recognition models.

[0021] Furthermore, to improve the accuracy of speech recognition and semantic analysis, a segmentation strategy combining time windows and speech activity detection can be adopted to adaptively and intelligently segment the call audio stream. Specifically, a preset fixed-duration sliding window (e.g., a 2-second window with a 1-second step) is used to initially segment the continuous call audio stream. Simultaneously, a dynamic endpoint detection algorithm based on speech activity detection technology is used to finely identify and segment active speech segments from non-silent segments. For detected active speech segments, if the continuous duration exceeds a preset threshold (e.g., more than 3 seconds), further segmentation is performed using a fixed-duration sliding window (e.g., a 2-second window with a 1-second step), generating multiple overlapping or continuous speech segments to ensure that the duration of each segment is suitable for the input requirements of the subsequent speech recognition model. If there are multiple adjacent active speech segments with short continuous durations (e.g., less than 0.5 seconds) and small time intervals (e.g., less than 0.3 seconds), they are merged into one speech segment to maintain semantic integrity and avoid semantic fragmentation due to excessive segmentation. For longer silent segments, they are separately segmented into silent segments for background noise analysis. Finally, a sequence of audio clips with millisecond-level timestamps is generated.

[0022] For example, in financial scenarios, such as credit card installment sales calls, a 2-second sliding window is used to segment the continuous call stream, and voice activity detection technology is used to accurately distinguish the salesperson's marketing script segments, such as "interest-free installment this period," and the customer's silent inquiry segments. The generated audio segments are accompanied by millisecond-level timestamps (e.g., 145210-147580 milliseconds), voiceprint identifiers (e.g., agent ID A038), and background noise spectrum characteristics (keyboard typing sound distribution), forming standardized audio data units with complete metadata. In medical scenarios, such as online medical consultations, a 2-second sliding window is used to segment the continuous call stream, and voice activity detection technology is used to distinguish doctor inquiry segments, such as "How is your blood pressure control lately?", and patient description segments, such as "My blood pressure was 150 / 95 this morning." Each segment is labeled with a millisecond-level timestamp (e.g., 320540-322120 milliseconds), a role voiceprint identifier (licensed physician Li XX), and environmental acoustic characteristics (background TV sound distribution).

[0023] By employing the above method, a segmentation strategy combining time windows and voice activity detection effectively balances memory pressure and real-time requirements in long audio processing.

[0024] S22: Use a speech recognition model to perform speech recognition on audio segment sequences and generate text segment sequences.

[0025] In this step, the processed audio segment sequence is input into a pre-trained large-scale speech recognition model. This model, optimized with a connection-time classification loss function and specifically trained for multi-speaker scenarios, can accurately recognize speech with different accents and technical terms from fields such as finance and medicine. For each audio segment, the model outputs the most probable text transcription result, along with its corresponding temporal boundary confidence level, ultimately generating a structured text segment sequence.

[0026] By using the above methods to perform text conversion with a speech recognition model, the recognition adaptability in complex scenarios is significantly enhanced.

[0027] S23: Perform splicing and time alignment on the text fragment sequence to generate call text data.

[0028] In this step, after obtaining discrete text fragments, the recognition results that overlap or are adjacent in time are first merged using the prefix tree algorithm. Then, the large language model is used to intelligently complete the missing parts of the text caused by unclear pronunciation, accent, or noise. Subsequently, based on the accurate time stamp and dialogue turn order, the text data is reorganized into a standard text data with complete time information and speaker differentiation with a clear structure.

[0029] By employing the above methods and a semantic coherence reconstruction mechanism, the temporal accuracy and logical consistency of text data are ensured, thus establishing a highly reliable data foundation for subsequent risk analysis.

[0030] S30: Generate semantic feature data based on call audio data and call text data.

[0031] In this step, after obtaining the structured text sequence, a large language model is used to perform context-aware vectorization encoding on the call text data. Simultaneously, acoustic feature vectors such as tone, speech rate, and pauses extracted from the audio data are fused. Multimodal feature alignment and fusion are achieved through a cross-modal attention mechanism to generate a comprehensive feature vector that integrates semantic understanding and speech emotion. This vector contains multi-dimensional semantic representation information such as dialogue intent category, key entities and their relationships, emotional polarity intensity, and overall dialogue logical structure.

[0032] For example, in a financial credit approval scenario, when a customer states, "My monthly after-tax income is about 40,000 yuan, plus rental income," the large language model encodes this as a high-dimensional vector. Simultaneously, combining the stable and confident acoustic features of the speech, it generates a fused feature vector containing the entity relationship between income declaration and multiple sources, the intent of explaining financial status, and positive sentiment. Subsequently, when the approver asks, "Do you have bank statements showing your rental income?" the customer replies, "Well... I haven't applied for a separate card yet," with an abnormal 0.8-second pause. This pause detects tension features through voiceprint fluctuations, generating a fused feature vector of "lack of evidence - hesitant response." In a medical medication guidance scenario, when a pharmacist tells a patient, "This antibiotic needs to be taken continuously for seven days," this is encoded as a standard medication guidance vector. However, when a patient asks, "Can I stop taking it earlier if I get better?" the pharmacist's reply, "Actually... usually three days is enough," is accompanied by a drop in tone. This, combined with the uncertainty expressed in the acoustic features and the contradiction between "three days" in the text and the prescription's "seven days," generates a feature vector of "medication cycle conflict - non-standard advice." When a patient asks, "Will it conflict with antihypertensive drugs?" the pharmacist's reply, "It should be fine," is detected as having a weakly certain tone. Combined with the text features lacking specific drug interaction explanations, this forms a feature vector of "insufficient drug interaction information."

[0033] In one embodiment of this application, such as Figure 4 As shown, a specific feature extraction scheme is provided. In step S30, semantic feature data is generated based on the call audio data and call text data, specifically including the following steps S31-S33: S31: Extract voiceprint features from the call audio data and combine them with turn-by-turn analysis to generate speaker identification data.

[0034] In this step, the acquired call audio data is first preprocessed, including audio noise reduction, silent segment detection, and audio frame segmentation. Then, a deep neural network model is used to extract voiceprint features from the preprocessed audio data. This deep neural network model employs an architecture based on deep clustering or metric learning, capable of mapping each audio frame to a high-dimensional voiceprint feature vector space, generating a discriminative voiceprint feature vector. For each continuous speech activity region, the voiceprint feature vectors of all audio frames within that region are aggregated (e.g., by averaging or using an attention mechanism for weighted aggregation) to generate a voiceprint feature representation for that speech segment.

[0035] Subsequently, the speaker is distinguished and identified based on the speaker switching time point. Combining the similarity calculation of the voiceprint feature vectors of the preceding and following speech segments, if the similarity of the voiceprint features of two adjacent speech segments is lower than a preset threshold (e.g., cosine similarity less than 0.7), it is confirmed as a switch between different speakers; otherwise, it is considered as continuous speech by the same speaker.

[0036] Furthermore, the voiceprint feature vector is compared with the voiceprint feature centers of existing speaker categories for similarity. If the similarity with an existing category exceeds a preset threshold, the speech segment is assigned to that speaker category, and the voiceprint feature center of that category is updated. If the similarity with all existing categories is below the threshold, a new speaker category is created for the speech segment. In this way, as the call progresses, the voiceprint categories of all participating speakers are gradually constructed. After speaker differentiation and clustering, a unique identity label is generated for each speaker. Simultaneously, based on the results of turn-taking analysis, each speech segment is labeled with its corresponding speaker identity label, and the voiceprint matching confidence level is calculated. The voiceprint matching confidence level is determined based on the similarity between the speech segment's voiceprint feature vector and the voiceprint feature center of its assigned category; the higher the similarity, the higher the confidence level. Finally, speaker identification data containing the speaker's unique identity label and its voiceprint matching confidence level is generated for subsequent role text data generation.

[0037] For example, in a financial credit approval scenario, voiceprint features are extracted from audio recordings of conversations involving two participants: a credit manager (a steady male voice) and a customer (a female voice with a regional accent). By analyzing the silence intervals in the conversation, such as an average 1.2-second pause before the customer answers a question, the voiceprints of the two parties are separated, and corresponding identity data is generated, including: credit manager (voiceprint matching confidence score 0.94) and customer (voiceprint matching confidence score 0.99). In a multidisciplinary medical consultation scenario, the chief physician (authoritative tone), resident physician (fast speaking speed), and patient's family member (with choked-up features) are separated from audio recordings in a noisy conference room. By detecting turn-taking patterns (e.g., the physician's average continuous speaking time is 8 seconds vs. the family member's 3 seconds), identity data is generated: chief physician (confidence score 0.96), resident physician (confidence score 0.91), and family member (confidence score 0.82), providing a basis for subsequent medical liability determination.

[0038] S32: Generate role text data based on speaker identification data and call text data.

[0039] In this step, firstly, based on the voiceprint feature vectors in the speaker identification data, a voiceprint clustering algorithm is used to group audio segments with similar acoustic features into the same speaker category, generating a unique identity label for each speaker. Based on this, role mapping is performed using preset role templates for the call scenario. For example, in a financial customer service scenario, preset role categories include customer service agents, customers, and system announcers; in a medical consultation scenario, preset role categories include doctors, patients, and family members.

[0040] Furthermore, by analyzing turn-taking patterns, the system identifies the alternation of roles in the dialogue. Based on silence interval detection and speaker switching time points, it automatically divides the dialogue into rounds and binds the text content of each round to a corresponding speaker identity tag. Subsequently, a knowledge-based mapping method is used to associate the speaker identity tags with known information in the business system. Finally, the above association results are integrated to generate a structured role text data sequence containing dialogue texts from multiple roles such as customer, customer service, expert, and system announcements, along with their millisecond-level timestamps. Each text fragment is labeled with a unique speaker identity tag, a predefined role category, a speaking time range, and text content.

[0041] For example, in a financial sales scenario, the voiceprints of credit managers and customers are aligned with the real-time transcribed text at the millisecond timeline. By detecting silence intervals exceeding 0.8 seconds, dialogue turns are automatically divided, generating structured sequence data: "01:23-01:45; Financial Manager: This product has an annualized return of 5.2%", "01:47-02:11; Customer: Is there any risk?", "02:13-02:20; Financial Manager: Investment involves risk, and past performance does not guarantee future results", forming a complete multi-role dialogue flow that labels the speaker's identity and business role.

[0042] By using the above method, bimodal speaker recognition technology that integrates voiceprint biometrics and text semantics is used to generate role text, effectively solving the problems of role confusion and identity misjudgment caused by similar voice timbres, multiple people having cross-talk, or environmental interference in traditional solutions.

[0043] S33: Perform natural language processing on character text data based on a large language model to generate semantic feature data.

[0044] In this step, the text data that distinguishes speaker roles is encoded using a pre-trained large language model for contextual understanding. Attention weights are adjusted in conjunction with role type information to extract multi-dimensional semantic features, including dialogue intent and emotional inclination, while maintaining the correspondence between these features and the original speaker roles.

[0045] By employing the above methods, speaker recognition that integrates voiceprints and text, and semantic feature extraction based on role perception, the subsequent risk assessment model can accurately distinguish the responsibility boundaries and behavioral intentions of different dialogue subjects, thereby significantly improving the accuracy of risk assessment.

[0046] In one embodiment of this application, such as Figure 5 As shown, a specific semantic feature generation scheme is provided. In step S33, that is, natural language processing is performed on the character text data based on the large language model to generate semantic feature data, specifically including the following steps S331-S334: S331: Segment the character text data based on a preset segmentation strategy to generate a set of sentences.

[0047] In this step, a dual segmentation strategy of dialogue punctuation and silence intervals is adopted to perform sentence-level intelligent segmentation on text data with role labels while maintaining the continuity of speaker identity. Specifically, basic sentence units are first divided based on punctuation marks such as question marks, exclamation marks, and full periods. Then, secondary fine segmentation is performed by detecting speech gaps exceeding a preset time threshold (such as 0.5 seconds), and finally, a standardized set of sentences that maintains the original time order and role attribution is generated.

[0048] For example, in a financial insurance claims follow-up scenario, the dialogue text includes customer service and customer role labels. First, the customer service statement "Have you received your claim payment?" is segmented using punctuation marks as separators. Then, by detecting a 1.2-second silence interval, the long customer sentence "Received... but the amount is incorrect, the hospitalization allowance is missing" is segmented a second time. There is no obvious silence at the comma, but a 2-second pause at the end of the sentence triggers segmentation, generating a standardized set of statements: Customer Service: "Have you received your claim payment?"; Customer: "Received"; Customer: "But the amount is incorrect, the hospitalization allowance is missing".

[0049] S332: Based on a large language model, perform intent recognition on a set of statements and generate intent feature data.

[0050] In this step, the sentence set after sentence segmentation is input into a large language model that has been optimized for business scenarios. Through multi-task learning, intent judgment and key information extraction are performed simultaneously to output vectorized intent features of business operation intent, risk-related intent, and key entity filling results.

[0051] For example, in a financial credit card upgrade scenario, in response to a customer's statement, "Can you increase my credit limit to 100,000? I urgently need to pay for renovations," the large language model, specifically optimized for the business scenario, outputs multi-dimensional intent features: the business operation intent is "credit limit adjustment application," the risk-related intent is "vague specific purpose of funds," and the entity slot filling result is "application amount: 100,000 yuan, purpose: renovation funds." This feature is associated with the speaker role as the customer. In response to the customer service representative's reply, "We can expedite your approval process, but you need to purchase a dedicated insurance policy first," the model extracts intent features: the business operation intent is "cross-recommendation of value-added services," the risk-related intent is "bundled sales inducement behavior," and the entity slot is "channel type: expedited approval, associated product: insurance product." This feature is associated with the role as the customer service representative. In a medical internet consultation scenario, for a patient's complaint, "I've had a headache for three days, and I've taken ibuprofen but it hasn't worked," the big language model outputs structured intent features: the medical complaint intent is "symptom duration and medication feedback," the risk-related intent is "self-medication," and the entity slot filling result is "symptoms: headache, duration: 3 days, medication used: ibuprofen, medication effect: ineffective." This feature is bound to the role of the patient. For the reply generated by the consultant, "It is recommended that you go to the neurology department immediately to rule out the possibility of cerebrovascular disease," the model extracts intent features: the medical suggestion intent is "specialist referral guidance," the risk-related intent is "implied severity of potential disease," and the entity slot is "suggested department: neurology, diagnosis to be ruled out: cerebrovascular disease." This feature is associated with the role of the consultant.

[0052] S333: Based on intent feature data, a preset sentiment lexicon, and preset intent type weight coefficients, perform sentiment calculations on a set of sentences to generate sentiment feature data.

[0053] In this step, each statement in the statement set undergoes text preprocessing, including word segmentation, part-of-speech tagging, and syntactic analysis, to facilitate subsequent sentiment word matching. Then, a pre-defined sentiment lexicon is used to match sentiment words with the processed statements. This lexicon contains a large number of sentiment words with sentiment polarity (e.g., positive, negative, neutral) and sentiment intensity scores, along with their corresponding sentiment categories (e.g., anger, anxiety, satisfaction, disappointment). By matching the sentiment words appearing in the statements, a basic sentiment tendency score is calculated as the initial sentiment feature. Subsequently, based on the intent type identified in the intent feature data (e.g., complaint intent, consultation intent, confirmation intent, etc.), different intent types have corresponding pre-defined sentiment weights. For example, complaint intents typically have a strong negative sentiment tendency, so a higher negative sentiment weight is preset (e.g., weight coefficient 1.5); consultation intents are typically more neutral, so a neutral sentiment weight is preset (e.g., weight coefficient 1.0); and gratitude intents are preset with a positive sentiment weight (e.g., weight coefficient 1.3). The basic sentiment tendency score is weighted by the corresponding intent type weight coefficient to obtain the adjusted sentiment feature value.

[0054] Furthermore, based on the density of sentiment words in the sentence, the degree of modification by sentiment words (such as adverbs of degree like "very" and "somewhat"), and the overlap of sentiment words, a sentiment intensity score is calculated. Simultaneously, combining the context and intent features of the sentence, specific emotion categories are identified; for example, anger is identified from "I want to complain!", anxiety from "I'm worried about repayment pressure", and satisfaction from "Thank you for your explanation". Finally, through the above sentiment calculation process, a multi-dimensional sentiment feature vector containing sentiment intensity scores and specific emotion categories is generated, serving as input data for subsequent semantic feature aggregation.

[0055] For example, in a financial complaint handling scenario, for the customer service statement "Your overdue payment has incurred penalty interest," the intent feature is fee notification. Sentiment analysis first matches the neutral word "incurred," and after calibration with the negative weight of this intent, outputs the sentiment feature: intensity 0.7, emotion category: anxiety. When responding to the customer's reply "I want to complain!", the intent feature is regulatory complaint. By matching the sentiment dictionary to complain as a negative word, and combining the reinforcement coefficient of the complaint intent, the generated sentiment feature is: intensity 0.95, emotion: anger.

[0056] S334: Aggregate intent feature data and sentiment feature data to generate semantic feature data.

[0057] In this step, the intention and sentiment information of the same sentence are weighted and combined through an attention mechanism, and then the semantic association between the sentences before and after are aggregated using a graph neural network. Finally, hierarchical semantic feature data that preserves the speaker order and role relationship is generated, which includes the fused joint vector of intention and sentiment.

[0058] S40: Generate risk assessment results for call audio data based on call text data, semantic feature data, preset business rule data, and preset risk assessment models.

[0059] In this step, the call text data containing speaker roles and temporal markers, semantic feature data including fused vectors of intent and emotion, and a pre-set business rule library integrating compliance clauses, risk case templates, and real-time strategies are first aligned and fused in multiple dimensions. Then, the fused feature matrix is ​​fed into a pre-set risk assessment model. This model employs a heterogeneous network structure based on graph neural networks, mapping text entities, semantic attributes, and rule clauses to a unified vector space through node representation learning. It dynamically calculates the correlation between text patterns, semantic cues, and rule requirements using a multi-layer attention mechanism, and tracks the development and changes of risk features in the dialogue through a temporal propagation module. During the analysis process, the model simultaneously performs rule matching verification, semantic intent verification, and contextual calibration, ultimately outputting a structured risk judgment result, including risk type labels and probabilistic risk levels. This achieves accurate, interpretable, and traceable intelligent identification of obvious and potential risks in call audio data.

[0060] For example, in an insurance sales call within a financial scenario, when a pre-defined risk assessment model receives the fused feature vectors of a customer's question, "How much compensation can I receive if I am diagnosed with cancer?" and the salesperson's answer, "500,000 yuan will be paid out immediately upon diagnosis, with no waiting period," it identifies a high-risk type of "false promise regarding the waiting period" by comparing it with the hard rule of "90-day waiting period" in the insurance terms knowledge base. This is marked as a P1 level risk with a confidence level of 92%, and the risk time point is located at 145 seconds into the conversation. Simultaneously, when the customer asks, "Will abnormal medical examination records affect this?" and the salesperson answers, "Our company does not check medical examination records," the model combines the "failure to disclose truthfully" risk pattern from the historical claim rejection case database to generate a risk label for "misleading health disclosure," which is associated with 210 seconds into the conversation, triggering a double-risk superposition warning rule. In medical aesthetic consultation calls, for dialogue segments where consultants promise "three treatments to eliminate nasolabial folds" and "non-invasive, painless, and zero recovery period", a pre-set risk assessment model identified two types of violations—"absolute promises of efficacy" and "false non-invasive descriptions"—through a medical advertising regulations knowledge base, marking them as P2 level risks with a confidence level of 91%, respectively.

[0061] In one embodiment of this application, a specific risk analysis scheme is provided. In step S40, that is, based on call text data, semantic feature data, preset business rule data, and preset risk assessment model, a risk assessment result of call audio data is generated, which specifically includes the following steps S41-S42: S41: Match call text data, semantic feature data, and preset business rule data to generate matching results.

[0062] In this step, call text data containing speaker tags and time-based dialogue statements, semantic feature data integrating intent and sentiment feature vectors, and pre-defined business rule data including compliance clauses, risk case templates, and adjustable strategies are matched in multiple dimensions. Specifically, firstly, fragments potentially involving rules are quickly identified in the text using keywords and regular expressions. Then, semantic similarity is calculated to map semantic features to risk patterns in the rule base. Finally, based on speaker weights and the dialogue time context, a ternary matching relationship is established between text fragments, semantic vectors, and rule clauses, ultimately generating a matching result set that includes matching credibility, rule triggering location, and semantic support level.

[0063] S42: Input the matching results into the preset risk assessment model to generate risk assessment results; The risk assessment results include at least one of the following: risk type and risk level.

[0064] In this step, the matching results are input into a pre-defined risk assessment model for hierarchical analysis. The model first integrates multiple matching pieces of evidence related to the same event using a graph attention network, then uses a temporal convolution module to analyze the persistence and trends of risky behavior in the dialogue. Finally, based on a risk probability calibration module and combined with historical false alarm data, it dynamically adjusts the judgment threshold to generate a structured risk assessment result. This result includes at least standardized risk type labels, such as "false promises" and "inducing sales," as well as quantified risk levels, such as P1 urgent, P2 high risk, and P3 medium risk.

[0065] In one embodiment of this application, a specific model training scheme is provided, which includes the following steps before generating the risk assessment result of the call audio data based on call text data, semantic feature data, preset business rule data, and preset risk assessment model: Obtain the historical call dataset, which includes historical call audio data and its corresponding risk labeling data; Convert historical call audio data into historical call text data; Based on historical call text data, a training feature set is generated, which includes intent training feature data and emotion training feature data. Based on preset business rule data, historical call text data, training feature sets and their corresponding risk labeling data, the initial machine learning model is trained to generate a preset risk assessment model.

[0066] In this embodiment, firstly, a historical call dataset containing historical call audio files and their corresponding risk labeling data is acquired. The risk labeling data includes manually reviewed and marked risk types, risk occurrence time segments, risk levels, and handling results, forming an audio-label pairing sample library with time alignment. Subsequently, the historical call audio data is batch-converted into timestamped historical call text data using a speech recognition model. The recognition results are then manually corrected and standardized to ensure semantic and temporal consistency between the text data and the original audio, providing a high-quality text foundation for feature extraction. Next, based on the historical call text data, intent training features are extracted and combined with the extracted sentiment training features to construct a training feature set containing multi-dimensional semantic and sentiment information, while preserving the correspondence between features and risk labels. Further, preset business rule data, historical call text data, the training feature set, and their corresponding risk labeling data are multi-source aligned and fused to construct a training sample set. A multi-task learning framework is used to train the initial machine learning model. The main task learns the mapping from semantic features to risk categories, while the auxiliary tasks simultaneously learn the generalized representation of business rule matching patterns and historical risk cases. A joint loss function balances the weights of each task. During the training process, a course learning strategy is introduced to gradually increase the sample complexity in order to improve the model’s ability to identify hidden risks, and finally generate a pre-set risk assessment model that can comprehensively understand semantic intent, sentiment, rule constraints and historical patterns.

[0067] Optionally, the initial machine learning model can be a heterogeneous information network based on graph neural networks.

[0068] By using the above methods and training with precisely labeled historical data, the pre-set risk assessment model can deeply understand the semantic patterns and emotional cues of risk expression in business scenarios. In financial marketing scenarios, it can accurately identify complex risks such as "implicit inducement" and "compliance avoidance". In medical consultation scenarios, the model significantly improves the recall rate of violations such as "non-standard disclosure" and "risk exaggeration". Moreover, the model has continuous learning capabilities and can maintain the timeliness and accuracy of the recognition effect through incremental training as business rules are updated.

[0069] In one embodiment of this application, a specific risk warning scheme is provided, which, after generating a risk assessment result for the call audio data based on call text data, semantic feature data, preset business rule data, and a preset risk assessment model, further includes the following steps: Based on the risk assessment results, the call text data is processed to locate evidence fragments and generate risk evidence fragment data. Based on the risk assessment results and fragmented risk evidence data, risk warning information is generated; Send risk warning information to the target terminal.

[0070] In this embodiment, firstly, the corresponding paragraph in the dialogue text is found based on the time point of the risk marker. Then, semantic similarity calculation is used to identify the sentence combination most relevant to the risk within that paragraph. Simultaneously, two rounds of dialogue before and after this paragraph are extracted as supplementary background information. Finally, structured risk evidence fragment data containing the core risk sentence, relevant contextual sentences, and time range information is generated. Based on the description content, evidence fragment text, occurrence time, and responsible person information in the risk assessment results, and adding corresponding handling suggestions or review requirements according to business rules, a standardized and complete risk warning message is generated. Subsequently, the risk warning message is automatically pushed to the target terminal based on the risk level and business affiliation, selecting the appropriate sending channel. For example, for a P1 level high risk, both SMS and email are simultaneously sent to operations personnel, facilitating timely intervention and effective measures to reduce losses caused by the risk event.

[0071] By using the above methods, risk evidence can be accurately located, providing operators with an intuitive and visual risk display interface. This makes the generated risk alerts highly interpretable and actionable, helping operators quickly understand the overall risk situation and formulate targeted management strategies, thereby significantly reducing the average response time for high-risk events.

[0072] As can be seen, the above solution converts call audio into structured text and integrates sound features and semantic information to extract multi-dimensional features. Based on this, and combined with a constantly updated business rule base and a continuously learning and optimizing risk assessment model, a highly efficient and accurate risk assessment mechanism is constructed by integrating speech recognition, multimodal semantic analysis, dynamic rule matching, and machine learning inference. This enables a comprehensive and automated assessment of both explicit and implicit risks in call content, significantly expanding the coverage of risk identification and accelerating response speed. It transforms the lag and low coverage of traditional manual spot checks into full-scale real-time monitoring. Furthermore, by leveraging multi-dimensional feature fusion, it significantly reduces false positives and false negatives, providing quantifiable, traceable, and actionable risk management capabilities for industries with high compliance requirements such as finance and healthcare, effectively ensuring business compliance and operational security.

[0073] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0074] In one embodiment, a call risk assessment device is provided, which corresponds one-to-one with the call risk assessment methods described in the above embodiments. For example... Figure 6As shown, the call risk assessment device includes an acquisition module 101, a data conversion module 102, a first generation module 103, and a second generation module 104. Detailed descriptions of each functional module are as follows: Module 101 is used to acquire call audio data; The data conversion module 102 is used to convert call audio data into call text data based on speech recognition technology; The first generation module 103 is used to generate semantic feature data based on call audio data and call text data; The second generation module 104 is used to generate risk assessment results for call audio data based on call text data, semantic feature data, preset business rule data, and preset risk assessment model.

[0075] In one embodiment, the data conversion module 102 is specifically used for: The call audio data is segmented to generate a sequence of audio segments; The audio segment sequence is recognized by a speech recognition model to generate a text segment sequence. The text fragment sequence is spliced ​​and time-aligned to generate call text data.

[0076] In one embodiment, the first generation module 103 is specifically used for: Voiceprint features are extracted from call audio data, and combined with turn-by-turn analysis to generate speaker identification data; Based on speaker identification data and call text data, generate role text data; Natural language processing is performed on character text data based on a large language model to generate semantic feature data.

[0077] In one embodiment, the first generation module 103 is further configured to: The character text data is segmented based on a preset segmentation strategy to generate a set of sentences; Intent recognition is performed on a set of sentences based on a large language model to generate intent feature data; Based on intent feature data, a preset sentiment lexicon, and preset intent type weight coefficients, sentiment calculation is performed on the set of sentences to generate sentiment feature data. Intent feature data and sentiment feature data are aggregated to generate semantic feature data.

[0078] In one embodiment, the second generation module 104 is specifically used for: The call text data, semantic feature data, and preset business rule data are matched to generate matching results; Input the matching results into the preset risk assessment model to generate risk assessment results, which include at least one of the following: risk type and risk level.

[0079] In one embodiment, the acquisition module 101 is further configured to acquire a historical call dataset, wherein the historical call dataset includes historical call audio data and its corresponding risk labeling data; The data conversion module 102 is also used to convert historical call audio data into historical call text data.

[0080] In one embodiment, the device further includes: The third generation module is used to generate a training feature set based on historical call text data. The training feature set includes intent training feature data and emotion training feature data. The fourth generation module is used to train the initial machine learning model based on preset business rule data, historical call text data, training feature sets and their corresponding risk labeling data, and generate a preset risk assessment model.

[0081] In one embodiment, the device further includes: The fifth generation module is used to perform evidence fragment location processing on the call text data based on the risk assessment results, and generate risk evidence fragment data. The sixth generation module is used to generate risk warning information based on the risk assessment results and fragments of risk evidence data; The sending module is used to send risk warning information to the target terminal.

[0082] This invention provides a call risk assessment device that converts call audio into structured text and simultaneously extracts multi-dimensional features by fusing sound features and semantic information. Based on this, and combined with a constantly updated business rule base and a continuously learning and optimized risk assessment model, a highly efficient and accurate risk assessment mechanism is constructed by integrating speech recognition, multimodal semantic analysis, dynamic rule matching, and machine learning inference. This enables a comprehensive and automated assessment of both explicit and implicit risks in call content, significantly expanding the coverage of risk identification and accelerating response speed. It transforms the lag and low coverage of traditional manual spot checks into full-scale real-time monitoring. Furthermore, by leveraging multi-dimensional feature fusion, it greatly reduces false positives and false negatives, providing quantifiable, traceable, and actionable risk management capabilities for industries with high compliance requirements such as finance and healthcare, effectively ensuring business compliance and operational security.

[0083] Specific limitations regarding the call risk assessment device can be found in the limitations of the call risk assessment method described above, and will not be repeated here. Each module in the aforementioned call risk assessment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0084] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a call risk assessment method on the server side.

[0085] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a call risk assessment method on the client side.

[0086] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain call audio data; Based on speech recognition technology, call audio data is converted into call text data; Semantic feature data is generated based on call audio and text data. Based on call text data, semantic feature data, preset business rule data, and preset risk assessment models, risk assessment results are generated for call audio data.

[0087] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain call audio data; Based on speech recognition technology, call audio data is converted into call text data; Semantic feature data is generated based on call audio and text data. Based on call text data, semantic feature data, preset business rule data, and preset risk assessment models, risk assessment results are generated for call audio data.

[0088] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0091] It should be noted that any neural network models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0092] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for assessing call risk, characterized in that, include: Obtain call audio data; Based on speech recognition technology, the call audio data is converted into call text data; Based on the call audio data and the call text data, semantic feature data is generated; Based on the call text data, the semantic feature data, the preset business rule data, and the preset risk assessment model, a risk assessment result for the call audio data is generated.

2. The call risk assessment method as described in claim 1, characterized in that, The process of converting the call audio data into call text data based on speech recognition technology includes: The call audio data is segmented to generate an audio segment sequence; The audio segment sequence is subjected to speech recognition using a speech recognition model to generate a text segment sequence; The text segment sequence is spliced ​​and time-aligned to generate the call text data.

3. The call risk assessment method as described in claim 1, characterized in that, The step of generating semantic feature data based on the call audio data and the call text data includes: Voiceprint features are extracted from the call audio data, and speaker identification data is generated by combining turn-taking analysis. Based on the speaker identification data and the call text data, role text data is generated; Natural language processing is performed on the character text data based on a large language model to generate the semantic feature data.

4. The call risk assessment method as described in claim 3, characterized in that, The step of performing natural language processing on the character text data based on a large language model to generate the semantic feature data includes: The character text data is segmented based on a preset segmentation strategy to generate a set of sentences; Based on the large language model, intent recognition is performed on the set of sentences to generate intent feature data; Based on the intent feature data, the preset sentiment lexicon, and the preset intent type weight coefficient, sentiment calculation is performed on the sentence set to generate sentiment feature data; The intention feature data and the emotion feature data are aggregated to generate the semantic feature data.

5. The call risk assessment method as described in claim 1, characterized in that, The process of generating a risk assessment result for the call audio data based on the call text data, the semantic feature data, the preset business rule data, and the preset risk assessment model includes: The call text data, the semantic feature data, and the preset business rule data are matched to generate a matching result; The matching result is input into the preset risk assessment model to generate the risk assessment result, wherein the risk assessment result includes at least one of the following: risk type and risk level.

6. The call risk assessment method as described in claim 1, characterized in that, Before generating the risk assessment result of the call audio data based on the call text data, the semantic feature data, the preset business rule data, and the preset risk assessment model, the method further includes: Obtain a historical call dataset, wherein the historical call dataset includes historical call audio data and its corresponding risk labeling data; Convert the historical call audio data into historical call text data; Based on the historical call text data, a training feature set is generated, wherein the training feature set includes intent training feature data and emotion training feature data; Based on the preset business rule data, the historical call text data, the training feature set and its corresponding risk labeling data, the initial machine learning model is trained to generate the preset risk assessment model.

7. The call risk assessment method as described in claim 1, characterized in that, After generating a risk assessment result for the call audio data based on the call text data, the semantic feature data, the preset business rule data, and the preset risk assessment model, the method further includes: Based on the risk assessment results, the call text data is processed to locate evidence fragments, generating risk evidence fragment data. Based on the risk assessment results and the risk evidence fragments, a risk warning message is generated; The risk warning information is sent to the target terminal.

8. A call risk assessment device, characterized in that, include: The acquisition module is used to acquire call audio data; The data conversion module is used to convert the call audio data into call text data based on speech recognition technology; The first generation module is used to generate semantic feature data based on the call audio data and the call text data; The second generation module is used to generate a risk assessment result for the call audio data based on the call text data, the semantic feature data, the preset business rule data, and the preset risk assessment model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the call risk assessment method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the call risk assessment method as described in any one of claims 1 to 7.

Images