Call content quality inspection method, device, equipment, medium and program product

Abstract

This application provides a method, apparatus, device, storage medium, and program product for quality inspection of call content, which can be applied to the field of artificial intelligence technology. The method includes: acquiring the call content text corresponding to the call to be inspected; extracting corresponding semantic feature vectors based on the call content text using the feature extraction layer of a compliance detection model; determining the predicted out-of-bounds probability corresponding to a preset quality inspection rule based on the semantic feature vectors using multiple independent rule judgment units in the compliance detection model, wherein each independent rule judgment unit corresponds to a preset quality inspection rule; determining multiple rule matching results based on the call content text and the multiple preset quality inspection rules; determining target out-of-bounds probabilities corresponding to the multiple preset quality inspection rules based on the predicted out-of-bounds probabilities and the rule matching results; and determining the set of quality inspection rules matched by the call content text according to the target out-of-bounds probabilities.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and specifically to a method, apparatus, device, medium, and program product for quality inspection of call content. Background Technology

[0002] Currently, compliance checks on call content after business communication with clients primarily rely on manual sampling. Manual sampling not only has low coverage, making it difficult to detect many violations, but also relies on the inspector's personal experience, leading to significant subjective judgment differences and potential misjudgments or omissions. Furthermore, checking individual calls is time-consuming, resulting in low efficiency. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a method, apparatus, device, medium, and program product for quality inspection of call content.

[0004] According to a first aspect of this application, a method for quality inspection of call content is provided, comprising: obtaining call content text corresponding to a call to be inspected; extracting corresponding semantic feature vectors based on the call content text using a feature extraction layer of a compliance detection model; determining a predicted out-of-bounds probability corresponding to a preset quality inspection rule based on the semantic feature vectors using multiple independent rule judgment units in the compliance detection model, wherein each independent rule judgment unit corresponds to a preset quality inspection rule; determining multiple rule matching results based on the call content text and the multiple preset quality inspection rules; determining target out-of-bounds probabilities corresponding to the multiple preset quality inspection rules based on the predicted out-of-bounds probabilities and the rule matching results; and determining a set of quality inspection rules matched by the call content text according to the target out-of-bounds probabilities.

[0005] According to an embodiment of this application, based on the call content text, the corresponding semantic feature vector is extracted using the feature extraction layer of the compliance detection model, including: segmenting the call content text and mapping the segmented call content text to word feature vectors; performing contextual semantic modeling based on the word feature vectors to extract feature vectors corresponding to multiple different semantic dimensions; fusing the multiple feature vectors to obtain a deep semantic feature vector; and aggregating and compressing the deep semantic feature vectors to obtain the semantic feature vector.

[0006] According to embodiments of this application, the plurality of feature vectors include at least an interaction state feature vector, an expression mode feature vector, and a behavioral tendency feature vector.

[0007] According to an embodiment of this application, the method of using multiple independent rule judgment units in the compliance detection model to determine the predicted out-of-bounds probability corresponding to a preset quality inspection rule based on the semantic feature vector includes: using each of the independent rule judgment units to perform weighted calculation on the semantic feature vector according to the target weight associated with the corresponding preset quality inspection rule to generate an original evaluation score; and determining the predicted out-of-bounds probability based on the original evaluation score.

[0008] According to an embodiment of this application, determining the set of quality inspection rules for the call content text hit based on the target excess probability includes: for each of a plurality of preset quality inspection rules, comparing the corresponding target excess probability with a preset threshold; if the target excess probability is greater than the preset threshold, determining the preset quality inspection rule corresponding to the call content text hit; and generating the set of quality inspection rules based on at least one preset quality inspection rule for the call content text hit.

[0009] According to an embodiment of this application, based on the call content text and multiple preset quality inspection rules, multiple rule matching results are determined, including: for each preset quality inspection rule, performing a prohibited word query on the call content text based on the corresponding rule target prohibited word library to obtain a keyword hit score; for each preset quality inspection rule, performing semantic matching on the call content text based on the corresponding rule target transgression semantic template library to obtain a semantic hit score; and determining the rule matching result of the corresponding preset quality inspection rule based on the keyword hit score and the semantic hit score.

[0010] According to an embodiment of this application, the method further includes: inputting the call content text, the semantic feature vector, the target weight, and the original evaluation score into a large language model to obtain the judgment criteria for the corresponding preset quality inspection rules, wherein the judgment criteria include at least one of the interaction state dimension criteria, expression mode dimension criteria, and behavioral tendency dimension criteria; and generating an over-the-horizon analysis report for the corresponding preset quality inspection rules based on the judgment criteria. According to a second aspect of this application, a call content quality inspection device is provided, comprising: a call content text acquisition module, configured to acquire call content text corresponding to a call to be inspected; a semantic feature vector extraction module, configured to extract corresponding semantic feature vectors based on the call content text using a feature extraction layer of a compliance detection model; a predicted excess probability determination module, configured to determine a predicted excess probability corresponding to a preset quality inspection rule based on the semantic feature vectors using multiple independent rule judgment units in the compliance detection model, wherein each independent rule judgment unit corresponds to a preset quality inspection rule; a rule matching result determination module, configured to determine multiple rule matching results based on the call content text and multiple preset quality inspection rules; a target excess probability determination module, configured to determine a target excess probability corresponding to each of the multiple preset quality inspection rules based on the predicted excess probability and the rule matching results; and a quality inspection rule set determination module, configured to determine the set of quality inspection rules matched by the call content text according to the target excess probability.

[0011] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0012] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0013] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0014] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0015] Figure 1 The illustration schematically depicts an application scenario of a call content quality inspection method, apparatus, device, medium, and program product according to embodiments of this application;

[0016] Figure 2This schematic diagram illustrates the structural block diagram of a compliance detection model according to an embodiment of this application;

[0017] Figure 3 A flowchart illustrating a call content quality inspection method according to an embodiment of this application is shown schematically.

[0018] Figure 4 This schematic diagram illustrates a structural block diagram of a call content quality inspection device according to an embodiment of this application;

[0019] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a quality inspection method for call content according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0022] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0023] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0024] In the technical solution of this application, the user information (including but not limited to call content, call content text generated based on call content, user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0025] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0026] It's important to note that the term "neural network" can refer to a machine learning network based on deep learning. A neural network processes input and provides corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between them. Neural networks used in deep learning applications often include many hidden layers, increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer serves as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output becomes the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each processing the input from the layer above.

[0027] It should be understood that machine learning generally includes three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0028] In one or more embodiments described herein, a large language model is a type of large model; the term "large model" can refer to a deep learning model with a large number of model parameters, which can include hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. Large models can also be called foundational models / basic models. They are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability, such as large language models and multimodal pre-trained models. It should be understood that in practical applications, large models only require a small number of samples to fine-tune the pre-trained model before being applied to different tasks. Large models can be widely used in natural language processing, computer vision, and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering, image captioning, and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. Major application scenarios for large models can include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0029] Figure 1 This illustration schematically depicts an application scenario of a call content quality inspection method, apparatus, device, medium, and program product according to embodiments of this application. For example... Figure 1 As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send information, etc.

[0030] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal devices. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, and it can include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, and can also be in the form of web pages.

[0031] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and basic cloud computing services such as big data. The server can be the backend server of the aforementioned target application, used to provide backend services to the clients of the target application.

[0032] It should be noted that the call content quality inspection method provided in this application embodiment can generally be executed by server 105 and / or terminal devices 101-103. Accordingly, the call content quality inspection device provided in this application embodiment can generally be set in server 105 and / or terminal devices 101-103.

[0033] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0034] Figure 2 A schematic diagram illustrating the structure of a compliance detection model according to an embodiment of this application is provided. Figure 2As shown, the compliance detection model 200 includes a feature extraction layer 210, a first independent rule judgment unit 221, a second independent rule judgment unit 222, a third independent rule judgment unit 223, and a rule matching calibration unit 224. Figure 1 The compliance detection model 200 shown is merely exemplary, and this application does not limit the specific construction method of the compliance detection model 200. The number of independent rule judgment units in the compliance detection model 200 can be set as needed. The feature extraction layer 210 is connected to multiple independent rule judgment units 221-223 and rule matching calibration unit 224; the multiple independent rule judgment units 221-223 and rule matching calibration unit 224 are connected to the probability fusion calibration unit 230.

[0035] According to one implementation, the feature extraction layer 210 is trained to extract semantic feature vectors from the call content text. The feature extraction layer 210 includes a text embedding layer, a semantic encoding layer, and a global pooling layer. Specifically, the text encoding layer is used to segment the call content text and may also include a pre-trained word vector model to map the segmented call content text into word feature vectors. The semantic encoding layer can be implemented based on a Transformer encoder, used to perform contextual semantic modeling based on word feature vectors, extracting feature vectors corresponding to multiple different semantic dimensions; and fusing multiple feature vectors to obtain a deep semantic feature vector. When fusing multiple feature vectors, they can be fused by concatenation; this application does not limit the specific method of feature vector fusion. The multiple feature vectors include at least an interaction state feature vector, an expression mode feature vector, and a behavioral tendency feature vector. The interaction state feature vector represents the emotional characteristics of both parties in the call, such as anger or calmness; the expression mode feature vector represents the communication attitude and tone intensity of the business personnel, such as harshness, gentleness, or provocation; and the behavioral tendency feature vector represents the intent and purpose of the call, such as threats, misleading, or insults. The global pooling layer is used to aggregate and compress deep semantic feature vectors to obtain semantic feature vectors. The more characters included in the call content text, the longer the fused deep semantic feature vector will be. Therefore, the deep semantic feature vector is a variable-length sequence. The global pooling layer aggregates and compresses the deep semantic feature vector to ensure that call content texts of different lengths have semantic feature vectors of the same length, guaranteeing the stability of subsequent model calculations.

[0036] According to one implementation, each independent rule judgment unit is equipped with a corresponding preset quality inspection rule. The preset quality inspection rule is a rule set for business operations that cannot be violated, such as insulting customers, impersonating public authorities, using language that violates public order and good morals, continuously harassing irrelevant personnel, disclosing internal information, engaging in unauthorized or excessive operations, causing disturbances, requesting video recordings or images, including false statements, misleading or inciting users, and involving users outside the scope permitted by the business. The embodiments of this application do not limit the specific number or form of the preset quality inspection rules; they can be set as needed.

[0037] The independent rule judgment unit determines the original evaluation score corresponding to the preset quality inspection rule based on the input semantic feature vector, representing the tendency of the call content text to violate the preset quality inspection rule. Multiple independent rule judgment units may have the same structure, and the parameters of each independent rule judgment unit are independent. Each independent rule judgment unit includes a dedicated feature weighting layer and a weighted summation layer. When receiving the semantic feature vector, the independent rule judgment unit uses the dedicated feature weighting layer to calculate multiple weighted values ​​based on the target weights associated with the corresponding preset quality inspection rule, thereby extracting feature information used to determine whether the corresponding preset quality inspection rule is violated; then, the weighted summation layer is used to sum the multiple weighted values ​​to obtain the original evaluation score representing the tendency to violate the rule.

[0038] According to one implementation, when training the compliance detection model, the training samples consist of preprocessed call content text and multi-label annotations. The input is the call content text, and the labels are binary classification labels of multiple preset quality inspection rules, with out-of-bounds errors recorded as 1 and compliance as 0. The compliance detection model can be trained end-to-end, with the training objects being the feature extraction layer and multiple independent rule judgment units. During forward computation, the text outputs a fixed-length semantic feature vector through the feature extraction layer, which is then input into multiple independent rule judgment units to obtain the predicted out-of-bounds probabilities for each rule. The total loss is the sum of the cross-entropy loss between the predicted probabilities and the true labels. The parameters of the feature extraction layer, as well as the target weights and biases of each independent rule judgment unit, are synchronously updated through backpropagation. Under multi-label task supervision, the compliance detection model implicitly decouples the interaction state feature vector, expression mode feature vector, and behavioral tendency feature vector.

[0039] Below, we will explain in detail the working process of each module in the inference phase of the compliance detection model 200, based on the execution process of the call content quality inspection method.

[0040] Figure 3 A flowchart illustrating a call content quality inspection method according to an embodiment of this application is shown schematically. Figure 3As shown, the call content quality inspection method according to the embodiments of this application may include steps 310 to 360.

[0041] In step 310, the text of the call content corresponding to the call to be inspected is obtained. According to one embodiment, the quality inspection call can specifically be a call between business personnel and customers for quality inspection, and the call content can specifically be business communication content, including repayment reminders, product introductions, and handling complaints. The embodiments of this application do not limit the specific business type communicated in the call content.

[0042] According to one implementation, user consent or authorization can be obtained before acquiring call recordings with customers. For example, before step 310, a request can be sent to the user to acquire call recordings or call content text. If the user consents or authorizes the acquisition of call recordings or call content text, the call recording is acquired or step 310 is executed. When acquiring the call content text corresponding to the call to be inspected, the corresponding call content text can be obtained from the call recording; specifically, this includes: inputting the call recording to be inspected into a speech-to-text model, and through speech recognition and noise suppression processing, outputting the original call transcription text containing the complete dialogue between the business personnel and the customer. Subsequently, the original call transcription text is segmented into sentences according to natural pauses, meaningless interjections and repetitive sentences are removed, and environmental noise is removed to obtain a clean call content text that can be used as input to the compliance inspection model.

[0043] In step 320, based on the call content text, the corresponding semantic feature vector is extracted using the feature extraction layer of the compliance detection model.

[0044] In step 330, multiple independent rule judgment units in the compliance detection model determine the predicted out-of-middle probability corresponding to a preset quality inspection rule based on semantic feature vectors. Each independent rule judgment unit corresponds to one preset quality inspection rule. The preset quality inspection rule is a rule set for business operations that cannot be violated. The embodiments of this application do not limit the specific number or form of the preset quality inspection rules; they can be set as needed.

[0045] In step 340, based on the call content text and multiple preset quality inspection rules, multiple rule matching results are determined.

[0046] In step 350, based on the predicted excess probability and rule matching results, the target excess probability corresponding to each of the multiple preset quality inspection rules is determined.

[0047] In step 360, the set of quality inspection rules for the text content of the call is determined based on the target over-probability.

[0048] In the embodiments of this application, the semantic feature vector fully extracts the semantic information in the text of the call content, thereby fully mining the relevant features in the text; furthermore, the semantic feature vector is input into a dedicated rule judgment unit to judge whether it exceeds the bounds, thereby improving the accuracy of judging whether the corresponding rules are violated.

[0049] According to one implementation, based on the call content text, the corresponding semantic feature vector is extracted using the feature extraction layer of a compliance detection model. This includes: segmenting the call content text and mapping the segmented text to word feature vectors; performing contextual semantic modeling based on the word feature vectors to extract feature vectors corresponding to multiple different semantic dimensions. In this embodiment, by extracting feature vectors of multiple dimensions, the semantic information in the call content text can be extracted more comprehensively. Subsequently, multiple feature vectors are fused to obtain a deep semantic feature vector. According to one implementation, when fusing multiple feature vectors, they can be concatenated. Finally, the deep semantic feature vector is aggregated and compressed to obtain a semantic feature vector. According to one implementation, the feature vector includes at least an interaction state feature vector, an expression mode feature vector, and a behavioral tendency feature vector. The interaction state feature vector represents the emotional characteristics of both parties in the call, such as anger or calmness; the expression mode feature vector represents the communication attitude and tone intensity of the business personnel, such as harshness, gentleness, or provocation; and the behavioral tendency feature vector represents the intent and purpose of the call, such as threats, misleading, or insults. In the embodiments of this application, by extracting features of interaction state, expression mode and behavioral tendency, a more comprehensive analysis of call content can be performed in a quality inspection scenario to determine whether it has exceeded the bounds.

[0050] According to one implementation, multiple independent rule judgment units in a compliance detection model determine the predicted violation probability corresponding to a preset quality inspection rule based on semantic feature vectors. This includes: weighting the semantic feature vectors using the target weights associated with the corresponding preset quality inspection rule to generate an original evaluation score; and determining the predicted violation probability based on the original evaluation score. According to another implementation, when determining the predicted violation probability based on the original evaluation score, a Sigmoid nonlinear transformation can be used to compress the original evaluation score from any numerical range to a standard probability range of 0 to 1, achieving score standardization and obtaining the predicted violation probability. In the embodiments of this application, the independent rule judgment units, through pre-trained target weights, can extract original evaluation scores specific to evaluating whether a business rule has been violated and obtain the predicted violation probability, thereby improving the accuracy of compliance evaluation for each business rule.

[0051] According to one implementation, based on the call content text and multiple preset quality inspection rules, multiple rule matching results are determined, including: for each preset quality inspection rule, using a rule matching calibration unit to query the call content text for prohibited words based on the corresponding rule target prohibited word library, obtaining a keyword hit score; for each preset quality inspection rule, using a rule matching calibration unit to perform semantic matching on the call content text based on the corresponding rule target transgression semantic template library, obtaining a semantic hit score; and determining the rule matching result of the corresponding preset quality inspection rule based on the keyword hit score and the semantic hit score. According to one implementation, for each preset quality inspection rule, a dedicated rule target prohibited word library can be set, the target prohibited word library including at least one prohibited word that is prohibited from appearing in the call content text under the preset quality inspection rule; prohibited words can be divided into first-level, second-level, and third-level categories, and assigned different weight coefficients, such as weight coefficients of 1.0, 0.6, and 0.3. The preprocessed call content text is segmented into sentences and Chinese words. Each word is precisely matched against the prohibited keyword database of the current rules, and all prohibited words and their corresponding weights are recorded. The weights of the prohibited words are summed to obtain the original keyword score. Then, the maximum-minimum normalization method is used to map the original score to the interval between 0 and 1 to obtain the normalized keyword hit score, which is used to represent the intensity of the text's violation of the keyword dimension.

[0052] According to one implementation, for each preset quality inspection rule, a dedicated rule target excess semantic template library can be set. This library includes at least one excess semantic that is prohibited from appearing in the call content text under the preset quality inspection rule. When performing semantic matching on the call content text to obtain a semantic hit score, a pre-trained text embedding model can be used to encode the call content text to be detected and each excess semantic template into a fixed-dimensional semantic vector. The cosine similarity between the text semantic vector and the semantic vector of each excess template is calculated, and the maximum value among all similarities is taken as the original semantic score, used to characterize the excess hit strength of the text in the deep semantic dimension. Weight coefficients are pre-configured for the keyword dimension and the semantic dimension, for example, a keyword weight of 0.6 and a semantic weight of 0.4. When determining the rule matching result for the corresponding preset quality inspection rule based on the keyword hit score and the semantic hit score, the keyword hit score and the semantic hit score are weighted and fused to obtain a rule matching auxiliary score. A higher rule matching auxiliary score indicates a higher hit strength of the quality inspection rule. The rule matching auxiliary score is used as the rule matching result. In the embodiments of this application, by querying prohibited words and transgressive semantics, a hit score and a matching probability are obtained, so as to correct the probability value and thereby improve the accuracy of compliance evaluation.

[0053] According to one implementation, when determining the target excess probability corresponding to multiple preset quality inspection rules based on the predicted excess probability and rule matching results, for each preset quality inspection rule, a probability fusion calibration unit can be used to perform a weighted fusion calculation of the predicted excess probability and the rule matching result according to preset weights to obtain the target excess probability corresponding to that preset quality inspection rule. For example, the preset weights can be set to 0.5 for both the predicted excess probability and the rule matching result; the predicted excess probability is used to characterize the model's judgment result for the rule, and the rule matching result is used to characterize the hit strength of the rule at the keyword and semantic levels. By weighted fusion, the predicted excess probability is calibrated, thereby obtaining a more accurate target excess probability.

[0054] According to one implementation, a set of quality inspection rules for call content text matching is determined based on a target violation probability. This includes: for each of a plurality of preset quality inspection rules, comparing the corresponding target violation probability with a preset threshold; if the target violation probability is greater than the preset threshold, determining the preset quality inspection rule corresponding to the call content text matching; and generating a quality inspection rule set based on at least one preset quality inspection rule matching the call content text. According to one implementation, the preset threshold can be set to 0.6. This application does not limit the specific value of the preset threshold and can continue to set it as needed. In the embodiments of this application, the outputs of multiple independent rule judgment units are processed to output a set of violated business rules; by inputting only one passable content text, compliance checks of multiple business rules can be completed, improving the efficiency of compliance evaluation.

[0055] According to one implementation, the method further includes: inputting the call content text, semantic feature vector, target weight, and original evaluation score into a large language model to obtain the judgment criteria for corresponding preset quality inspection rules. The judgment criteria include at least one of the following: interaction state dimension criteria, expression mode dimension criteria, and behavioral tendency dimension criteria; and generating a violation analysis report for the corresponding preset quality inspection rules based on the judgment criteria. Based on the above information, the large language model analyzes the violation judgment process by combining the three feature dimensions of interaction state, expression mode, and behavioral tendency, and automatically generates the judgment criteria for the corresponding preset quality inspection rules. The judgment criteria include at least one of the following: interaction state dimension criteria, expression mode dimension criteria, and behavioral tendency dimension criteria. Specifically, the interaction state dimension criteria are used to explain whether the states of anger, calmness, and excitement reflected in the text support violation judgment from the perspective of the emotional characteristics of both parties in the call; the expression mode dimension criteria are used to explain whether the expressions of harshness and gentleness reflected in the text support violation judgment from the perspective of the communication attitude and tone intensity of the business personnel; and the behavioral tendency dimension criteria are used to explain whether the behavioral tendencies reflected in the text support violation judgment from the perspective of call intent and behavioral purpose. When generating judgment criteria, the large language model can combine the corresponding feature components in the semantic feature vector, the target weight of the independent rule judgment unit, and the original evaluation score to clarify the correspondence between features and judgment results, and locate specific text fragments related to violations in the call content, making the criteria traceable, interpretable, and verifiable. After obtaining the judgment criteria, a violation analysis report corresponding to the preset quality inspection rules is further generated based on these criteria. The report includes at least: basic call information, the preset quality inspection rules that were hit, the model's original evaluation score, the target violation probability, at least one dimension criterion, and a description of the text fragments related to violations, thus forming a violation analysis report that can be directly used for compliance review and manual review. In the embodiments of this application, the large model interprets the model analysis process and outputs a violation analysis report, which can describe the corresponding parts that violate relevant regulations and locate the violation content, thereby facilitating subsequent improvements to the repayment reminder service.

[0056] Based on the above-described call content quality inspection method, embodiments of this application also provide a call content quality inspection device. The following will be combined with... Figure 4 The device is described in detail.

[0057] Figure 4 A schematic block diagram of a call content quality inspection device according to an embodiment of this application is shown.

[0058] like Figure 4As shown, the call content quality inspection device 400 of this embodiment includes a call content text acquisition module 410, a semantic feature vector extraction module 420, a predicted excess probability determination module 430, a rule matching result determination module 440, a target excess probability determination module 450, and a quality inspection rule set determination module 460.

[0059] The call content text acquisition module 410 is used to acquire the call content text corresponding to the call to be inspected. In one embodiment, the call content text acquisition module 410 can be used to execute step 310 described above, which will not be repeated here.

[0060] The semantic feature vector extraction module 420 is used to determine the predicted out-of-bounds probability corresponding to a preset quality inspection rule based on the semantic feature vectors of multiple independent rule judgment units in the compliance detection model. Each independent rule judgment unit corresponds to a preset quality inspection rule. In one embodiment, the semantic feature vector extraction module 420 can be used to execute step 320 described above, which will not be repeated here.

[0061] The semantic feature vector extraction module 420 includes a word feature vector mapping module, used to segment the call content text and map the segmented call content text into word feature vectors; a multi-semantic dimension feature vector extraction module, used to perform contextual semantic modeling based on word feature vectors and extract feature vectors corresponding to multiple different semantic dimensions; and a feature vector fusion module, used to fuse multiple feature vectors to obtain deep semantic feature vectors. The feature vectors include at least interaction state feature vectors, expression mode feature vectors, and behavioral tendency feature vectors.

[0062] The predicted excess probability determination module 430 is used to determine the predicted excess probability corresponding to a preset quality inspection rule based on semantic feature vectors using multiple independent rule judgment units in the compliance detection model. Each independent rule judgment unit corresponds to a preset quality inspection rule. In one embodiment, the predicted excess probability determination module 430 can be used to execute step 330 described above, which will not be repeated here.

[0063] The prediction excess probability determination module 430 includes an original evaluation score generation module, which uses each independent rule judgment unit to perform weighted calculation on the semantic feature vector according to the target weight associated with the corresponding preset quality inspection rule to generate an original evaluation score; and a prediction excess probability determination submodule, which is used to determine the prediction excess probability based on the original evaluation score.

[0064] The rule matching result determination module 440 is used to determine multiple rule matching results based on the call content text and multiple preset quality inspection rules. In one embodiment, the rule matching result determination module 440 can be used to execute step 340 described above, which will not be repeated here.

[0065] The rule matching result determination module 440 includes a keyword hit score acquisition module, which is used to perform prohibited word queries on the call content text based on the corresponding rule target prohibited word library for each of the multiple preset quality inspection rules, and obtain a keyword hit score; a semantic hit score acquisition module, which is used to perform semantic matching on the call content text based on the corresponding rule target transgression semantic template library for each preset quality inspection rule, and obtain a semantic hit score; and a rule matching result determination submodule, which is used to determine the rule matching result of the corresponding preset quality inspection rule based on the keyword hit score and the semantic hit score.

[0066] The target violation probability determination module 450 is used to determine the target violation probability corresponding to each of the multiple preset quality inspection rules based on the predicted violation probability and the rule matching result. In one embodiment, the target violation probability determination module 450 can be used to execute step 350 described above, which will not be repeated here.

[0067] The quality inspection rule set determination module 460 is used to determine the quality inspection rule set for the text hit of the call content based on the target violation probability. In one embodiment, the quality inspection rule set determination module 460 can be used to perform step 360 described above, which will not be repeated here.

[0068] The quality inspection rule set determination module 460 includes a preset threshold comparison module, used to compare the corresponding target excess probability with a preset threshold for each preset quality inspection rule among multiple preset quality inspection rules; a preset quality inspection rule hit determination module, used to determine the preset quality inspection rule that the call content text hits when the target excess probability is greater than the preset threshold; and a quality inspection rule set generation module, used to generate a quality inspection rule set based on at least one preset quality inspection rule hit by the call content text.

[0069] The call content quality inspection device 400 also includes a violation analysis report generation module, which is used to input the call content text, semantic feature vector, target weight and original evaluation score into the large language model to obtain the judgment basis of the corresponding preset quality inspection rules. The judgment basis includes at least one of the following: interaction state dimension basis, expression mode dimension basis and behavior tendency dimension basis; and generates a violation analysis report of the corresponding preset quality inspection rules based on the judgment basis.

[0070] According to embodiments of this application, any and multiple modules among the call content text acquisition module 410, semantic feature vector extraction module 420, predicted excess probability determination module 430, rule matching result determination module 440, target excess probability determination module 450, and quality inspection rule set determination module 460 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the call content text acquisition module 410, semantic feature vector extraction module 420, predicted excess probability determination module 430, rule matching result determination module 440, target excess probability determination module 450, and quality inspection rule set determination module 460 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or any other reasonable means of integrating or packaging circuits, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the call content text acquisition module 410, semantic feature vector extraction module 420, predicted excess probability determination module 430, rule matching result determination module 440, target excess probability determination module 450, and quality inspection rule set determination module 460 can be at least partially implemented as computer program modules, which can perform corresponding functions when the computer program module is run.

[0071] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a quality inspection method for call content according to an embodiment of this application.

[0072] like Figure 5 As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage portion 508 into a random access memory 503. The processor 501 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0073] Random access memory 503 stores various programs and data required for the operation of electronic device 500. Processor 501, read-only memory 502, and random access memory 503 are interconnected via bus 504. Processor 501 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 502 and / or random access memory 503. It should be noted that the programs may also be stored in one or more memories other than read-only memory 502 and random access memory 503. Processor 501 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0074] According to embodiments of this application, the electronic device 500 may further include an input / output interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card, such as a local area network card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0075] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0076] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 502, and / or random access memory 503, and / or one or more memories other than read-only memory 502 and random access memory 503 described above.

[0077] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0078] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0079] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0080] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0082] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A method for quality inspection of call content, characterized in that, The method includes: Retrieve the text of the call content corresponding to the call to be inspected; Based on the text of the call content, the corresponding semantic feature vector is extracted using the feature extraction layer of the compliance detection model. The compliance detection model utilizes multiple independent rule judgment units to determine the predicted out-of-bounds probability corresponding to a preset quality inspection rule based on the semantic feature vector, wherein each independent rule judgment unit corresponds to a preset quality inspection rule; Based on the call content text and multiple preset quality inspection rules, multiple rule matching results are determined; Based on the predicted excess probability and the rule matching result, the target excess probability corresponding to the plurality of preset quality inspection rules is determined respectively; Based on the target out-of-bounds probability, determine the set of quality inspection rules for the call content text hit.

2. The method according to claim 1, characterized in that, The step of extracting corresponding semantic feature vectors based on the call content text using the feature extraction layer of the compliance detection model includes: The call content text is segmented, and the segmented call content text is mapped into word feature vectors. Contextual semantic modeling is performed based on the word feature vectors, and feature vectors corresponding to multiple different semantic dimensions are extracted. Multiple feature vectors are fused to obtain a deep semantic feature vector; The deep semantic feature vector is aggregated and compressed to obtain the semantic feature vector.

3. The method according to claim 2, characterized in that, The multiple feature vectors include at least an interaction state feature vector, an expression mode feature vector, and a behavioral tendency feature vector.

4. The method according to claim 1, characterized in that, The step of using multiple independent rule judgment units in the compliance detection model to determine the predicted out-of-bounds probability corresponding to a preset quality inspection rule based on the semantic feature vector includes: Each independent rule judgment unit performs weighted calculation on the semantic feature vector according to the target weight associated with the corresponding preset quality inspection rule to generate an original evaluation score; The predicted excess probability is determined based on the original evaluation score.

5. The method according to claim 1, characterized in that, The step of determining the set of quality inspection rules for text matching in the call content based on the target violation probability includes: For each of the multiple preset quality inspection rules, the corresponding target out-of-bounds probability is compared with a preset threshold. If the probability of the target exceeding the threshold is greater than a preset threshold, it is determined that the text content of the call matches the preset quality inspection rule. The quality inspection rule set is generated based on at least one preset quality inspection rule matched by the text of the call content.

6. The method according to any one of claims 1-5, characterized in that, The determination of multiple rule matching results based on the call content text and multiple preset quality inspection rules includes: For each of the multiple preset quality inspection rules, the call content text is searched for prohibited words based on the corresponding prohibited word library to obtain a keyword hit score. For each preset quality inspection rule, semantic matching of the call content text is performed based on the corresponding rule target over-the-middle semantic template library to obtain a semantic hit score; Based on the keyword hit score and the semantic hit score, the rule matching result corresponding to the preset quality inspection rule is determined.

7. The method according to claim 4, characterized in that, The method further includes: The call content text, the semantic feature vector, the target weight, and the original evaluation score are input into the large language model to obtain the judgment criteria for the corresponding preset quality inspection rules. The judgment criteria include at least one of the following: interaction state dimension criteria, expression mode dimension criteria, and behavior tendency dimension criteria. An overrun analysis report is generated based on the judgment criteria and corresponding preset quality inspection rules.

8. A device for quality inspection of call content, characterized in that, The device includes: The call content text acquisition module is used to acquire the call content text corresponding to the call to be inspected. The semantic feature vector extraction module is used to extract the corresponding semantic feature vector based on the text of the call content using the feature extraction layer of the compliance detection model. The prediction excess probability determination module is used to determine the prediction excess probability corresponding to a preset quality inspection rule based on the semantic feature vector using multiple independent rule judgment units in the compliance detection model. Each independent rule judgment unit corresponds to a preset quality inspection rule. The rule matching result determination module is used to determine multiple rule matching results based on the call content text and multiple preset quality inspection rules; The target excess probability determination module is used to determine the target excess probability corresponding to the plurality of preset quality inspection rules based on the predicted excess probability and the rule matching result. The quality inspection rule set determination module is used to determine the quality inspection rule set for the call content text hit based on the target violation probability.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1-7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1-7.

Images