Dialogue voice supervision method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of business safety supervision, is applied to a bank business online handling scene, and relates to a dialogue voice supervision method, device and equipment and a storage medium thereof. Performing language feature analysis on the real-time dialogue voices of the two dialogue parties; the dialogue safety state of the two parties is identified according to the language features of the service party and the served party, and the leakage risk of secret-related content during voice dialogue is reduced. When the dialogue voice supervision method is applied to a bank online business handling scene, business handling can be performed in combination with dialogue voice when a service party and a served party are recognized to be in a safe state, and due to the fact that bank business handling often relates to secret-related personal privacy information, the business handling efficiency is greatly improved. When the service party and the served party are both in the safe state, service handling is carried out, and more strict safety guarantee can be provided for customers.

Description

Technical Field

[0001] The present application relates to the technical field of business security supervision, and is applied to online banking business processing scenarios, and in particular to a conversational voice supervision method, device, equipment and storage medium thereof. Background Art

[0002] Remote telephone banking services require customers to confirm their identity or sensitive information, such as passwords, ID numbers, names, verification codes, home addresses, and family information. In an open office environment, real-time voice communication between telephone banking representatives and customers can easily lead to private information leakage, which can, in serious cases, cause significant financial losses to customers.

[0003] Currently, in order to prevent privacy information from being leaked during real-time voice communication between telephone banking service representatives and customers, the method basically adopted is to remind customers to pay attention to call security and surrounding security before the formal business content conversation. However, this method can neither guarantee that the customer's answering is in a safe state, nor can it guarantee that the voice environment in which the telephone banking service representative is located meets security requirements. How to combine the conversation voice to perform conversation security identification and ensure the security of the bank's online business processing has become an urgent problem that needs to be solved. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to propose a conversational voice supervision method, apparatus, device and storage medium thereof to solve the problem of how to combine conversational voice to perform conversation security identification and ensure the security of online banking business processing.

[0005] To solve the above technical problems, the present application provides a method for monitoring conversational speech, which adopts the following technical solutions:

[0006] A method for supervising conversational speech comprises the following steps:

[0007] Acquire real-time voice chat between two parties, where the two parties are the service provider and the service recipient of the target service;

[0008] Performing language feature analysis on the real-time conversation voice of both parties to obtain language features of the service provider and the service recipient, respectively, wherein the language features include conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text;

[0009] Identify whether the service provider meets the preset service security requirements based on the conversation environment characteristics, conversation audio characteristics, conversation text characteristics, and confidentiality characteristics of the conversation text in the service provider's language characteristics;

[0010] Identifying whether the service environment of the service recipient is secure based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the service recipient's language features;

[0011] If the service provider meets the preset service security requirements and the service environment of the service recipient is in a secure state, the security level corresponding to the real-time conversation voice is identified, and the security encryption policy corresponding to the security level is invoked to encrypt the real-time conversation voice, and the encryption result is sealed in a preset supervision terminal;

[0012] If the service provider meets the preset service security requirements but the service environment of the service recipient is in an unsafe state, a service suspension prompt is sent to the service provider, and a service environment change prompt is sent to the service recipient;

[0013] If the service provider does not meet the preset service security requirements but the service environment of the service recipient is in a safe state, a service rectification instruction is sent to the service provider, and a service risk reminder is sent to the service recipient;

[0014] If the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state, a service alarm prompt is sent to the supervisory end.

[0015] Furthermore, after executing the step of obtaining the real-time conversation voice of the two parties, the method further includes:

[0016] Differentiating and marking the conversation sentences in the real-time conversation voice according to different speakers;

[0017] Obtaining segmented conversation text content corresponding to the real-time conversation voice using speech-to-text technology and the distinguishing mark;

[0018] A preset error correction model is used to perform text error correction processing on the segmented dialogue text content to obtain the dialogue text content after error correction processing, wherein the preset error correction model is a text error correction model based on KenLM.

[0019] Furthermore, the step of obtaining the segmented conversation text content corresponding to the real-time conversation speech using the speech-to-text technology and the distinguishing mark specifically includes:

[0020] Performing preliminary segmentation processing on the text content of the conversation according to the conversation time interval in the real-time conversation voice to obtain text content in segments;

[0021] The text contents of the segments are adjusted based on the distinguishing marks to obtain segmented dialogue text contents corresponding to different speakers.

[0022] Furthermore, the step of performing preliminary segmentation processing on the conversation text content according to the conversation time interval in the real-time conversation voice to obtain text content segments specifically includes:

[0023] Determining whether the conversation time interval exceeds a preset time interval;

[0024] If the conversation time interval does not exceed the preset time interval, the speech before and after the conversation time interval is uttered by the same speaker, and the conversation text content corresponding to the speech before and after the conversation time interval is not segmented;

[0025] If the conversation time interval exceeds a preset time interval, the voices before and after the conversation time interval are uttered by different speakers, and the conversation text contents corresponding to the voices before and after the conversation time interval are segmented.

[0026] Furthermore, before executing the step of performing language feature analysis on the real-time conversation speech of the two parties to obtain the language features of the service provider and the language features of the served party, the method further includes:

[0027] According to the distinguishing marks of the conversation sentences, the voices of the service provider and the service recipient are sorted out from the real-time conversation voices of the two parties to obtain the sorting results;

[0028] Based on the sorting result, generating a first speech sequence set corresponding to the service provider and generating a second speech sequence set corresponding to the service recipient;

[0029] The step of analyzing the language features of the real-time conversation speech of the two parties to obtain the language features of the service provider and the language features of the served party specifically includes:

[0030] sequentially inputting the first speech sequence set and the second speech sequence set as speech recognition sets into a preset language feature recognition model;

[0031] Performing language feature recognition on the speech recognition set using the language feature recognition model;

[0032] Acquiring the service-side language features output by the language feature recognition model based on the first speech sequence set; and

[0033] Acquire the language features of the served party output by the language feature recognition model based on the second speech sequence set.

[0034] Furthermore, the language feature recognition model includes a conversation environment feature recognition plug-in, a conversation audio feature recognition plug-in, a conversation text feature recognition plug-in, and a conversation text confidentiality feature recognition plug-in. The step of performing language feature recognition on the speech recognition set using the language feature recognition model specifically includes:

[0035] Identifying the noise index of the pronunciation environment in which the speech recognition set is located through the conversation environment feature recognition plug-in, and marking the pronunciation environment as an unsafe conversation environment if the noise index exceeds a set value; and marking the pronunciation environment as a safe conversation environment if the noise index does not exceed the set value;

[0036] Identifying the emotional state of the speaker of the speech recognition set through the conversation audio feature recognition plug-in; if the emotional state is a natural state, marking the pronunciation environment as a safe conversation environment; if the emotional state is a nervous, excited or angry state, marking the pronunciation environment as an unsafe conversation environment;

[0037] By using the conversation text feature recognition plug-in, the total number of pronunciation text contents in the speech recognition set is counted;

[0038] By using the conversation text confidential feature recognition plug-in, the total amount of confidential text content in the speech recognition set is counted;

[0039] The ratio of the total amount of the confidential text content to the total amount of the pronounced text content is calculated, and the security level of the speech recognition set is marked according to the size of the ratio.

[0040] Furthermore, before performing the steps of identifying the security level corresponding to the real-time conversation voice and invoking the security encryption policy corresponding to the security level to encrypt the real-time conversation voice, the method further includes:

[0041] Different security encryption strategies are set in advance according to different security levels, where the higher the security level, the higher the encryption level.

[0042] The step of identifying the security level corresponding to the real-time conversation voice and invoking the security encryption policy corresponding to the security level to encrypt the real-time conversation voice specifically includes:

[0043] Obtain the security level of the service provider's conversation voice content and the security level of the service recipient's conversation voice content respectively;

[0044] Integration formula based on preset security level:

[0045] a=a1·ω1+a2·ω2,

[0046] Calculate the security level corresponding to the real-time conversation voice, where a represents the security level corresponding to the real-time conversation voice, a1 represents the security level of the service provider's conversation voice content, a2 represents the security level of the service recipient's conversation voice content, ω1 represents the security weight value corresponding to the service provider's conversation voice content, ω2 represents the security weight value corresponding to the service recipient's conversation voice content, ω1+ω2=1, and ω2>ω1;

[0047] A corresponding security encryption strategy is selected based on the security level corresponding to the real-time conversation voice, and encryption processing is performed on the real-time conversation voice.

[0048] In order to solve the above technical problems, the embodiment of the present application also provides a conversation voice supervision device, which adopts the following technical solutions:

[0049] A conversational voice monitoring device, comprising:

[0050] A conversation voice acquisition module is used to acquire the real-time conversation voice of the two parties in the conversation, wherein the two parties in the conversation are the service provider and the service recipient of the target business respectively;

[0051] A language feature analysis module is used to analyze the language features of the real-time conversation between the two parties to obtain the service provider's language features and the service recipient's language features, wherein the language features include conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text;

[0052] A first identification module is used to identify whether the service provider meets the preset service security requirements based on the conversation environment characteristics, conversation audio characteristics, conversation text characteristics, and conversation text confidentiality characteristics in the service provider's language characteristics;

[0053] a second identification module for identifying whether the service environment in which the service recipient is located is a secure state based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the service recipient's language features;

[0054] A first branch processing module is configured to, if the service provider meets preset service security requirements and the service environment of the service recipient is secure, identify the security level corresponding to the real-time conversation voice, invoke a security encryption policy corresponding to the security level to encrypt the real-time conversation voice, and store the encrypted result in a preset monitoring terminal;

[0055] A second branch processing module is configured to send a service suspension prompt to the service provider and a service environment change prompt to the service provider if the service provider meets the preset service security requirements but the service environment of the service recipient is in an unsafe state;

[0056] A third branch processing module is configured to send a service rectification instruction to the service provider and a service risk reminder to the service provider if the service provider does not meet the preset service security requirements but the service environment of the service recipient is in a safe state;

[0057] The fourth branch processing module is used to send a service alarm prompt to the supervision end if the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state.

[0058] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0059] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned conversational speech supervision method when executing the computer-readable instructions.

[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0061] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the above-mentioned method for conversational speech supervision.

[0062] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0063] The conversational voice supervision method described in the embodiment of the present application obtains the real-time conversational voice of the two parties; performs language feature analysis on the real-time conversational voice of the two parties to obtain the language features of the service provider and the language features of the served party respectively; identifies the service security status of the service provider based on the language features of the service provider; identifies the security status of the service environment of the served party based on the language features of the served party, thereby combining the conversational security status of the two parties to reduce the risk of leakage of confidential content during voice conversations. The conversational voice supervision method described in the present application is applied to the online banking business processing scenario. It can be combined with the conversational voice to conduct business processing after identifying that both the service provider and the served party are in a safe state. Since the processing of banking business often involves confidential personal privacy information, conducting business processing when both the service provider and the served party are in a safe state can provide customers with more stringent security protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0065] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0066] Figure 2 is a flow chart of an embodiment of a method for monitoring conversational speech according to the present application;

[0067] Figure 3 yes Figure 2 A flowchart of a specific embodiment of pre-processing the real-time conversation speech after step 201 is shown;

[0068] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 302 is shown;

[0069] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 401 is shown;

[0070] Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 202 is shown;

[0071] Figure 7 yes Figure 6 A flowchart of a specific embodiment of step 602 is shown;

[0072] Figure 8 is a structural diagram of an embodiment of a conversational speech monitoring device according to the present application;

[0073] Figure 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0075] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0076] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0077] like Figure 1 As shown, system architecture 100 may include a terminal device 101, a network 102, and a server 103. Terminal device 101 may be a laptop computer 1011, a tablet computer 1012, or a mobile phone 1013. Network 102 is a medium for providing a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0078] The user can use the terminal device 101 to interact with the server 103 via the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0079] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0080] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0081] It should be noted that the conversation voice supervision method provided in the embodiment of the present application is generally executed by a terminal device, and accordingly, the conversation voice supervision device is generally provided in the terminal device.

[0082] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0083] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for monitoring conversational speech according to the present application. The method for monitoring conversational speech comprises the following steps:

[0084] Step 201: obtaining real-time conversation voice between two parties, wherein the two parties are respectively a service provider and a service recipient of a target service.

[0085] In this embodiment, the two parties in the conversation can be understood as the banking customer service party and the customer party in the online banking business processing scenario. At this time, the service party is the banking customer service party, and the customer party is the served party.

[0086] Specifically, real-time audio recordings of the conversation between the two parties can be obtained by monitoring the online call. By obtaining real-time audio recordings of the conversation between the two parties, the security of online banking transactions can be guaranteed, ensuring that customers can conduct banking transactions safely.

[0087] Step 202 , performing language feature analysis on the real-time conversation speech of the two parties to obtain language features of the service provider and language features of the served party, respectively. The language features include conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text.

[0088] By analyzing the language features of the real-time conversation voice between the two parties, we can obtain the language features of the service provider and the service recipient respectively. This allows us to jointly identify security risks from the perspectives of the service provider and the service recipient, ensuring that banking services are handled for customers while the conversation between both parties is secure.

[0089] Step 203 , based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the service provider's language features, it is identified whether the service provider meets the preset service security requirements.

[0090] By using the service provider's conversation environment characteristics, conversation audio characteristics, conversation text characteristics and conversation text confidentiality characteristics, it is possible to identify whether the service provider meets the preset service security requirements, and to make timely service security adjustments to the service provider when the service provider does not meet the service security requirements.

[0091] Step 204 , based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the language features of the served party, it is identified whether the served environment of the served party is in a safe state.

[0092] By using the conversation environment characteristics, conversation audio characteristics, conversation text characteristics and confidentiality characteristics of the conversation text of the service recipient, it is possible to identify whether the service environment of the service recipient is in a safe state. When the service recipient is not in a safe state, the service can be suspended in time to avoid leakage of customer confidential information.

[0093] In step 205, if the service provider meets the preset service security requirements and the service environment of the service recipient is in a secure state, the security level corresponding to the real-time conversation voice is identified, and the security encryption policy corresponding to the security level is called to encrypt the real-time conversation voice, and the encryption processing result is sealed in the preset supervision end.

[0094] By identifying the security level corresponding to the real-time conversation voice and calling the security encryption policy corresponding to the security level to encrypt the real-time conversation voice, the encryption processing result is sealed to the preset supervision end. When both the service provider and the service recipient are in a safe state, the encryption processing result of the real-time conversation voice is sealed, which is convenient for relevant departments to supervise the bank's online business processing, and at the same time, it also avoids the leakage of the conversation voice content.

[0095] Step 206: If the service provider meets the preset service security requirements but the service environment of the service recipient is in an unsafe state, a service suspension prompt is sent to the service provider, and a service environment change prompt is sent to the service recipient.

[0096] If the service provider meets the preset service security requirements but the service environment of the service recipient is in an unsafe state, a service suspension prompt will be sent to the service provider, and a service environment change prompt will be sent to the service recipient, thereby preventing the service recipient from leaking personal confidential information in public places, such as personal ID information, password information, etc.

[0097] Step 207: If the service provider does not meet the preset service security requirements but the service environment of the service recipient is in a safe state, a service rectification instruction is sent to the service provider, and a service risk reminder is sent to the service recipient.

[0098] When it is discovered that the service provider does not meet the preset service security requirements, service rectification instructions will be sent to the service provider in a timely manner, avoiding the bank's customer service party from handling business for customers in an unsafe state and preventing customer information from being leaked at the customer service party; at the same time, it also reminds the service recipient that there are service risks in the online business handling provided by the current service provider.

[0099] Step 208: If the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state, a service alarm prompt is sent to the supervisory end.

[0100] When it is found that the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state, considering that illegal business may be being operated, a service alarm prompt is sent to the supervision end in a timely manner.

[0101] The conversational voice supervision method described in this application is applied to the online banking business processing scenario. It can be combined with conversational voice to conduct business processing after identifying that both the service provider and the service recipient are in a safe state. Since the processing of banking business often involves confidential personal privacy information, conducting business processing when both the service provider and the service recipient are in a safe state can provide more stringent security protection for the customer's property.

[0102] Continue to refer Figure 3 , Figure 3 yes Figure 2 The flowchart of a specific embodiment of pre-processing the real-time conversation voice after step 201 is shown. In some optional implementations, pre-processing the real-time conversation voice after step 201 further includes the following steps:

[0103] Step 301: Differentiate and mark the conversation sentences in the real-time conversation voice according to different speakers;

[0104] Step 302: Obtain segmented text content corresponding to the real-time conversation speech using speech-to-text technology and the distinguishing mark;

[0105] Step 303 : Using a preset error correction model to perform text error correction processing on the segmented dialogue text content to obtain the dialogue text content after error correction processing, wherein the preset error correction model is a text error correction model based on KenLM.

[0106] Specifically, the KenLM is an open source natural language processing toolkit that uses a natural language processing model pre-trained by KenLM and can correct Chinese content errors by combining rule methods and confusion sets.

[0107] In this embodiment, by pre-processing the real-time conversation speech, the conversation text content after segmentation and error correction is obtained, thereby ensuring the accuracy of subsequent language feature analysis.

[0108] Continue to refer Figure 4 , Figure 4 yes Figure 3The flowchart of a specific embodiment of step 302 is shown. In some optional implementations, step 302 includes the following steps:

[0109] Step 401, performing preliminary segmentation processing on the text content of the real-time conversation voice according to the conversation time interval in the real-time conversation voice to obtain text content segments;

[0110] Step 402 : adjusting the text content of each segment based on the distinguishing marks to obtain segmented dialogue text content corresponding to different speakers.

[0111] Specifically, the dialogue scenario is fully considered first, and the dialogue text content is preliminarily segmented according to the dialogue time interval in the question-and-answer dialogue mode. Then, the preliminary segmentation processing results are adjusted in combination with the distinguishing marks to ensure that the obtained segmented dialogue text content corresponds to the speaker.

[0112] Continue to refer Figure 5 , Figure 5 yes Figure 4 The flowchart of a specific embodiment of step 401 is shown. In some optional implementations, step 401 includes the following steps:

[0113] Step 501, determining whether the conversation time interval exceeds a preset time interval;

[0114] Step 502: If the conversation time interval does not exceed the preset time interval, the speech before and after the conversation time interval is produced by the same speaker, and the conversation text content corresponding to the speech before and after the conversation time interval is not segmented;

[0115] Step 503: If the conversation time interval exceeds the preset time interval, the speech before and after the conversation time interval is uttered by different speakers, and the conversation text content corresponding to the speech before and after the conversation time interval is segmented.

[0116] In this embodiment, before executing the step of performing language feature analysis on the real-time conversation speech of the two parties to obtain the language features of the service party and the language features of the served party respectively, the method further includes: sorting out the speech emitted by the service party and the served party respectively from the real-time conversation speech of the two parties to obtain a sorting result according to the distinguishing marks of the conversation sentences; and generating a first speech sequence set corresponding to the service party and a second speech sequence set corresponding to the served party based on the sorting result.

[0117] By sorting out the voices emitted by the service provider and the service recipient respectively, generating a first voice sequence set corresponding to the service provider and a second voice sequence set corresponding to the service recipient, it is possible to identify the security environment of both parties in the conversation voice separately. Afterwards, the security environment identification results of both parties are combined to determine the security of the entire conversation voice, ensuring the combined identification of the service provider and the service recipient for the security of the bank's online business processing, which is more comprehensive and protects the security of the conversation data.

[0118] Continue to refer Figure 6 , Figure 6 yes Figure 2 The flowchart of a specific embodiment of step 202 is shown. In some optional implementations, step 202 includes the following steps:

[0119] Step 601: sequentially input the first speech sequence set and the second speech sequence set as speech recognition sets into a preset language feature recognition model;

[0120] Step 602: performing language feature recognition on the speech recognition set using the language feature recognition model;

[0121] Step 603: obtaining the service-side language features output by the language feature recognition model based on the first speech sequence set; and

[0122] Step 604: Acquire the language features of the served party output by the language feature recognition model based on the second speech sequence set.

[0123] In this embodiment, the language feature recognition model includes a conversation environment feature recognition plug-in, a conversation audio feature recognition plug-in, a conversation text feature recognition plug-in, and a conversation text confidentiality feature recognition plug-in.

[0124] Continue to refer Figure 7 , Figure 7 yes Figure 6 The flowchart of a specific embodiment of step 602 is shown. In some optional implementations, step 602 includes the following steps:

[0125] Step 701: Identify the noise index of the pronunciation environment in which the speech recognition set is located through the conversation environment feature recognition plug-in. If the noise index exceeds a set value, mark the pronunciation environment as an unsafe conversation environment. If the noise index does not exceed the set value, mark the pronunciation environment as a safe conversation environment.

[0126] Step 702: Identify the emotional state of the speaker in the speech recognition set using the conversation audio feature recognition plug-in. If the emotional state is natural, mark the pronunciation environment as a safe conversation environment. If the emotional state is nervous, excited, or angry, mark the pronunciation environment as an unsafe conversation environment.

[0127] Step 703: Counting the total number of pronounced text contents in the speech recognition set using the conversation text feature recognition plug-in;

[0128] Step 704: Count the total amount of confidential text content in the speech recognition set using the conversation text confidential feature recognition plug-in;

[0129] Step 705: Calculate the ratio of the total amount of the confidential text content to the total amount of the pronounced text content, and mark the security level of the speech recognition set according to the size of the ratio.

[0130] Specifically, the security level of the speech recognition set is marked according to the ratio value. In principle, the larger the ratio of the total amount of confidential text content to the total amount of pronounced text content, the more personal privacy data is involved in the conversation content and the higher the security level.

[0131] In some optional implementations, step 203 identifies whether the service provider meets preset service security requirements based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the service provider's language features, including the following steps:

[0132] If the conversation environment characteristics and the conversation audio characteristics determine that the service provider is in a secure conversation environment, and if the conversation text characteristics and the confidentiality characteristics of the conversation text determine that the security level of the service provider's conversation voice content exceeds the set security level, then the service provider meets the preset service security requirements; otherwise, the service provider does not meet the preset service security requirements.

[0133] Specifically, the service provider does not meet the preset service security requirements, including the following three situations:

[0134] Determining, based on the conversation environment characteristics and the conversation audio characteristics, that the service provider is in a secure conversation environment, but determining, based on the conversation text characteristics and the confidentiality characteristics of the conversation text, that the security level of the service provider's conversation voice content does not exceed a set security level;

[0135] Determining, based on the conversation environment characteristics and the conversation audio characteristics, that the service provider is in a non-secure conversation environment, but determining, based on the conversation text characteristics and the confidentiality characteristics of the conversation text, that the security level of the service provider's conversation voice content exceeds a set security level;

[0136] Through the conversation environment characteristics and the conversation audio characteristics, it is determined that the service party is in a non-secure conversation environment, and through the conversation text characteristics and the confidentiality characteristics of the conversation text, it is determined that the security level of the service party's conversation voice content does not exceed the set security level.

[0137] Correspondingly, the following method can be used to determine whether the service provider is in a secure conversation environment through the conversation environment characteristics and the conversation audio characteristics: determine whether the conversation environment corresponding to the conversation environment characteristics and the conversation audio characteristics is a secure environment; if both are secure environments, the service provider is in a secure conversation environment; if either one is an insecure environment, the service provider is in an insecure conversation environment.

[0138] Similarly, in some optional implementations, step 204 identifies whether the service environment of the service recipient is in a safe state based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the language features of the service recipient, which is basically consistent with the specific identification method of step 203 and will not be repeated here.

[0139] By separately identifying whether the service provider and the service recipient are in the required security state during the voice conversation, the security of online banking transactions is further guaranteed, providing customers with more secure business protection and avoiding the leakage of customers' personal information.

[0140] In this embodiment, before executing the step of identifying the security level corresponding to the real-time conversation voice and calling the security encryption policy corresponding to the security level to encrypt the real-time conversation voice, the method also includes: setting different security encryption policies in advance according to different security levels, wherein the higher the security level, the higher the corresponding encryption level.

[0141] In this embodiment, the step of identifying the security level corresponding to the real-time conversation voice and invoking the security encryption policy corresponding to the security level to encrypt the real-time conversation voice specifically includes: obtaining the security level of the service provider's conversation voice content and the security level of the service recipient's conversation voice content; and integrating the security levels according to a preset security level formula:

[0142] a=a1·ω1+a2·ω2,

[0143] Calculate the security level corresponding to the real-time conversation voice, where a represents the security level corresponding to the real-time conversation voice, a1 represents the security level of the service party's conversation voice content, a2 represents the security level of the service party's conversation voice content, ω1 represents the security weight value corresponding to the service party's conversation voice content, ω2 represents the security weight value corresponding to the service party's conversation voice content, ω1+ω2=1, and ω2>ω1; select a corresponding security encryption strategy based on the security level corresponding to the real-time conversation voice, and encrypt the real-time conversation voice.

[0144] This application obtains the real-time conversation voice of the two parties in the conversation; performs language feature analysis on the real-time conversation voice of the two parties in the conversation to obtain the language features of the service provider and the language features of the served party respectively; identifies the service security status of the service provider based on the language features of the service provider; identifies the security status of the service environment of the served party based on the language features of the served party, thereby combining the conversation security status of the two parties in the conversation to reduce the risk of leakage of confidential content during voice conversation. The conversation voice supervision method described in this application is applied to the online business processing scenario of the bank. It can be combined with the conversation voice to conduct business processing after identifying that both the service provider and the served party are in a safe state. Since the processing of bank business often involves confidential personal privacy information, conducting business processing when both the service provider and the served party are in a safe state can provide customers with more stringent security protection.

[0145] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0146] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0147] In the embodiment of the present application, the real-time conversation voice of the two parties is obtained; the language characteristics of the real-time conversation voice of the two parties are analyzed to obtain the language characteristics of the service provider and the language characteristics of the service recipient respectively; the service security status of the service provider is identified based on the language characteristics of the service provider; the security status of the service environment of the service recipient is identified based on the language characteristics of the service recipient, thereby combining the conversation security status of the two parties to reduce the risk of leakage of confidential content during voice conversation. The conversation voice supervision method described in this application is applied to the online banking business processing scenario. It can be combined with the conversation voice to conduct business processing after identifying that both the service provider and the service recipient are in a safe state. Since the processing of banking business often involves confidential personal privacy information, conducting business processing when both the service provider and the service recipient are in a safe state can provide customers with more stringent security protection.

[0148] Further references Figure 8 , as a response to the above Figure 2 The present application provides an embodiment of a conversation voice supervision device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0149] like Figure 8 As shown, the conversational speech monitoring device 800 of this embodiment includes: a conversational speech acquisition module 801, a language feature analysis module 802, a first recognition module 803, a second recognition module 804, a first branch processing module 805, a second branch processing module 806, a third branch processing module 807, and a fourth branch processing module 808. Among them:

[0150] The conversation voice acquisition module 801 is used to acquire the real-time conversation voice of the two parties in the conversation, wherein the two parties in the conversation are the service provider and the service recipient of the target business respectively;

[0151] The language feature analysis module 802 is used to perform language feature analysis on the real-time speech of the conversation between the two parties to obtain language features of the service provider and the service recipient, respectively. The language features include conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text.

[0152] The first identification module 803 is used to identify whether the service provider meets the preset service security requirements based on the conversation environment characteristics, conversation audio characteristics, conversation text characteristics, and conversation text confidentiality characteristics in the service provider's language characteristics;

[0153] The second identification module 804 is configured to identify whether the service environment of the service recipient is secure based on the conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text in the service recipient's language features;

[0154] The first branch processing module 805 is configured to, if the service provider meets the preset service security requirements and the service environment of the service recipient is secure, identify the security level corresponding to the real-time conversation voice, invoke the security encryption policy corresponding to the security level to encrypt the real-time conversation voice, and store the encryption result in a preset monitoring terminal;

[0155] The second branch processing module 806 is configured to send a service suspension prompt to the service provider and a service environment change prompt to the service provider if the service provider meets the preset service security requirements but the service environment of the service recipient is in an unsafe state;

[0156] The third branch processing module 807 is configured to send a service rectification instruction to the service provider and a service risk reminder to the service provider if the service provider does not meet the preset service security requirements but the service environment of the service recipient is in a safe state;

[0157] The fourth branch processing module 808 is configured to send a service alarm to the supervisory end if the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state.

[0158] This application obtains the real-time conversation voice of the two parties in the conversation; performs language feature analysis on the real-time conversation voice of the two parties in the conversation to obtain the language features of the service provider and the language features of the served party respectively; identifies the service security status of the service provider based on the language features of the service provider; identifies the security status of the service environment of the served party based on the language features of the served party, thereby combining the conversation security status of the two parties in the conversation to reduce the risk of leakage of confidential content during voice conversation. The conversation voice supervision method described in this application is applied to the online business processing scenario of the bank. It can be combined with the conversation voice to conduct business processing after identifying that both the service provider and the served party are in a safe state. Since the processing of bank business often involves confidential personal privacy information, conducting business processing when both the service provider and the served party are in a safe state can provide customers with more stringent security protection.

[0159] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0160] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0161] To solve the above technical problems, the present application also provides a computer device. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.

[0162] The computer device 9 includes a memory 9a, a processor 9b, and a network interface 9c that are interconnected via a system bus. Figure 9 Only a computer device 9 having components such as a memory 9a, a processor 9b, and a network interface 9c is shown. However, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead. Those skilled in the art will understand that the computer device herein is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0163] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0164] The memory 9a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 9a can be an internal storage unit of the computer device 9, such as the hard disk or memory of the computer device 9. In other embodiments, the memory 9a can also be an external storage device of the computer device 9, such as a plug-in hard disk equipped on the computer device 9, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, etc. Of course, the memory 9a can also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 9a is generally used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions of a conversational voice supervision method. In addition, the memory 9a can also be used to temporarily store various types of data that have been output or are to be output.

[0165] In some embodiments, the processor 9b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 9b is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is used to execute computer-readable instructions stored in the memory 9a or process data, such as computer-readable instructions for executing the conversational speech supervision method.

[0166] The network interface 9c may include a wireless network interface or a wired network interface. The network interface 9c is generally used to establish a communication connection between the computer device 9 and other electronic devices.

[0167] The computer device proposed in this embodiment belongs to the field of business security supervision technology and is applied to online banking business processing scenarios. This application obtains the real-time conversation voice of the two parties in the conversation; performs language feature analysis on the real-time conversation voice of the two parties in the conversation to obtain the service party's language features and the serviced party's language features respectively; identifies the service security status of the service party based on the service party's language features; identifies the security status of the serviced environment of the serviced party based on the serviced party's language features, thereby combining the conversation security status of the two parties in the conversation to reduce the risk of leakage of confidential content during voice conversations. The conversation voice supervision method described in this application is applied to the online banking business processing scenario. It can be combined with the conversation voice to conduct business processing after identifying that both the service party and the serviced party are in a safe state. Since the processing of banking business often involves confidential personal privacy information, conducting business processing when both the service party and the serviced party are in a safe state can provide customers with more stringent security protection.

[0168] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the above-mentioned conversational speech supervision method.

[0169] The computer-readable storage medium proposed in this embodiment belongs to the field of business security supervision technology and is applied to online banking business processing scenarios. This application obtains the real-time conversation voice of the two parties in the conversation; performs language feature analysis on the real-time conversation voice of the two parties in the conversation to obtain the service party's language features and the serviced party's language features respectively; identifies the service security status of the service party based on the service party's language features; identifies the security status of the serviced environment of the serviced party based on the serviced party's language features, thereby combining the conversation security status of the two parties in the conversation to reduce the risk of leakage of confidential content during voice conversations. The conversation voice supervision method described in this application is applied to the online banking business processing scenario. It can be combined with the conversation voice to conduct business processing after identifying that both the service party and the serviced party are in a safe state. Since the processing of banking business often involves confidential personal privacy information, conducting business processing when both the service party and the serviced party are in a safe state can provide customers with more stringent security protection.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0171] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A method for supervising conversational speech, characterized in that: The steps include: Acquire real-time conversation voice of two parties in the conversation, wherein the two parties in the conversation are respectively a service provider and a service recipient of a target business; Performing language feature analysis on the real-time conversation voice of the two parties to obtain language features of the service provider and the service recipient, respectively, wherein the language features include conversation environment features, conversation audio features, conversation text features, and confidentiality features of the conversation text; Identify whether the service provider meets the preset service security requirements based on the conversation environment features, conversation audio features, conversation text features and confidentiality features of the conversation text in the service provider's language features; Identify whether the service environment where the service recipient is located is in a safe state based on the conversation environment features, conversation audio features, conversation text features and confidentiality features of the conversation text in the language features of the service recipient; If the service provider meets the preset service security requirements and the service environment of the service recipient is in a safe state, the security level corresponding to the real-time conversation voice is identified, and the security encryption policy corresponding to the security level is called to encrypt the real-time conversation voice, and the encryption result is sealed in a preset supervision terminal; If the service provider meets the preset service safety requirements but the service environment of the service recipient is in an unsafe state, a service suspension prompt is sent to the service provider, and a service environment change prompt is sent to the service recipient; If the service provider does not meet the preset service safety requirements but the service environment of the service recipient is in a safe state, a service rectification instruction is sent to the service provider, and a service risk reminder is sent to the service recipient; If the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state, a service alarm prompt is sent to the supervisory end.

2. The method for monitoring conversational speech according to claim 1, characterized in that: After executing the step of acquiring the real-time conversation voice of the two parties, the method further includes: Differentiating and marking the dialogue sentences in the real-time dialogue voice according to different speakers; Obtaining segmented conversation text content corresponding to the real-time conversation speech through speech-to-text technology and the distinguishing mark; The segmented dialogue text content is subjected to text error correction processing using a preset error correction model to obtain the dialogue text content after error correction processing, wherein the preset error correction model is a text error correction model based on KenLM.

3. The method for monitoring conversational speech according to claim 2, characterized in that: The step of obtaining the segmented conversation text content corresponding to the real-time conversation speech by using the speech-to-text technology and the distinguishing mark specifically includes: According to the conversation time interval in the real-time conversation voice, the conversation text content is preliminarily segmented to obtain text content in segments; The text contents of the segments are adjusted based on the distinguishing marks to obtain segmented dialogue text contents corresponding to different speakers.

4. The method for monitoring conversational speech according to claim 3, characterized in that: The step of performing preliminary segmentation processing on the conversation text content according to the conversation time interval in the real-time conversation voice to obtain text content in segments specifically includes: Determining whether the conversation time interval exceeds a preset time interval; If the conversation time interval does not exceed the preset time interval, the speech before and after the conversation time interval is emitted by the same speaker, and the conversation text content corresponding to the speech before and after the conversation time interval is not segmented; If the conversation time interval exceeds a preset time interval, the speech before and after the conversation time interval is uttered by different speakers, and the conversation text content corresponding to the speech before and after the conversation time interval is segmented.

5. The method for monitoring conversational speech according to claim 2, characterized in that: Before executing the step of analyzing the language features of the real-time conversation speech of the two parties to obtain the language features of the service provider and the language features of the served party, the method further includes: According to the distinguishing marks of the conversation sentences, the voices respectively emitted by the service provider and the service recipient are sorted out from the real-time conversation voices of the two parties to obtain the sorting result; Based on the sorting result, generating a first speech sequence set corresponding to the service provider and generating a second speech sequence set corresponding to the service recipient; The step of analyzing the language features of the real-time conversation speech of the two parties to obtain the language features of the service provider and the language features of the served party respectively includes: sequentially inputting the first speech sequence set and the second speech sequence set as speech recognition sets into a preset language feature recognition model; Using the language feature recognition model to perform language feature recognition on the speech recognition set; Acquiring the service-side language features output by the language feature recognition model based on the first speech sequence set; and The language feature of the served party output by the language feature recognition model based on the second speech sequence set is obtained.

6. The method for monitoring conversational speech according to claim 5, characterized in that: The language feature recognition model includes a conversation environment feature recognition plug-in, a conversation audio feature recognition plug-in, a conversation text feature recognition plug-in, and a conversation text confidential feature recognition plug-in. The step of using the language feature recognition model to perform language feature recognition on the speech recognition set specifically includes: The noise index of the pronunciation environment in which the speech recognition set is located is identified through the conversation environment feature identification plug-in. If the noise index exceeds a set value, the pronunciation environment is marked as an unsafe conversation environment. If the noise index does not exceed the set value, the pronunciation environment is marked as a safe conversation environment. Identify the emotional state of the speaker of the speech recognition set through the conversation audio feature recognition plug-in, and if the emotional state is a natural state, mark the pronunciation environment as a safe conversation environment; if the emotional state is a nervous, excited or angry state, mark the pronunciation environment as an unsafe conversation environment; By using the dialog text feature recognition plug-in, the total number of pronunciation text contents in the speech recognition set is counted; By using the dialog text confidential feature recognition plug-in, the total amount of confidential text content in the speech recognition set is counted; The ratio of the total amount of the confidential text content to the total amount of the pronounced text content is calculated, and the security level of the speech recognition set is marked according to the size of the ratio.

7. The method for monitoring conversational speech according to any one of claims 1 to 6, characterized in that: Before executing the step of identifying the security level corresponding to the real-time conversation voice and calling the security encryption policy corresponding to the security level to encrypt the real-time conversation voice, the method further includes: Different security encryption strategies are set in advance according to different security levels, where the higher the security level, the higher the corresponding encryption level; The step of identifying the security level corresponding to the real-time conversation voice and invoking the security encryption policy corresponding to the security level to encrypt the real-time conversation voice specifically includes: Respectively obtain the security level of the service provider's conversation voice content and the security level of the service recipient's conversation voice content; Integration formula according to preset security level: a=a1·ω1+a2·ω2, Calculate the security level corresponding to the real-time conversation voice, where a represents the security level corresponding to the real-time conversation voice, a1 represents the security level of the service party's conversation voice content, a2 represents the security level of the service party's conversation voice content, ω1 represents the security weight value corresponding to the service party's conversation voice content, ω2 represents the security weight value corresponding to the service party's conversation voice content, ω1+ω2=1, and ω2>ω1; A corresponding security encryption strategy is selected based on the security level corresponding to the real-time conversation voice, and encryption processing is performed on the real-time conversation voice.

8. A conversation voice monitoring device, characterized in that: include: A conversation voice acquisition module, used to acquire the real-time conversation voice of two parties in the conversation, wherein the two parties in the conversation are the service provider and the service recipient of the target business respectively; A language feature analysis module is used to perform language feature analysis on the real-time conversation voice of the two parties in the conversation, and obtain the service party's language features and the service party's language features respectively, wherein the language features include conversation environment features, conversation audio features, conversation text features and confidentiality features of the conversation text; A first identification module is used to identify whether the service provider meets the preset service security requirements based on the conversation environment characteristics, conversation audio characteristics, conversation text characteristics and conversation text confidentiality characteristics in the service provider's language characteristics; The second identification module is used to identify whether the service environment where the service recipient is located is a safe state according to the conversation environment features, conversation audio features, conversation text features and confidentiality features of the conversation text in the language features of the service recipient; The first branch processing module is used to identify the security level corresponding to the real-time conversation voice, call the security encryption policy corresponding to the security level to encrypt the real-time conversation voice, and seal the encryption result to a preset supervision end if the service provider meets the preset service security requirements and the service environment of the service recipient is in a safe state; The second branch processing module is used to send a service suspension prompt to the service provider and a service environment change prompt to the service provider if the service provider meets the preset service security requirements but the service environment of the service recipient is in an unsafe state; The third branch processing module is used to send a service rectification instruction to the service provider and a service risk reminder to the service provider if the service provider does not meet the preset service security requirements but the service environment of the service recipient is in a safe state; The fourth branch processing module is used to send a service alarm prompt to the supervision end if the service provider does not meet the preset service security requirements and the service environment of the service recipient is also in an unsafe state.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the conversation speech supervision method as described in any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the conversational speech supervision method according to any one of claims 1 to 7.

Images