Abnormal behavior recognition method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses an abnormal behavior recognition method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: acquiring voice data in a call process, and acquiring user portrait data corresponding to the call; extracting a first voiceprint feature and a synthetic voice detection feature of the voice data, and determining a user behavior feature according to the user portrait data; determining a call content text corresponding to the voice data, and extracting a text semantic feature corresponding to the call content text; and adopting an abnormal behavior recognition model to analyze the first voiceprint feature, the synthetic voice detection feature, the user behavior feature and the text semantic feature to obtain a recognition result corresponding to the call, the recognition result being used for representing whether the call relates to an abnormal behavior. According to the method and the device, the technical problem that the abnormal behavior recognition accuracy is poor due to the fact that the abnormal behavior in the call process is detected only by depending on a text or voiceprint single mode in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of network communication security technology, and more specifically, to an abnormal behavior identification method, device, electronic device, and non-volatile storage medium. Background Art

[0002] With the development of information technology, abnormal behavior in telecommunications networks (e.g., online fraud) has become increasingly diverse and complex, threatening the information security of individuals and society. Related technologies for detecting abnormal behavior in telecommunications networks primarily rely on single-modal analysis, such as text content analysis or voiceprint recognition. These technologies have significant limitations, resulting in low accuracy in identifying abnormal behavior and failing to effectively meet the demand for precise detection of abnormal behavior in telecommunications networks.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, electronic device, and non-volatile storage medium for identifying abnormal behavior, so as to at least solve the technical problem of poor accuracy in identifying abnormal behavior caused by the fact that related technologies rely only on a single modality, such as text or voiceprint, to detect abnormal behavior during calls.

[0005] According to one aspect of an embodiment of the present application, a method for identifying abnormal behavior is provided, including: obtaining voice data during a call, and obtaining user portrait data corresponding to the call, wherein the user portrait data is used to characterize basic personal information and network behavior data of both parties to the call; extracting a first voiceprint feature and a synthesized voice detection feature of the voice data, and determining the user behavior feature based on the user portrait data, wherein the synthesized voice detection feature is used to characterize the distribution of Mel spectrum mutation points in the voice spectrum of the voice data; determining the call content text corresponding to the voice data, and extracting text semantic features corresponding to the call content text; using an abnormal behavior recognition model to analyze the first voiceprint feature, the synthesized voice detection feature, the user behavior feature, and the text semantic features to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

[0006] Optionally, an abnormal behavior recognition model is used to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature to obtain the recognition result corresponding to the call, including: determining a first similarity parameter between the first voiceprint feature and the historical voiceprint feature, and a second similarity parameter between the first voiceprint feature and the second voiceprint feature, and determining a first risk score based on the first similarity parameter and the second similarity parameter, wherein the historical voiceprint feature is the voiceprint feature of the call object in the historical call of the number corresponding to the call, and the second voiceprint feature is the voiceprint feature of the object determined to have abnormal behavior in the historical recognition process and stored in the abnormal object voiceprint library; determining the second risk score based on the synthesized speech detection feature, wherein, The second risk score is used to characterize the probability that the voice data is machine-synthesized voice; determine the third similarity parameter between the user behavior characteristics and the historical user behavior characteristics, and the fourth similarity parameter between the user behavior characteristics and the abnormal behavior characteristics, and determine the third risk score based on the third similarity parameter and the fourth similarity parameter; determine the fifth similarity parameter between the text semantic characteristics and the vector features corresponding to the abnormal behavior sensitive words, and determine the fourth risk score based on the fifth similarity parameter, wherein the abnormal behavior sensitive words are pre-stated words whose frequency of appearance during the abnormal behavior call exceeds a preset frequency threshold; determine the recognition result corresponding to the call based on the first risk score, the second risk score, the third risk score and the fourth risk score.

[0007] Optionally, determining the recognition result corresponding to the call based on the first risk score, the second risk score, the third risk score and the fourth risk score includes: determining the risk level of the scenario corresponding to the call, and determining the target risk score threshold based on the risk level; using a gated attention mechanism to determine the weight coefficients corresponding to the first risk score, the second risk score, the third risk score and the fourth risk score respectively based on the information of the scenario; determining the target risk score corresponding to the call based on the first risk score, the second risk score, the third risk score, the fourth risk score, and the corresponding weight coefficients; and determining the recognition result corresponding to the call by comparing the size relationship between the target risk score and the target risk score threshold.

[0008] Optionally, the abnormal behavior recognition model is a student model deployed on the edge device, and the student model is obtained by knowledge distillation based on the teacher model deployed on the central server; wherein, the training steps of the teacher model include: obtaining a first training data set, wherein the first training data set contains voice data samples of multiple calls in different environments using different languages ​​by different groups of people, and user portrait data corresponding to the calls, and the voice data samples include: positive samples corresponding to normal calls and negative samples corresponding to calls involving abnormal behavior; performing adversarial sample enhancement processing on the first training data set to obtain a second training data set, wherein the adversarial sample enhancement processing is used to enhance the diversity of sample data in the data set; determining the target features corresponding to the voice data samples in the second training data set, wherein the target features include at least one of the following: voiceprint features, synthetic speech detection features, text semantic features, and user behavior features; using the target features corresponding to the voice data samples to train the initial model to obtain a teacher model.

[0009] Optionally, adversarial sample enhancement processing is performed on the first training data set to obtain a second training data set, including: determining the scenario corresponding to the call, adding background noise corresponding to the scenario to the voice data sample corresponding to the call, and adjusting the speaking speed and pitch of the voice data sample to obtain a new voice data sample; and / or, using a speech synthesis tool to generate a synthesized speech containing the call content corresponding to the abnormal behavior, and adding the synthesized speech as a negative sample to the first training data set; and / or, adding mutation points to the Mel spectrum of the negative sample in the first training data set to simulate the abnormal characteristics of the synthesized speech; and / or, adding user portrait data with abnormal risks to the negative samples in the first training data set, wherein the abnormal risk includes at least one of the following: the Internet Protocol address corresponding to the call object is an abnormal address, and the historical network behavior data of the call object characterizes that the call object is involved in abnormal behavior.

[0010] Optionally, the target features corresponding to the speech data samples are used to train the initial model to obtain the teacher model, including: using the initial model to determine the predicted labels corresponding to the speech data samples based on the target features; determining the loss function value based on the predicted labels and the true labels corresponding to the speech data samples in the second training data set, wherein the loss function is jointly determined based on the text semantic classification loss, the voiceprint authenticity discrimination loss, the synthesized speech detection loss, and the behavior risk assessment loss; and adjusting the model parameters of the initial model based on the loss function value to obtain the teacher model.

[0011] Optionally, after obtaining the teacher model, the method also includes: determining the computing power and load status of the edge device, and determining the model parameter quantity of the student model based on the computing power and load status; performing knowledge distillation on the teacher model according to the model parameter quantity to obtain a student model, wherein the loss function applied in the knowledge distillation process is jointly determined based on the soft label distillation loss, task loss, feature alignment loss, and privacy enhancement loss, the soft label distillation loss is used to characterize the degree of difference between the predicted label output by the student model and the predicted label output by the teacher model, the task loss is used to characterize the degree of difference between the predicted label output by the student model and the true label in the second training set, the feature alignment loss is used to characterize the degree of difference between the intermediate layer features of the student model and the intermediate layer features of the teacher model, and the privacy enhancement loss is used to characterize the degree of protection of the student model for the input privacy data.

[0012] According to another aspect of an embodiment of the present application, an abnormal behavior identification device is also provided, including: a call data acquisition module, used to obtain voice data during a call, and obtain user portrait data corresponding to the call, wherein the user portrait data is used to characterize the personal basic information and network behavior data of both parties to the call; a first feature extraction module, used to extract a first voiceprint feature and a synthesized voice detection feature of the voice data, and determine the user behavior feature based on the user portrait data, wherein the synthesized voice detection feature is used to characterize the distribution of Mel spectrum mutation points in the voice spectrum of the voice data; a second feature extraction module, used to determine the call content text corresponding to the voice data, and extract the text semantic features corresponding to the call content text; an abnormal behavior classification module, used to use an abnormal behavior recognition model to analyze the first voiceprint feature, the synthesized voice detection feature, the user behavior feature and the text semantic features to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

[0013] According to another aspect of the embodiments of the present application, an electronic device is provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the abnormal behavior identification method is executed when the program is run.

[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided. The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the abnormal behavior identification method by running the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which implements the steps of the abnormal behavior identification method when executed by a processor.

[0016] In an embodiment of the present application, voice data during a call is obtained, and user portrait data corresponding to the call is obtained, wherein the user portrait data is used to characterize the basic personal information and network behavior data of both parties to the call; the first voiceprint feature and the synthesized voice detection feature of the voice data are extracted, and the user behavior feature is determined based on the user portrait data, wherein the synthesized voice detection feature is used to characterize the distribution of Mel spectrum mutation points in the voice spectrum of the voice data; the call content text corresponding to the voice data is determined, and the text semantic feature corresponding to the call content text is extracted; an abnormal behavior recognition model is used to analyze the first voiceprint feature, the synthesized voice detection feature, the user behavior feature and the text semantic feature to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior. Through multi-modal (such as text, voiceprint, synthesized voice detection, user network behavior, etc.) fusion analysis, the purpose of improving the accuracy of abnormal behavior recognition is achieved, thereby solving the technical problem of poor accuracy of abnormal behavior recognition caused by the fact that the relevant technology only relies on a single modality of text or voiceprint to detect abnormal behavior during a call. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for abnormal behavior identification provided in an embodiment of the present application;

[0019] Figure 2 This is a schematic diagram of a method flow for identifying abnormal behavior provided in an embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of the architecture of a method for detecting abnormal telecommunication behavior based on multimodal adversarial distillation and association with network behavior, provided in accordance with an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a gated attention mechanism provided according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of a distillation framework provided according to an embodiment of the present application;

[0023] Figure 6 Schematic diagram of the structure of an abnormal behavior recognition device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] To facilitate those skilled in the art to better understand the embodiments of the present application, some technical terms or nouns involved in the embodiments of the present application are explained as follows:

[0027] ASR (Automatic Speech Recognition): Converts speech signals into text and supports end-to-end modeling (such as the Conformer architecture).

[0028] Voiceprint Anti-Spoofing: This function identifies synthesized speech or counterfeit voiceprints by analyzing speech spectral features (such as MFCC (Mel-Frequency Cepstral Coefficients) and fundamental frequency) and dynamic behavioral characteristics (such as speech rate fluctuations).

[0029] Adversarial Distillation: Introducing adversarial sample enhancement into the knowledge distillation process to improve the robustness of the student model against unknown attacks.

[0030] Dynamic Thresholding: Adaptively adjusts the threshold for abnormal behavior based on call scenarios (such as urgent transfers and requests for personal information).

[0031] Relevant technologies have obvious shortcomings in dealing with abnormal behavior in AI-based telecommunications networks, including the lack of synthetic voice detection, neglect of network behavior analysis, lack of dynamic adaptability and robustness, and poor recognition efficiency in high-risk scenarios.

[0032] In order to solve the above problems, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0033] According to an embodiment of the present application, an embodiment of a method for identifying abnormal behavior is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or electronic device) for implementing an abnormal behavior identification method. Figure 1 As shown, the computer terminal 10 (or electronic device) may include one or more (illustrated as 102a, 102b, ..., 102n in the figure) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0035] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or electronic device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the abnormal behavior identification method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above-mentioned abnormal behavior identification method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0038] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or electronic device).

[0039] In the above operating environment, the embodiment of the present application provides a method for identifying abnormal behavior. Figure 2 is a schematic diagram of a method flow for identifying abnormal behavior according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0040] Step S202: Acquire voice data during the call and obtain user profile data corresponding to the call, wherein the user profile data is used to represent basic personal information and network behavior data of both parties of the call;

[0041] Step S204: extracting a first voiceprint feature and a synthesized speech detection feature from the speech data, and determining user behavior features based on the user portrait data, wherein the synthesized speech detection feature is used to characterize the distribution of Mel-spectrogram mutation points in the speech spectrum of the speech data;

[0042] Step S206, determining the call content text corresponding to the voice data, and extracting text semantic features corresponding to the call content text;

[0043] Step S208: Using an abnormal behavior recognition model, the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature are analyzed to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

[0044] Through the above steps, through multi-modal (such as text, voiceprint, synthetic speech detection, user network behavior, etc.) fusion analysis, the purpose of improving the accuracy of abnormal behavior identification is achieved, thereby solving the technical problem of poor accuracy in abnormal behavior identification caused by the fact that related technologies only rely on a single modality of text or voiceprint to detect abnormal behavior during calls.

[0045] The following further introduces the abnormal behavior identification method in steps S202 to S208 of the embodiment of the present application.

[0046] Figure 3 Schematic diagram of the architecture of a method for detecting abnormal telecommunication behavior based on multimodal adversarial distillation and network behavior association according to an embodiment of the present application, such as Figure 3 As shown, the embodiment of the present application improves the recognition coverage of AIGC attacks through the joint modeling of four modes: ASR text analysis, voiceprint authentication, synthetic speech detection, and network behavior; and designs an adaptive distillation strategy based on task complexity to support real-time reasoning of edge devices (<200ms response); in addition, a dynamic threshold mechanism can be introduced to improve the judgment sensitivity for high-risk scenarios (such as transfer requests), which is explained in detail below.

[0047] When identifying abnormal behavior (for example, telecommunications fraud), after acquiring the voice data during a call, Voice Activity Detection (VAD) technology can be used to capture valid segments and simultaneously output ASR text (i.e., the call content text corresponding to the voice data). For example, a speech recognition submodule (such as an end-to-end Conformer model that supports dialect recognition) can be used to generate the call content text.

[0048] In addition, user portrait data corresponding to both parties of the call can also be obtained, including but not limited to: age, gender, region, IP address, network behavior data, etc.

[0049] Afterwards, the first voiceprint features and synthesized speech detection features of the voice data can be extracted, and the user behavior features can be determined based on the user portrait data, as well as the text semantic features corresponding to the call content text. The first voiceprint features, synthesized speech detection features, user behavior features and text semantic features are further analyzed, as follows.

[0050] In some embodiments of the present application, an abnormal behavior recognition model is used to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature to obtain the recognition result corresponding to the call, including: determining a first similarity parameter between the first voiceprint feature and the historical voiceprint feature, and a second similarity parameter between the first voiceprint feature and the second voiceprint feature, and determining a first risk score based on the first similarity parameter and the second similarity parameter, wherein the historical voiceprint feature is the voiceprint feature of the call object in the historical call of the number corresponding to the call, and the second voiceprint feature is the voiceprint feature of the object determined to have abnormal behavior in the historical recognition process stored in the abnormal object voiceprint library; determining the second risk score based on the synthesized speech detection feature. , wherein the second risk score is used to characterize the probability that the voice data is machine-synthesized voice; determining a third similarity parameter between the user behavior characteristics and the historical user behavior characteristics, and a fourth similarity parameter between the user behavior characteristics and the abnormal behavior characteristics, and determining a third risk score based on the third similarity parameter and the fourth similarity parameter; determining a fifth similarity parameter between the text semantic characteristics and the vector features corresponding to the abnormal behavior sensitive words, and determining a fourth risk score based on the fifth similarity parameter, wherein the abnormal behavior sensitive words are pre-stated words whose frequency of appearance during the abnormal behavior call exceeds a preset frequency threshold; determining the recognition result corresponding to the call based on the first risk score, the second risk score, the third risk score and the fourth risk score.

[0051] Specifically, an abnormal behavior recognition model can be adopted, and a voiceprint authentication sub-module can be used to extract the dynamic GMM-UBM (Dynamic Gaussian Mixture Model-Universal Background Model) voiceprint feature (i.e., the first voiceprint feature) of the voice data. Then, by analyzing the first voiceprint feature of the user in the current call voice, the similarity with the voiceprint feature in the historical calls of the user number, and the similarity between the first voiceprint feature and the second voiceprint feature in the known fraud voiceprint library, the first risk score of the voiceprint counterfeiting can be determined. At the same time, speech rate fluctuation analysis (such as abnormal pause detection) can also be combined to further improve the accuracy of identifying counterfeit voiceprints.

[0052] At the same time, the synthetic speech detection submodule can be used to identify AI-generated speech (such as WaveNet and Tacotron 2) through spectrum anomaly detection (such as Mel spectrum mutation point analysis) and adversarial sample separation technology to obtain a second risk score corresponding to the confidence of the synthetic speech.

[0053] When analyzing user network behavior, on the one hand, IP traceability and risk analysis (IP risk scoring) can be performed by collecting the IP addresses of both parties to the call, verifying the legitimacy of the IP (such as the organization to which it belongs, geographical location) in conjunction with the registered information database, and detecting abnormal IP behavior (such as high-frequency cross-regional calls, use of virtual IP addresses). On the other hand, the calculation of the degree of deviation of user behavior characteristics (user behavior anomaly index) can be combined (for example, risk analysis can be performed by calculating the similarity between user behavior characteristics and the user's historical user behavior characteristics, and the similarity between user behavior characteristics and abnormal behavior characteristics) to determine a third risk score (such as elderly people + unfamiliar IP + large transfers triggering high-risk labels). In addition, dynamic portraits can be constructed by aggregating basic user information (such as age, occupation, etc.), historical call records, transaction behavior, etc., and analyzing user social relationship networks through graph neural networks (GNN) to identify abnormal associations (such as frequent interactions with high-risk accounts).

[0054] In addition, the semantic similarity between the text semantic features of the call content and the preset abnormal behavior sensitive words can be calculated to analyze whether the current call content involves abnormal behavior and obtain a fourth risk score.

[0055] After obtaining the risk scores corresponding to the features of the four modal dimensions of text, voiceprint, synthetic detection, and user network behavior, the gated attention mechanism can be used to dynamically assign weight coefficients corresponding to these four modal dimensions and combine them with dynamic threshold rules to determine the recognition result corresponding to the call, as shown below.

[0056] In some embodiments of the present application, determining the recognition result corresponding to a call based on the first risk score, the second risk score, the third risk score, and the fourth risk score includes: determining the risk level of the scenario corresponding to the call, and determining a target risk score threshold based on the risk level; using a gated attention mechanism to determine the weight coefficients corresponding to the first risk score, the second risk score, the third risk score, and the fourth risk score, respectively, based on information of the scenario; determining the target risk score corresponding to the call based on the first risk score, the second risk score, the third risk score, the fourth risk score, and the corresponding weight coefficients; and determining the recognition result corresponding to the call by comparing the size relationship between the target risk score and the target risk score threshold.

[0057] Specifically, in this embodiment, the risk level of the current call scenario can be determined first, so as to dynamically adjust the target risk score threshold. For example, for high-risk scenarios (such as transfer requests, elderly users + unfamiliar IP + large transfer requests, etc.), the threshold can be lowered to 0.6, and the false alarm rate can be reduced by 35%; for ordinary scenarios, the threshold can be maintained at 0.85 to balance precision and recall.

[0058] Furthermore, a gated attention mechanism can be used to dynamically assign weights to the four modalities of text, voiceprint, synthetic detection, and user network behavior based on scene information. Specifically, the feature distributions of multimodal data (such as text, voiceprint, user behavior, etc.) vary greatly, and the gating mechanism needs to learn the contribution mapping relationship of different modalities through training to eliminate the semantic gap between heterogeneous features. Through dynamic adjustment, misjudgments caused by single-modal noise (such as dialect accents) can be avoided. For example, when the voiceprint weight is reduced, it relies on text and user behavior verification; the gated attention mechanism can dynamically assign weights based on context perception to resist multiple attacks, solving the problems of high false alarm rate and poor robustness of traditional methods in related technologies in complex scenarios.

[0059] like Figure 4 As shown in Figure 2, the gated attention mechanism combines attention weight calculation with gating signal control to dynamically allocate the contribution of multimodal features. Multimodal features (text, voiceprint, synthesized speech detection, and online behavior) are input, and the self-attention mechanism calculates the initial weight of each modality (e.g., 40% for text, 30% for voiceprint). Next, the gated layer uses a sigmoid function to generate dynamic coefficients (incorporating context such as scene labels) to adjust the attention weights (e.g., increasing the weight of online behavior in high-risk scenarios).

[0060] After determining the weight coefficients for each modal feature, a weighted fusion approach can be used to output a final target risk score. Dynamic thresholds are then used to trigger the final recognition results and decisions. For example, if a call is identified as involving abnormal behavior (i.e., the target risk score exceeds the threshold), the call can be blocked in real time, automatically terminated, and account transactions frozen. Blockchain can also be used to store high-risk event IP logs, user behavior, and other information on the chain to support forensics.

[0061] For example, assuming that the current call involves an AI synthesized voice fraud scenario, the process of identifying abnormal behavior in the current call is as follows: the user receives a voice call, and the ASR outputs the text "There is an abnormality in your account and you need to transfer money to a safe account immediately"; the voiceprint authentication detects abnormal fundamental frequency fluctuations (82% match with the known fraud voiceprint library); the synthesized voice detection recognizes the presence of WaveNet generation features in the audio (91% confidence); the multimodal model comprehensively determines that the probability of fraud is 95%, and the system blocks the call in real time and reports it to the regulatory platform.

[0062] Assuming that the current call involves a cross-language fraud attack scenario, the process of identifying abnormal behavior in the current call is as follows: the attacker uses AI to generate Cantonese fraud voice, and ASR recognizes the text "Please provide a verification code to lift account restrictions"; the voiceprint authentication detects an abnormal dialect accent (73% different from the user's historical voiceprint); the synthetic voice detection recognizes the traces generated by Tacotron2; the system triggers dynamic threshold adjustment, determines it as high risk, and automatically hangs up.

[0063] Assuming that the current call involves a cross-regional IP spoofing attack scenario, the process of identifying abnormal behavior in the current call is as follows: a user (elderly group) receives a strange call, and IP tracing shows that the location is region A (a high-risk area); ASR recognition text "Medical insurance account needs to be verified" triggers fraud keywords; user portraits show that there are no similar call records in the past three days, and the deviation from historical behavior is 82%; the multimodal model comprehensively determines that the probability of fraud is 93%, and the system automatically hangs up and sends an anti-fraud SMS.

[0064] Assuming that the current call involves a virtual IP + synthesized voice combined attack scenario, the process of identifying abnormal behavior in the current call is as follows: the attacker uses a virtual IP to call the user and synthesizes the voice to simulate bank customer service; the voiceprint authentication detects abnormal fundamental frequency fluctuations (with a 79% match with the known fraud voiceprint database); the user portrait shows an account balance of 500,000 yuan and no recent overseas transaction records; the system triggers a multi-dimensional decision rule (IP risk 0.85 + voiceprint anomaly 0.82 + high user assets), determines it as a high-risk event, and freezes the account.

[0065] The following further introduces the training and deployment process of the abnormal behavior recognition model used in the embodiments of the present application, as follows.

[0066] In some embodiments of the present application, the abnormal behavior recognition model is a student model deployed on an edge device, and the student model is obtained by knowledge distillation based on a teacher model deployed on a central server; wherein, the training steps of the teacher model include: obtaining a first training data set, wherein the first training data set contains voice data samples of multiple calls in different environments using different languages ​​by different groups of people, and user portrait data corresponding to the calls, and the voice data samples include: positive samples corresponding to normal calls and negative samples corresponding to calls involving abnormal behavior; performing adversarial sample enhancement processing on the first training data set to obtain a second training data set, wherein the adversarial sample enhancement processing is used to enhance the diversity of sample data in the data set; determining the target features corresponding to the voice data samples in the second training data set, wherein the target features include at least one of the following: voiceprint features, synthetic speech detection features, text semantic features, and user behavior features; using the target features corresponding to the voice data samples to train the initial model to obtain a teacher model.

[0067] like Figure 5 As shown, a dynamic knowledge distillation architecture is adopted in the embodiment of the present application, adversarial sample enhancement is used in the training phase, and a lightweight sub-model (i.e., an abnormal behavior recognition model, such as MobileBERT) is generated through dynamic distillation in the deployment phase, which can support edge computing deployment and make the inference speed <200ms to meet the needs of real-time blocking.

[0068] Specifically, a training data set can be collected first to obtain voice data samples (including normal (positive) and abnormal (negative) samples) from multiple calls between different groups of people in different languages ​​and in different environments. At the same time, user profile data such as the IP addresses, device information, basic user information (age / occupation), and historical transaction records of both parties can be collected to obtain the first training data set. When collecting data, as many dimensions as possible can be covered. For example, data can be collected in layers by gender (male / female), age group (youth / middle-aged / elderly), and occupation (student / white-collar / retired); covering dialects such as Cantonese, Sichuanese, and Minnan; and collecting fraudulent rhetoric in a variety of different emergency scenarios (such as "account freezing") and inducement scenarios (such as "winning notification").

[0069] Afterwards, the first training dataset can be enhanced with adversarial samples by injecting generated synthetic speech, virtual IP attack samples, and other adversarial samples to improve the diversity of the training set data, as follows.

[0070] In some embodiments of the present application, adversarial sample enhancement processing is performed on the first training data set to obtain the second training data set, including: determining the scenario corresponding to the call, adding background noise corresponding to the scenario to the voice data sample corresponding to the call, and adjusting the speaking speed and pitch of the voice data sample to obtain a new voice data sample; and / or, using a speech synthesis tool to generate a synthesized speech containing the call content corresponding to the abnormal behavior, and adding the synthesized speech as a negative sample to the first training data set; and / or, adding mutation points to the Mel spectrum of the negative sample in the first training data set to simulate the abnormal characteristics of the synthesized speech; and / or, adding user portrait data with abnormal risks to the negative samples in the first training data set, wherein the abnormal risk includes at least one of the following: the Internet Protocol address corresponding to the call object is an abnormal address, and the historical network behavior data of the call object characterizes that the call object is involved in abnormal behavior.

[0071] Specifically, for voice data samples, the enhancement processing that can be performed includes but is not limited to: 1) Noise injection: adding a noise background according to the scene (such as streets, subways, schools, high-speed railways, etc.) to simulate a real call environment; 2) Speed ​​and pitch change: adjusting the speaking speed (±20%) and pitch (±10%) to enhance the robustness of voiceprint authentication; 3) Synthetic speech generation: using speech synthesis tools such as WaveNet and Tacotron 2 to generate realistic speech, simulate fraud scenarios (such as "transferring money to a safe account"), covering multiple modalities such as text, voiceprints, and synthetic detection; 4) Spectral perturbation injection: adding mutation points (such as high-frequency noise) to the Mel spectrum to simulate the abnormal characteristics of AI-generated speech.

[0072] In addition, the user portrait data corresponding to the voice data sample can also be enhanced with adversarial samples. For example, legal data can be collected first, including: 1) basic attributes: age, gender, occupation (obtained through operator data desensitization), geographical distribution (combined with IP address and device GPS positioning), etc.; 2) behavioral data: call records (frequency, duration, contact type), transaction behavior (amount, frequency, risk level of the recipient), etc.; and, generation of abnormal data: 1) Generate abnormal behavior of high-risk users for training the network behavior analysis layer; 2) Use GNN to simulate abnormal social relationships (such as frequent interactions with fraudulent accounts) to enhance the robustness of the dynamic risk map.

[0073] Alternatively, the IP data in the user portrait data can also be enhanced, including: building a whitelist using the registered IP database; converting it into structured data through tools; generating virtual IPs and abnormal behaviors (high-frequency cross-regional calls); simulating abnormal behavior to attack group IPs (the same IP controls multiple virtual numbers).

[0074] Through the adversarial sample injection strategy (dynamic mixed injection), a second training data set with more diverse data can be obtained; further, the target features corresponding to the samples in the second training data set can be determined, for example, by integrating text semantic features (768-dimensional BERT sentence vector), voiceprint features (128 dimensions), synthetic detection features (64 dimensions), and network features (IP score + behavior index, a total of 64 dimensions) to form a 928-dimensional joint feature vector.

[0075] It should be noted that in the embodiments of the present application, based on the need for privacy protection, a federated learning framework can be used to encrypt sensitive data (such as transaction records) to ensure compliance.

[0076] Afterwards, the above target features can be used to train the teacher model as follows.

[0077] In some embodiments of the present application, the target features corresponding to the speech data samples are used to train the initial model to obtain the teacher model, including: using the initial model to determine the predicted labels corresponding to the speech data samples based on the target features; determining the loss function value based on the predicted labels and the true labels corresponding to the speech data samples in the second training data set, wherein the loss function is jointly determined based on the text semantic classification loss, the voiceprint authenticity discrimination loss, the synthesized speech detection loss, and the behavior risk assessment loss; and adjusting the model parameters of the initial model based on the loss function value to obtain the teacher model.

[0078] In this embodiment, a multi-task learning head can be designed based on a pre-trained multimodal large model, and multi-task learning can be performed based on Qwen-Audio. The training can be performed simultaneously on abnormal behavior text classification (cross entropy loss), voiceprint authenticity discrimination (binary classification loss), synthetic speech detection (MSE (Mean Squared Error) loss), and network behavior risk assessment (regression loss). Specifically, the model parameters can be updated through backpropagation of the joint loss function, forcing the model to learn cross-modal associations between multiple modalities during training. The joint loss function is shown in the following formula:

[0079]

[0080] Among them, λ1-λ4 are dynamic weights, They correspond to text semantic classification loss, voiceprint authenticity discrimination loss, synthetic speech detection loss, and behavioral risk assessment loss respectively.

[0081] After the teacher model training is completed, dynamic knowledge distillation can be further used to achieve lightweight deployment of the student model (abnormal behavior recognition model), as follows.

[0082] In some embodiments of the present application, after obtaining the teacher model, the method also includes: determining the computing power and load status of the edge device, and determining the model parameter quantity of the student model based on the computing power and load status; performing knowledge distillation on the teacher model according to the model parameter quantity to obtain a student model, wherein the loss function applied in the knowledge distillation process is jointly determined based on the soft label distillation loss, task loss, feature alignment loss, and privacy enhancement loss, the soft label distillation loss is used to characterize the degree of difference between the predicted label output by the student model and the predicted label output by the teacher model, the task loss is used to characterize the degree of difference between the predicted label output by the student model and the true label in the second training set, the feature alignment loss is used to characterize the degree of difference between the intermediate layer features of the student model and the intermediate layer features of the teacher model, and the privacy enhancement loss is used to characterize the degree of protection of the student model for the input privacy data.

[0083] Specifically, we can adaptively distill sub-models (student models) with different parameter sizes (such as MobileBERT (128 dimensions) or TinyBERT (64 dimensions)) from the teacher model based on the computing power and load status of the device during deployment. The feature spaces of the student model and the teacher model can be aligned using the MSE loss to ensure a performance loss of less than 5% after lightweighting.

[0084] Among them, the combined loss of the distillation stage is shown as follows:

[0085]

[0086] Among them, λ1-λ4 are dynamic weights, For the soft label distillation loss, the cross entropy can be calculated using the soft labels output by the teacher model and the output of the student model, as shown in the following formula:

[0087]

[0088] Among them, T(x) is the predicted label output by the teacher model, and S(x) is the predicted label output by the student model.

[0089] is the task loss, as shown below:

[0090]

[0091] Among them, y is the true label.

[0092] Feature alignment loss can align the intermediate layer features of the teacher and student models through MSE loss, as shown in the following formula:

[0093]

[0094] in, is the middle layer feature of the teacher model, is the intermediate layer feature of the student model.

[0095] Privacy enhancement loss is used to add noise to the loss function to prevent model inversion attacks, as shown in the following formula:

[0096]

[0097] User data is encrypted locally before participating in distillation to prevent leakage of original data. Furthermore, by using a noise injection defense mechanism, differential privacy noise is added during the distillation process to further defend against model inversion attacks.

[0098] Through multimodal fusion analysis, this application solution significantly improves the accuracy and coverage of recognition, especially when dealing with complex scenarios such as AI-generated voice and cross-regional abnormal behavior attacks. Its robustness and real-time response capabilities have been greatly improved. It has significant advantages in coverage, recognition accuracy, real-time response, robustness, multimodal fusion, privacy protection and dynamic adaptability, providing strong technical support for real-time detection and defense of abnormal behavior in telecommunications networks, as shown in the following table.

[0099]

[0100] According to an embodiment of the present application, an embodiment of an abnormal behavior identification device is also provided. Figure 6 This is a schematic diagram of the structure of an abnormal behavior identification device provided according to an embodiment of the present application. Figure 6 As shown, the device includes:

[0101] A call data acquisition module 60 is used to acquire voice data during a call and to acquire user profile data corresponding to the call, wherein the user profile data is used to characterize the basic personal information and network behavior data of both parties of the call;

[0102] A first feature extraction module 62 is configured to extract a first voiceprint feature and a synthesized speech detection feature from the speech data, and determine user behavior characteristics based on the user profile data. The synthesized speech detection feature is used to characterize the distribution of Mel-spectrogram mutation points in the speech spectrum of the speech data.

[0103] A second feature extraction module 64 is used to determine the call content text corresponding to the voice data and extract text semantic features corresponding to the call content text;

[0104] The abnormal behavior classification module 66 is used to use the abnormal behavior recognition model to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature and the text semantic feature to obtain the recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

[0105] Optionally, an abnormal behavior recognition model is used to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature to obtain the recognition result corresponding to the call, including: determining a first similarity parameter between the first voiceprint feature and the historical voiceprint feature, and a second similarity parameter between the first voiceprint feature and the second voiceprint feature, and determining a first risk score based on the first similarity parameter and the second similarity parameter, wherein the historical voiceprint feature is the voiceprint feature of the call object in the historical call of the number corresponding to the call, and the second voiceprint feature is the voiceprint feature of the object determined to have abnormal behavior in the historical recognition process and stored in the abnormal object voiceprint library; determining the second risk score based on the synthesized speech detection feature, wherein, The second risk score is used to characterize the probability that the voice data is machine-synthesized voice; determine the third similarity parameter between the user behavior characteristics and the historical user behavior characteristics, and the fourth similarity parameter between the user behavior characteristics and the abnormal behavior characteristics, and determine the third risk score based on the third similarity parameter and the fourth similarity parameter; determine the fifth similarity parameter between the text semantic characteristics and the vector features corresponding to the abnormal behavior sensitive words, and determine the fourth risk score based on the fifth similarity parameter, wherein the abnormal behavior sensitive words are pre-stated words whose frequency of appearance during the abnormal behavior call exceeds a preset frequency threshold; determine the recognition result corresponding to the call based on the first risk score, the second risk score, the third risk score and the fourth risk score.

[0106] Optionally, determining the recognition result corresponding to the call based on the first risk score, the second risk score, the third risk score and the fourth risk score includes: determining the risk level of the scenario corresponding to the call, and determining the target risk score threshold based on the risk level; using a gated attention mechanism to determine the weight coefficients corresponding to the first risk score, the second risk score, the third risk score and the fourth risk score respectively based on the information of the scenario; determining the target risk score corresponding to the call based on the first risk score, the second risk score, the third risk score, the fourth risk score, and the corresponding weight coefficients; and determining the recognition result corresponding to the call by comparing the size relationship between the target risk score and the target risk score threshold.

[0107] Optionally, the abnormal behavior recognition model is a student model deployed on the edge device, and the student model is obtained by knowledge distillation based on the teacher model deployed on the central server; wherein, the training steps of the teacher model include: obtaining a first training data set, wherein the first training data set contains voice data samples of multiple calls in different environments using different languages ​​by different groups of people, and user portrait data corresponding to the calls, and the voice data samples include: positive samples corresponding to normal calls and negative samples corresponding to calls involving abnormal behavior; performing adversarial sample enhancement processing on the first training data set to obtain a second training data set, wherein the adversarial sample enhancement processing is used to enhance the diversity of sample data in the data set; determining the target features corresponding to the voice data samples in the second training data set, wherein the target features include at least one of the following: voiceprint features, synthetic speech detection features, text semantic features, and user behavior features; using the target features corresponding to the voice data samples to train the initial model to obtain a teacher model.

[0108] Optionally, adversarial sample enhancement processing is performed on the first training data set to obtain a second training data set, including: determining the scenario corresponding to the call, adding background noise corresponding to the scenario to the voice data sample corresponding to the call, and adjusting the speaking speed and pitch of the voice data sample to obtain a new voice data sample; and / or, using a speech synthesis tool to generate a synthesized speech containing the call content corresponding to the abnormal behavior, and adding the synthesized speech as a negative sample to the first training data set; and / or, adding mutation points to the Mel spectrum of the negative sample in the first training data set to simulate the abnormal characteristics of the synthesized speech; and / or, adding user portrait data with abnormal risks to the negative samples in the first training data set, wherein the abnormal risk includes at least one of the following: the Internet Protocol address corresponding to the call object is an abnormal address, and the historical network behavior data of the call object characterizes that the call object is involved in abnormal behavior.

[0109] Optionally, the target features corresponding to the speech data samples are used to train the initial model to obtain the teacher model, including: using the initial model to determine the predicted labels corresponding to the speech data samples based on the target features; determining the loss function value based on the predicted labels and the true labels corresponding to the speech data samples in the second training data set, wherein the loss function is jointly determined based on the text semantic classification loss, the voiceprint authenticity discrimination loss, the synthesized speech detection loss, and the behavior risk assessment loss; and adjusting the model parameters of the initial model based on the loss function value to obtain the teacher model.

[0110] Optionally, after obtaining the teacher model, it also includes: determining the computing power and load status of the edge device, and determining the model parameter quantity of the student model based on the computing power and load status; performing knowledge distillation on the teacher model according to the model parameter quantity to obtain the student model, wherein the loss function applied in the knowledge distillation process is jointly determined based on the soft label distillation loss, task loss, feature alignment loss, and privacy enhancement loss. The soft label distillation loss is used to characterize the degree of difference between the predicted label output by the student model and the predicted label output by the teacher model, the task loss is used to characterize the degree of difference between the predicted label output by the student model and the true label in the second training set, the feature alignment loss is used to characterize the degree of difference between the intermediate layer features of the student model and the intermediate layer features of the teacher model, and the privacy enhancement loss is used to characterize the degree of protection of the student model for the input privacy data.

[0111] It should be noted that the various modules in the above-mentioned abnormal behavior identification device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0112] It should be noted that the abnormal behavior identification device provided in this embodiment can be used to perform Figure 2 The abnormal behavior identification method shown, therefore, the relevant explanations of the above abnormal behavior identification method are also applicable to the embodiments of the present application and will not be repeated here.

[0113] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following abnormal behavior identification method by running the computer program: obtaining voice data during a call, and obtaining user portrait data corresponding to the call, wherein the user portrait data is used to characterize the basic personal information and network behavior data of both parties to the call; extracting a first voiceprint feature and a synthesized voice detection feature of the voice data, and determining the user behavior feature based on the user portrait data, wherein the synthesized voice detection feature is used to characterize the distribution of Mel spectrum mutation points in the voice spectrum of the voice data; determining the call content text corresponding to the voice data, and extracting the text semantic feature corresponding to the call content text; using an abnormal behavior recognition model, analyzing the first voiceprint feature, the synthesized voice detection feature, the user behavior feature, and the text semantic feature to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

[0114] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the abnormal behavior identification method described in each embodiment of the present application: obtaining voice data during a call, and obtaining user portrait data corresponding to the call, wherein the user portrait data is used to characterize the basic personal information and network behavior data of both parties to the call; extracting a first voiceprint feature and a synthesized voice detection feature of the voice data, and determining the user behavior feature based on the user portrait data, wherein the synthesized voice detection feature is used to characterize the distribution of Mel spectrum mutation points in the voice spectrum of the voice data; determining the call content text corresponding to the voice data, and extracting the text semantic features corresponding to the call content text; using an abnormal behavior recognition model, analyzing the first voiceprint feature, the synthesized voice detection feature, the user behavior feature, and the text semantic features to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

[0115] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0116] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0119] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0121] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying abnormal behavior, characterized in that: include: Acquire voice data during a call and obtain user profile data corresponding to the call, wherein the user profile data is used to characterize basic personal information and network behavior data of both parties to the call; Extracting a first voiceprint feature and a synthesized speech detection feature from the speech data, and determining a user behavior feature based on the user portrait data, wherein the synthesized speech detection feature is used to characterize the distribution of Mel-spectrogram mutation points in the speech spectrum of the speech data; Determining a call content text corresponding to the voice data, and extracting text semantic features corresponding to the call content text; An abnormal behavior recognition model is used to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

2. The abnormal behavior identification method according to claim 1, characterized in that: An abnormal behavior recognition model is used to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature, and the recognition result corresponding to the call is obtained, including: determining a first similarity parameter between the first voiceprint feature and a historical voiceprint feature, and a second similarity parameter between the first voiceprint feature and a second voiceprint feature, and determining a first risk score based on the first similarity parameter and the second similarity parameter, wherein the historical voiceprint feature is a voiceprint feature of a call recipient in a historical call with the number corresponding to the call, and the second voiceprint feature is a voiceprint feature of an object determined to have abnormal behavior during a historical identification process and stored in an abnormal object voiceprint library; determining a second risk score based on the synthesized speech detection feature, wherein the second risk score is used to represent a probability that the speech data is machine-synthesized speech; determining a third similarity parameter between the user behavior feature and historical user behavior features, and a fourth similarity parameter between the user behavior feature and abnormal behavior features, and determining a third risk score based on the third similarity parameter and the fourth similarity parameter; Determining a fifth similarity parameter between the text semantic feature and a vector feature corresponding to an abnormal behavior sensitive word, and determining a fourth risk score based on the fifth similarity parameter, wherein the abnormal behavior sensitive word is a word that has a frequency exceeding a preset frequency threshold during calls involving abnormal behavior, as previously determined; The recognition result corresponding to the call is determined based on the first risk score, the second risk score, the third risk score, and the fourth risk score.

3. The abnormal behavior identification method according to claim 2, characterized in that: Determining the recognition result corresponding to the call based on the first risk score, the second risk score, the third risk score, and the fourth risk score includes: Determining a risk level for the scenario corresponding to the call, and determining a target risk score threshold based on the risk level; Using a gated attention mechanism, based on information about the scenario, determine weight coefficients corresponding to the first risk score, the second risk score, the third risk score, and the fourth risk score, respectively; Determining a target risk score corresponding to the call based on the first risk score, the second risk score, the third risk score, the fourth risk score, and the corresponding weight coefficients; The recognition result corresponding to the call is determined by comparing the target risk score with the target risk score threshold.

4. The abnormal behavior identification method according to claim 1, characterized in that: The abnormal behavior recognition model is a student model deployed on the edge device. The student model is obtained by knowledge distillation based on the teacher model deployed on the central server. The training steps of the teacher model include: Obtaining a first training data set, wherein the first training data set includes voice data samples of multiple calls between different groups of people in different languages ​​and in different environments, and the user profile data corresponding to the calls, wherein the voice data samples include: positive samples corresponding to normal calls and negative samples corresponding to calls involving abnormal behavior; performing adversarial sample enhancement processing on the first training dataset to obtain a second training dataset, wherein the adversarial sample enhancement processing is used to enhance the diversity of sample data in the dataset; Determining a target feature corresponding to the speech data sample in the second training data set, wherein the target feature includes at least one of the following: a voiceprint feature, a synthesized speech detection feature, a text semantic feature, and a user behavior feature; The target features corresponding to the speech data samples are used to train the initial model to obtain the teacher model.

5. The abnormal behavior identification method according to claim 4, characterized in that: Performing adversarial sample enhancement processing on the first training dataset to obtain a second training dataset includes: Determining a scene corresponding to the call, adding background noise corresponding to the scene to a voice data sample corresponding to the call, and adjusting the speaking speed and pitch of the voice data sample to obtain a new voice data sample; and / or, using a speech synthesis tool to generate a synthesized speech containing the call content corresponding to the abnormal behavior, and adding the synthesized speech as a negative sample to the first training dataset; and / or, adding mutation points to the Mel-spectrograms of negative samples in the first training dataset to simulate abnormal features of the synthesized speech; And / or, add user portrait data with abnormal risks to the negative samples in the first training data set, wherein the abnormal risks include at least one of the following: the Internet Protocol address corresponding to the call object is an abnormal address, and the historical network behavior data of the call object indicates that the call object is involved in abnormal behavior.

6. The abnormal behavior identification method according to claim 4, characterized in that: The target feature corresponding to the speech data sample is used to train the initial model to obtain the teacher model, including: Determining a predicted label corresponding to the speech data sample using the initial model and based on the target feature; Determining a loss function value based on the predicted label and the true label corresponding to the speech data sample in the second training dataset, wherein the loss function is jointly determined based on a text semantic classification loss, a voiceprint authenticity discrimination loss, a synthesized speech detection loss, and a behavior risk assessment loss; According to the loss function value, the model parameters of the initial model are adjusted to obtain the teacher model.

7. The abnormal behavior identification method according to claim 6, characterized in that: After obtaining the teacher model, the method further includes: Determining the computing power and load status of the edge device, and determining the model parameters of the student model based on the computing power and load status; According to the model parameters, knowledge distillation is performed on the teacher model to obtain the student model, wherein the loss function applied in the knowledge distillation process is jointly determined based on soft label distillation loss, task loss, feature alignment loss, and privacy enhancement loss. The soft label distillation loss is used to characterize the degree of difference between the predicted label output by the student model and the predicted label output by the teacher model, the task loss is used to characterize the degree of difference between the predicted label output by the student model and the true label in the second training set, the feature alignment loss is used to characterize the degree of difference between the intermediate layer features of the student model and the intermediate layer features of the teacher model, and the privacy enhancement loss is used to characterize the degree of protection of the student model for the input privacy data.

8. An abnormal behavior recognition device, characterized in that: include: A call data acquisition module is used to acquire voice data during a call and obtain user profile data corresponding to the call, wherein the user profile data is used to represent the basic personal information and network behavior data of both parties of the call; a first feature extraction module, configured to extract a first voiceprint feature and a synthesized speech detection feature from the speech data, and determine a user behavior feature based on the user profile data, wherein the synthesized speech detection feature is used to characterize a distribution of Mel-spectrogram mutation points in a speech spectrum of the speech data; A second feature extraction module is used to determine the call content text corresponding to the voice data and extract text semantic features corresponding to the call content text; An abnormal behavior classification module is used to use an abnormal behavior recognition model to analyze the first voiceprint feature, the synthesized speech detection feature, the user behavior feature, and the text semantic feature to obtain a recognition result corresponding to the call, wherein the recognition result is used to characterize whether the call involves abnormal behavior.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the abnormal behavior identification method according to any one of claims 1 to 7 is executed when the program is run.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the abnormal behavior identification method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the abnormal behavior identification method according to any one of claims 1 to 7 are implemented.

Images