A real-time big data processing system

Through the big data real-time processing system, fraud detection is performed using multi-source data and LSTM models, combined with behavioral analysis and user feedback optimization, it solves the problem that traditional technology is difficult to identify private number fraud and adapt to new methods, and achieves more accurate and real-time fraud detection.

CN118838981BActive Publication Date: 2025-04-11DIGITAL BAZHOU IND DEVELOPMENT INVESTMENT (GROUP) CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410708618.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-03
Publication Date
2025-04-11
Estimated Expiration
2044-06-03

AI Technical Summary

Technical Problem

Traditional fraud detection technology is difficult to effectively intercept fraud activities conducted using private numbers, and is poorly adaptable to emerging fraud methods, resulting in lag in detection.

Method used

A real-time processing system for big data is designed to obtain fraud data from multiple sources through the data acquisition module, perform multi-feature extraction and feature matrix creation, and use long and short-term memory network (LSTM) to build a fraud detection model, combine behavior analysis and communication pattern recognition technology for real-time detection, and optimize the model through user feedback.

Benefits of technology

It realizes more accurate and real-time detection of fraudulent activities, can effectively identify new fraud methods, and improves the adaptability and real-timeness of fraud detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118838981B_ABST
    Figure CN118838981B_ABST
Patent Text Reader

Abstract

The present invention discloses a big data real-time processing system, the operation method of the system comprises the following steps: step one: multi-source fraud data collection is performed through a data collection module; step two: multi-feature extraction is performed on multi-source data, and a feature matrix is ​​created; step three: construction of a fraud detection model is realized by selecting a long short-term memory network; step four: fraud behavior detection is realized by analyzing user calls; step five: real-time fraud detection and model optimization are realized through user feedback, wherein the fraud data collection and preprocessing module is used to obtain fraud case data from multiple sources, perform cleaning and text preprocessing, and the user call analysis and real-time feedback module is used to analyze user call behavior, monitor fraud situations in real time, perform real-time fraud detection through user authorization of recording authority, and provide instant feedback. The present invention has the characteristics of more accurate fraud detection and better real-time performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a big data real-time processing system. Background Art

[0002] Current fraud detection technology mainly focuses on the analysis of call numbers and locations, using rules and statistical models to identify potential fraudulent activities. This traditional approach has significant limitations when dealing with fraudulent activities involving private numbers.

[0003] In the field of communications, fraudsters usually use private numbers to conduct fraudulent activities. By frequently changing numbers and circumventing traditional number analysis methods, traditional fraud detection methods are difficult to effectively intercept such behaviors. The use of private numbers increases the anonymity of fraudsters, making traditional fraud detection methods helpless against them. In addition, traditional rules and statistical models are poorly adaptable to emerging fraud methods, and often require manual updating of the rule base, resulting in a lag in fraud detection. Therefore, traditional technology is relatively powerless in the face of ever-changing and innovative fraud methods. Therefore, it is necessary to design a big data real-time processing system that is more accurate and real-time in fraud detection. Summary of the invention

[0004] The purpose of the present invention is to provide a big data real-time processing system to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a big data real-time processing system, the operation method of the system comprises the following steps:

[0006] Step 1: Collect multi-source fraud data through the data collection module;

[0007] Step 2: Extract multiple features from multi-source data and create a feature matrix;

[0008] Step 3: Build a fraud detection model by selecting a long short-term memory network;

[0009] Step 4: Detect fraudulent behavior by analyzing user calls;

[0010] Step 5: Real-time fraud detection and model optimization through user feedback.

[0011] According to the above technical solution, the step of collecting multi-source fraud data through the data collection module includes:

[0012] The system uses a data collection module and web crawler technology to automatically search, crawl and extract relevant fraud cases on the Internet, including but not limited to recordings of fraudulent calls, fraud analysis reports, and fraud case databases. It can also be integrated with various data sources through an API interface. Subsequently, the collected data is cleaned. First, data deduplication is achieved by comparing key fields such as case ID and timestamp. For data fields containing text information, text data preprocessing is performed to reduce text noise and improve the quality of text data through word segmentation, stop word removal, and stemming techniques.

[0013] According to the above technical solution, the step of extracting multiple features from multi-source data and creating a feature matrix includes:

[0014] Select a speech recognition engine for text conversion and build a sentiment vocabulary for sentiment analysis;

[0015] Use the PyDub speech processing library for intonation and speech rate analysis.

[0016] According to the above technical solution, the step of selecting a speech recognition engine for text conversion and constructing a sentiment vocabulary for sentiment analysis includes:

[0017] Choose a commercial speech recognition engine or an open source engine to convert the collected fraud audio data into text, and then perform sentiment analysis on the converted text. First, build a sentiment vocabulary that contains words and phrases related to different emotions. Then segment the text information into separate words. Use the built sentiment vocabulary to match each word to identify which words belong to positive, negative or neutral emotions. Finally, assign a sentiment score to each word matching result, and finally obtain the overall sentiment characteristics. In addition, when performing sentiment recognition, speech signal processing is required for speaker recognition, that is, identifying the scammers and the scammed.

[0018] According to the above technical solution, the step of using the PyDub speech processing library to analyze intonation and speech rate includes:

[0019] To analyze the intonation and speech rate of speech data, first, load the recording data by using the PyDub speech processing library. Then, frame the speech signal and divide it into short time frames. When performing intonation analysis, each frame needs to be analyzed by fundamental frequency. The periodicity of its spectrum is estimated by the Fourier transform method to obtain the basic frequency of the speech signal, that is, the intonation. At the same time, the intonation intensity analysis is performed. By calculating the amplitude or amplitude spectrum of the fundamental frequency, the intensity information of the intonation is obtained, which provides insights into the speaker's emotional state. On the other hand, by calculating the time interval between frames, the frame rate of the speech signal, that is, the speech rate, is calculated. The specific implementation method is as follows: the position of the frame in the entire speech signal, that is, the timestamp, is calculated by the sampling rate and the position of the frame. Then, the time difference between the frames is obtained by subtracting the timestamps of adjacent frames. Finally, the frame rate is calculated by taking the inverse of these time differences, that is, the number of speech frames generated by the speaker in one second. The system also calculates the average value of the frame time interval: And the variance: The speaker's speaking speed change trend in different time periods can be obtained. In addition, temporal features and feature vocabulary extraction are also required.

[0020] According to the above technical solution, the step of constructing a fraud detection model by selecting a long short-term memory network includes:

[0021] Set LSTM model parameters and use binary cross entropy as the loss function and Adam optimizer to adjust model parameters;

[0022] The early stopping method is introduced to prevent overfitting.

[0023] According to the above technical solution, the steps of setting LSTM model parameters and using binary cross entropy as the loss function and Adam optimizer to adjust the model parameters include:

[0024] The model is designed to capture the speech features and emotional information in audio clips to provide support for the accurate identification of fraud cases. Specifically, a model with two layers of LSTM units is constructed, each layer containing 50 hidden units. In the feature input stage, the timing information of each audio clip is prepared and organized into a 3D tensor with the shape of time step and number of features. The time step of each audio clip also needs to be set to 50. In addition, in terms of optimizing model parameters, binary cross entropy is selected as the loss function, and the Adam optimizer is used. The initial learning rate is set to 0.001, and the learning rate decay is used to set the decay rate to 0.9. During the training process, the system divides the data into 80% training set and 20% validation set. The batch size used for each training is 32, and the total number of training iterations is 50.

[0025] According to the above technical solution, the step of realizing fraud detection by analyzing user calls includes:

[0026] The system will also use communication pattern recognition technology to analyze call frequency, call time, and call location information. By identifying these communication patterns, the system can identify abnormal patterns that are contrary to the user's normal behavior patterns, and then infer possible fraud scenarios. When the system identifies that the user may be defrauded through the sensitivity detection mechanism, it will trigger a prompt for the user to authorize recording permissions. This prompt will be presented to the user in the form of an instant notification. When the system prompts the user, the user is required to authorize recording. After the user completes the authorization, the system transmits the user's recording data to the fraud detection module. Finally, encryption technology and security protocols are used to ensure that the user's recording data is fully protected during transmission and storage to prevent information leakage and abuse.

[0027] According to the above technical solution, the step of implementing real-time fraud detection and model optimization through user feedback includes:

[0028] Process audio data in real time and apply previously trained fraud detection algorithms;

[0029] Users provide multi-dimensional feedback, including confirming fraudulent behavior and reporting false positives;

[0030] Adopt machine learning model update strategy to adjust weights and feature weights;

[0031] Monitor performance metrics such as precision and recall;

[0032] Use the confusion matrix to evaluate how well the model classifies fraud cases.

[0033] According to the above technical solution, the system includes:

[0034] Fraud data collection and preprocessing module, which is used to obtain fraud case data from multiple sources, perform cleaning and text preprocessing to prepare the data for subsequent fraud detection model training;

[0035] The fraud detection model building module is used to select and build fraud detection models, perform model training and parameter optimization to improve the accuracy of fraud cases;

[0036] The user call analysis and real-time feedback module is used to analyze user call behavior, monitor possible fraud scenarios in real time, perform real-time fraud detection through user-authorized recording permissions, and provide instant feedback, including risk warnings and user learning process.

[0037] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention first obtains fraud case data from multiple sources through a data acquisition module, including recordings of fraudulent calls and fraud analysis reports, then performs multi-feature extraction on the multi-source data to create a feature matrix, and then constructs a fraud detection model by selecting a long short-term memory network (LSTM), trains and optimizes the extracted features, and then analyzes user calls through algorithms such as behavior analysis and communication pattern recognition, establishes a normal user behavior model, and monitors abnormal patterns in real time. When the system detects that the user may be defrauded, it triggers a prompt for the user to authorize the recording permission. After the user authorizes, fraud behavior detection is performed. Finally, real-time fraud detection and model optimization are achieved through user feedback. Users can provide different types of feedback. The system continuously improves using machine learning model update strategies to ensure that the model more accurately identifies new fraud methods. This method can effectively detect fraudulent calls and has higher real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0039] Figure 1 A flowchart of a method for real-time processing of big data provided in Embodiment 1 of the present invention;

[0040] Figure 2 A schematic diagram of the module composition of a big data real-time processing system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Embodiment 1: Figure 1 This is a flow chart of a method for real-time processing of big data provided by Embodiment 1 of the present invention. This embodiment can be applied to the scenario of fraud information identification. This method can be executed by a real-time processing system for big data provided by this embodiment. Figure 1 As shown, the method specifically comprises the following steps:

[0043] Step 1: Collect multi-source fraud data through the data collection module;

[0044] In an embodiment of the present invention, a data collection module is established in the system, which obtains fraud case data from multiple sources, including but not limited to recordings of fraud calls and fraud analysis reports;

[0045] Exemplarily, the system utilizes a data collection module and web crawler technology to automatically search, crawl and extract relevant fraud cases on the Internet, including but not limited to recordings of fraudulent calls, fraud analysis reports, fraud case databases, etc. It can also be integrated with various data sources through an API interface. Subsequently, the collected data is cleaned. First, data deduplication is achieved by comparing key fields, such as case ID and timestamp. For data fields containing text information, text data preprocessing is performed to reduce text noise and improve the quality of text data through techniques such as word segmentation, stop word removal, and stemming extraction.

[0046] Step 2: Extract multiple features from multi-source data and create a feature matrix;

[0047] In an embodiment of the present invention, a commercial speech recognition engine or an open source engine is selected to convert the collected fraudulent audio data into text, and then the converted text is subjected to sentiment analysis. First, a sentiment vocabulary is constructed, which contains words and phrases related to different emotions. Then, the text information is segmented into separate words. Each word is matched using the constructed sentiment vocabulary to identify which words belong to positive, negative or neutral emotions. Finally, a sentiment score is assigned to the result of each word matching, and the overall sentiment feature is finally obtained. In addition, when performing sentiment recognition, speech signal processing is required to perform speaker recognition, that is, to identify the fraudster and the deceived.

[0048] Exemplarily, the intonation and speech rate analysis of speech data is performed. First, the recording data is loaded by using the PyDub speech processing library. Then, the speech signal is framed and divided into short time frames. When performing intonation analysis, each frame requires fundamental frequency analysis. The Fourier transform method is used to estimate the periodicity of its spectrum, and then the basic frequency of the speech signal, that is, the intonation, is obtained. At the same time, the intonation intensity analysis is performed. By calculating the amplitude or amplitude spectrum of the fundamental frequency, the intensity information of the intonation is obtained, providing insights into the speaker's emotional state. On the other hand, by calculating the time interval between frames, the frame rate of the speech signal, that is, the speech rate, is calculated. The specific implementation method is as follows: the position of the frame in the entire speech signal, that is, the timestamp, is calculated by the sampling rate and the position of the frame. Then, the time stamps of adjacent frames are subtracted to obtain the time difference between the frames. Finally, the frame rate is calculated by taking the inverse of these time differences, that is, the number of speech frames generated by the speaker in one second. The system also calculates the average value of the frame time interval: And the variance: The speaker's speaking speed change trend in different time periods can be obtained. In addition, time series features and feature vocabulary extraction are also required. After the feature extraction is completed, a feature matrix is ​​created, and the features extracted from the intonation, speaking speed, sentiment analysis and other steps are organized into a table to ensure that each feature occupies a column. The features are standardized or normalized to ensure that they have similar scales. Secondly, the feature data is converted into a numerical matrix to ensure that the data type is consistent. Through this step, the feature information of the fraud data can be extracted, so as to better train the model.

[0049] Step 3: Build a fraud detection model by selecting a long short-term memory network;

[0050] In the embodiment of the present invention, a long short-term memory network (LSTM) is selected to build an effective fraud detection model;

[0051] Exemplarily, the model is designed to capture the speech features and emotional information in the audio clips to provide support for the accurate identification of fraud cases. Specifically, a model containing two layers of LSTM units is constructed, each layer containing 50 hidden units. In the feature input stage, the timing information of each audio clip is prepared and organized into a 3D tensor with the shape of time step and number of features. The time step of each audio clip needs to be set to 50. In addition, in terms of optimizing the model parameters, binary cross entropy is selected as the loss function, and the Adam optimizer is used. The initial learning rate is set to 0.001, and the learning rate decay is used, and the decay rate is set to 0.9. During the training process, the system divides the data into 80% training set and 20% validation set. The batch size used for each training is 32, and the total number of training iterations is 50.

[0052] For example, in order to prevent overfitting, the system introduces the early stopping method and monitors the validation set loss. When the validation set loss does not decrease in five consecutive iterations, the training process stops automatically. In the model monitoring and evaluation stage, the system uses performance indicators such as accuracy and recall for monitoring. Through tools such as TensorBoard, the system can visualize the changing trends of the loss function and accuracy. Finally, the confusion matrix is ​​generated using the test set to evaluate the classification effect of the model on fraud cases. After training, the model parameters, including weights and biases, need to be saved, and a model version management mechanism is established to track and manage different versions of models. This step ensures the effectiveness and controllability of the LSTM model in the fraud case detection task.

[0053] Step 4: Detect fraudulent behavior by analyzing user calls;

[0054] In the embodiment of the present invention, the system adopts a behavior analysis algorithm to establish a normal behavior model of the user through real-time monitoring and analysis of multi-dimensional data such as the user's dialing mode, call duration, and frequently used contacts. Moreover, through a machine learning algorithm, the system can gradually learn the user's normal behavior pattern and form a baseline understanding of normal communication behavior;

[0055] Exemplarily, the system will also use communication pattern recognition technology to analyze the frequency, time period, and location of calls. By identifying these communication patterns, the system can identify abnormal patterns that are contrary to the user's normal behavior patterns, and then infer possible fraud scenarios. When the system identifies that the user may be defrauded through the sensitivity detection mechanism, it will trigger a prompt for the user to authorize recording permissions. This prompt will be presented to the user in the form of an instant notification. When the system prompts the user, the user is required to authorize the recording. After the user completes the authorization, the system transmits the user's recording data to the fraud detection module. Finally, encryption technology and security protocols are used to ensure that the user's recording data is fully protected during transmission and storage to prevent information leakage and abuse.

[0056] Step 5: Real-time fraud detection and model optimization through user feedback.

[0057] In the embodiment of the present invention, the acquired audio data will be sent to the fraud detection model for real-time processing and analysis. At this stage, the system will pre-process the audio data, including voice signal conversion, feature extraction, etc. Then, the model will apply the fraud detection algorithm trained in the previous steps, including keyword recognition, intonation analysis, emotional feature extraction, etc., to identify potential fraudulent behavior. After the model detection is completed, the system will provide real-time feedback based on the results output by the model. If the model believes that there are signs of fraudulent behavior in the call, the system will trigger the corresponding early warning mechanism, such as immediately notifying the user, recording fraudulent behavior data, etc.;

[0058] For example, after receiving a risk warning, the user can choose different types of feedback, such as confirming whether fraud has been encountered, reporting a false positive, or providing additional information. These feedback types provide multi-dimensional information for the system's learning process. The system then adopts a machine learning model update strategy and uses the collected user feedback data to continuously improve, including updating weights and adjusting feature weights to ensure that the model can more accurately identify new fraud methods.

[0059] Embodiment 2: Embodiment 2 of the present invention provides a big data real-time processing system. Figure 2 A schematic diagram of the module composition of a big data real-time processing system provided in the second embodiment of the present invention is shown in FIG. Figure 2 As shown, the system includes:

[0060] Fraud data collection and preprocessing module, which is used to obtain fraud case data from multiple sources, perform cleaning and text preprocessing to prepare the data for subsequent fraud detection model training;

[0061] Fraud detection model building module, which is used to select and build fraud detection models (such as LSTM), perform model training and parameter optimization to improve the accuracy of fraud cases;

[0062] User call analysis and real-time feedback module, used to analyze user call behavior, monitor possible fraud scenarios in real time, conduct real-time fraud detection through user-authorized recording permissions, and provide instant feedback, including risk warnings and user learning process;

[0063] In some embodiments of the present invention, the fraud data collection and preprocessing module includes:

[0064] Data collection module, used to obtain fraud case data from multiple sources through the system's built-in data collection module;

[0065] The data cleaning and text preprocessing module is used to clean the collected data, remove duplicates by comparing key fields, and perform text preprocessing on fields containing text information;

[0066] The multi-feature extraction and feature matrix creation module is used to extract multiple features from multi-source data and organize the extracted features into a feature matrix;

[0067] In some embodiments of the present invention, the fraud detection model building module includes:

[0068] LSTM model selection and construction module, which is used to select the long short-term memory network (LSTM) as the fraud detection model and build a model containing two layers of LSTM units;

[0069] Model training and parameter optimization module, which is used to train models using fraud case data, use optimization algorithms, and adjust model parameters to improve the accuracy of fraud case identification;

[0070] Model application and version management module, which is used to apply the trained LSTM model for fraud detection, save the model parameters and architecture, and establish a model version management mechanism;

[0071] In some embodiments of the present invention, the user call analysis and real-time feedback module includes:

[0072] User behavior analysis and communication pattern recognition module, which uses behavior analysis algorithms and communication pattern recognition technology to monitor and analyze users' dialing patterns, call duration, call frequency and other multi-dimensional data in real time, and establish a normal behavior model for users;

[0073] The user authorization recording permission module is used to trigger the user's authorization recording permission prompt when the system detects that the user may be defrauded, ensuring that the user understands the operation and purpose of the system;

[0074] The real-time fraud detection and user feedback module is used to obtain user authorization, record the user's voice and apply the fraud detection algorithm trained in the previous step to detect fraudulent behavior in real-time calls.

[0075] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0076] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for real-time processing of big data, characterized by: The method comprises the following steps: Step 1: Collect multi-source fraud data through the data collection module; Step 2: Extract multiple features from multi-source data and create a feature matrix; Step 3: Build a fraud detection model by selecting a long short-term memory network; Step 4: Detect fraudulent behavior by analyzing user calls; Step 5: Real-time fraud detection and model optimization through user feedback; The step of collecting multi-source fraud data through the data collection module includes: The system utilizes a data collection module and web crawler technology to automatically search, crawl, and extract relevant fraud cases from the Internet, including but not limited to recordings of fraudulent calls, fraud analysis reports, and fraud case databases. The system can also be integrated with various data sources through API interfaces. After data collection is completed, the collected data is cleaned. First, data duplication is achieved by comparing key fields, case IDs, and timestamps. For data fields containing text information, text data preprocessing is performed to reduce text noise and improve text data quality through word segmentation, stop word removal, and stemming techniques. The step of extracting multiple features from multi-source data and creating a feature matrix includes: Select a speech recognition engine for text conversion and build a sentiment vocabulary for sentiment analysis; Use the PyDub speech processing library to analyze intonation and speech rate; The steps of selecting a speech recognition engine to perform text conversion and constructing a sentiment vocabulary to perform sentiment analysis include: Select a commercial or open-source speech recognition engine to convert the collected fraudulent audio data into text. Then, perform sentiment analysis on the converted text. First, construct a sentiment vocabulary containing words and phrases associated with different emotions. Then, perform token segmentation on the text information, breaking it down into individual words. Each word is matched against the established sentiment vocabulary to identify which words embody positive, negative, or neutral emotions. Finally, assign a sentiment score to each matching result, ultimately generating an overall sentiment profile. Furthermore, during sentiment recognition, speech signal processing is required for speaker identification, specifically to distinguish between the fraudster and the victim. The steps of using the PyDub speech processing library to analyze intonation and speech rate include: To analyze the intonation and speaking rate of speech data, first, we load the recorded data using the PyDub speech processing library. Then, we frame the speech signal and divide it into short time frames. When performing intonation analysis, each frame requires fundamental frequency analysis. The Fourier transform method is used to estimate the periodicity of its spectrum, and then the fundamental frequency of the speech signal, i.e., the intonation, is obtained. At the same time, the intonation intensity analysis is performed. By calculating the amplitude or amplitude spectrum of the fundamental frequency, the intensity information of the intonation is obtained, providing insights into the speaker's emotional state. On the other hand, by calculating the time interval between frames, the frame rate of the speech signal, i.e., the speaking rate, is calculated. The specific implementation method is as follows: the position of the frame in the entire speech signal, i.e., the timestamp, is calculated by using the sampling rate and the position of the frame. Then, the time difference between the frames is obtained by subtracting the timestamps of adjacent frames. Finally, the frame rate is calculated by taking the inverse of these time differences, i.e., the number of speech frames generated by the speaker in one second. The system also calculates the average value of the frame time interval: , and the variance: The speaker's speaking speed change trend in different time periods can be obtained. In addition, temporal features and feature vocabulary extraction are also required.

2. The method for real-time processing of big data according to claim 1, characterized in that: The step of constructing a fraud detection model by selecting a long short-term memory network includes: Set the LSTM model parameters and use binary cross entropy as the loss function and Adam optimizer for model parameter adjustment; The early stopping method is introduced to prevent overfitting.

3. The method for real-time processing of big data according to claim 2, characterized in that: The steps of setting LSTM model parameters and using binary cross entropy as the loss function and Adam optimizer to adjust the model parameters include: The model is designed to capture the speech features and emotional information in audio clips to provide support for the accurate identification of fraud cases. Specifically, a model with two layers of LSTM units is constructed, with each layer containing 50 hidden units. In the feature input stage, the timing information of each audio clip is prepared and organized into a 3D tensor with the shape of time step and number of features. The time step of each audio clip also needs to be set to 50. In addition, in terms of optimizing model parameters, binary cross entropy is selected as the loss function, and the Adam optimizer is used. The initial learning rate is set to 0.001, and the learning rate decay is used, and the decay rate is set to 0.

9. During the training process, the system divides the data into 80% training set and 20% validation set. The batch size used for each training is 32, and the total number of training iterations is 50.

4. The method for real-time processing of big data according to claim 1, wherein: The step of detecting fraudulent behavior by analyzing user calls includes: The system will also use communication pattern recognition technology to analyze the frequency, time and location of calls. By identifying these communication patterns, the system can identify abnormal patterns that are contrary to the user's normal behavior patterns, and then infer possible fraud scenarios. When the system identifies that the user may be defrauded through the sensitivity detection mechanism, it will trigger a prompt for the user to authorize recording permissions. This prompt will be presented to the user in the form of an immediate notification. When the system prompts the user, the user is required to authorize recording. After the user completes the authorization, the system transmits the user's recording data to the fraud detection module. Finally, encryption technology and security protocols are used to ensure that the user's recording data is fully protected during transmission and storage to prevent information leakage and abuse.

5. The method for real-time processing of big data according to claim 1, characterized in that: The steps of implementing real-time fraud detection and model optimization through user feedback include: Process audio data in real time and apply previously trained fraud detection models; Users provide multi-dimensional feedback, including confirming fraudulent behavior and reporting false positives; Adopt machine learning model update strategies to adjust weights and feature weights; Monitor performance metrics, such as precision and recall; Use the confusion matrix to evaluate how well the model classifies fraud cases.

6. A processing system for a real-time big data processing method according to any one of claims 1 to 5, characterized in that: The system comprises: The fraud data collection and preprocessing module is used to obtain fraud case data from multiple sources, perform cleaning and text preprocessing to prepare the data for subsequent fraud detection model training; The fraud detection model building module is used to select and build fraud detection models, perform model training and parameter optimization to improve the accuracy of fraud cases; The user call analysis and real-time feedback module is used to analyze user call behavior, monitor possible fraud scenarios in real time, perform real-time fraud detection through user-authorized recording permissions, and provide instant feedback, including risk warnings and user learning process.

Citation Information

Patent Citations

  • Fraud recognition method combining with audio analysis and video analysis, device and storage medium

    CN108053838A

  • Voice-based anomaly detection method and device, equipment and storage medium

    CN116863938A

  • Voice fraud attack detection method and device based on LSTM (Long Short Term Memory) model and electronic equipment

    CN117524233A