Data processing method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a data processing method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: acquiring network flow data generated by a business system of a financial institution within a preset duration; determining the scale of a financial institution according to the number of background servers of the service system, and determining a target rule for identifying the network flow data according to the scale of the financial institution; determining first abnormal data in the network flow data according to a target rule; inputting the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data; and releasing the first abnormal data and the second abnormal data. According to the method and the device, the technical problem that the identification accuracy of abnormal network flow data generated in a service system of a small and medium-sized financial institution is relatively low due to the fact that a related abnormal data identification model does not fully consider the service characteristics of the small and medium-sized financial institution is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular, to a data processing method and device, a non-volatile storage medium, and an electronic device. Background Art

[0002] At present, information network security of financial institutions is of paramount importance, and many items such as vulnerability repair, penetration analysis, and abnormal traffic monitoring are continuously undergoing AI-enabled transformation. However, the relevant technologies do not fully consider the business characteristics of small and medium-sized financial institutions, which leads to problems such as low accuracy in identifying abnormal network traffic data generated in the business systems of small and medium-sized financial institutions.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and device, a non-volatile storage medium, and an electronic device to at least solve the technical problem of low accuracy in identifying abnormal network traffic data generated in the business systems of small and medium-sized financial institutions due to the fact that the relevant abnormal data identification model does not fully consider the business characteristics of small and medium-sized financial institutions.

[0005] According to one aspect of an embodiment of the present application, a data processing method is provided, including: obtaining network traffic data generated by a business system of a financial institution within a preset time period; determining the scale of the financial institution based on the number of backend servers of the business system, and determining a target rule for identifying the network traffic data based on the scale of the financial institution; determining first abnormal data in the network traffic data based on the target rule; inputting the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data; and publishing the first abnormal data and the second abnormal data.

[0006] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if the number of times the abnormal Internet Protocol IP address connects to the preset port of the business system within a preset time period is greater than a first preset threshold, the network traffic data generated by the connection process is determined to be the first abnormal data, wherein the abnormal IP address is an IP address that is not in the preset IP address set.

[0007] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if the number of times the target employee account accesses a first business module in a business system that the target employee account has no authority to access within a preset time period is greater than a second preset threshold, determining that the network traffic data generated by the access process is the first abnormal data.

[0008] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if it is detected that the target data in the second business module in the business system whose importance level is greater than the preset level is transmitted to the external device of the business system, and the capacity of the target data is greater than a third preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data; if the number of data transmissions between the business system and the external payment system and / or the external credit system is greater than a fourth preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data.

[0009] Optionally, before inputting the network traffic data and the first abnormal data into the machine learning model to obtain the second abnormal data in the network traffic data, the method also includes: deploying the machine learning model when it is determined according to the target rule that the amount of first abnormal data in the network traffic data is greater than a preset threshold.

[0010] Optionally, after publishing the first abnormal data and the second abnormal data, the method further includes: obtaining an authenticity label of the second abnormal data, determining a target loss function based on a difference between the second abnormal data and the authenticity label, and iteratively training the machine learning model based on the target loss function.

[0011] Optionally, after publishing the first exception data and the second exception data, the method further includes: combining the first exception data and the second exception data with a preset sentence to obtain a target sentence, and calling the large language model through the target sentence; obtaining an output result generated by the large language model in response to the target sentence, wherein the output result at least includes: a processing solution for the exception data.

[0012] According to another aspect of the embodiment of the present application, a data processing device is also provided, including: an acquisition module, used to acquire network traffic data generated by the business system of a financial institution within a preset time period; a first determination module, used to determine the scale of the financial institution according to the number of backend servers of the business system, and determine the target rule for identifying the network traffic data according to the scale of the financial institution; a second determination module, used to determine the first abnormal data in the network traffic data according to the target rule; a third determination module, used to input the network traffic data and the first abnormal data into a machine learning model to obtain the second abnormal data in the network traffic data; and a publishing module, used to publish the first abnormal data and the second abnormal data.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data processing method.

[0014] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above data processing method is executed when the program is run.

[0015] According to yet another aspect of the embodiments of the present application, a computer program is further provided, wherein the above data processing method is implemented when the computer program is executed by a processor.

[0016] According to another aspect of the embodiments of the present application, a computer program product is provided, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above data processing method is implemented.

[0017] In an embodiment of the present application, the network traffic data generated by the business system of a financial institution within a preset time period is obtained; the scale of the financial institution is determined according to the number of backend servers of the business system, and the target rule for identifying the network traffic data is determined according to the scale of the financial institution; the first abnormal data in the network traffic data is determined according to the target rule; the network traffic data and the first abnormal data are input into a machine learning model to obtain the second abnormal data in the network traffic data; and the first abnormal data and the second abnormal data are published. This achieves the purpose of fully considering the business characteristics of small and medium-sized financial institutions, thereby achieving the technical effect of improving the accuracy of identifying abnormal network traffic data generated in the business systems of small and medium-sized financial institutions, and further solves the technical problem of low accuracy in identifying abnormal network traffic data generated in the business systems of small and medium-sized financial institutions due to the fact that the relevant abnormal data identification model does not fully consider the business characteristics of small and medium-sized financial institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a flow chart of a data processing method according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of another data processing method according to an embodiment of the present application;

[0021] Figure 3 is a structural diagram of a data processing device according to an embodiment of the present application;

[0022] Figure 4 It is a hardware structure block diagram of a computer terminal according to a data processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to an embodiment of the present application, a method embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0027] Step S101, obtaining network traffic data generated by the business system of the financial institution within a preset time period.

[0028] For example, deploy network traffic monitoring equipment, such as deep packet inspection equipment, network intrusion detection system or traffic analysis software, at the network boundary of financial institutions or the network exit of business systems. Set collection strategies, including collection frequency (such as every minute, every hour) and preset duration (such as 24 hours, one week). Traffic monitoring equipment collects network traffic data according to these strategies and may store it in a local or remote database. The above equipment captures data packets in the network in real time or periodically, and records information such as the source and destination IP addresses, port numbers, protocol types and data packet sizes of the communication.

[0029] Step S102, determining the scale of the financial institution according to the number of backend servers of the business system, and determining target rules for identifying network traffic data according to the scale of the financial institution.

[0030] For example, financial institutions with less than 500 backend servers in their business systems are identified as small and medium-sized financial institutions.

[0031] Step S103: determining the first abnormal data in the network traffic data according to the target rule.

[0032] For example, for the "traffic surge rule", if the total traffic entering a key server of the banking system exceeds 80% of the normal traffic bandwidth of the server in a short period of time, it will be recorded as an abnormal feature "traffic surge, possibly a DDoS attack". For the "insider authority rule", if it is detected that an internal employee account has accessed a system module outside of its authority, and the number of accesses exceeds 3 times within 1 hour, then this is considered an abnormal feature "abnormal access by insiders".

[0033] Step S104: input the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data.

[0034] Specifically, first, the collected network traffic data is preprocessed, including data cleaning (removing invalid or erroneous data), feature extraction (such as extracting IP addresses, ports, protocol types, packet sizes, timestamps, etc. as model input features) and data conversion (such as encoding and converting non-numerical features to make them suitable for machine learning models). Secondly, the first abnormal data (i.e., abnormal data marked by expert rule models or other means) is used as part of the training set to train the machine learning model. The model type can be, but is not limited to, support vector machine (SVM), random forest (RF), neural network (such as deep learning model), etc., which aims to learn the difference between normal and abnormal traffic, so as to identify new abnormal patterns. After the model training is completed, the validation set is used to evaluate the performance of the model, such as accuracy, recall rate, F1 score and other indicators. According to the evaluation results, the model may need to be adjusted, such as parameter optimization, feature selection, model type selection, etc., to improve the detection accuracy and efficiency of the model. The preprocessed network traffic data is input into the trained machine learning model, and the model will output abnormal prediction results about these data. The network traffic data marked as abnormal in the prediction result is the second abnormal data. The second abnormal data is analyzed to confirm whether it truly represents abnormal behavior in the network. If it is confirmed to be abnormal, further measures can be taken, such as triggering an alarm, conducting detailed traffic analysis, implementing blocking strategies, etc., to protect the network from potential threats.

[0035] Step S105, publishing the first abnormal data and the second abnormal data.

[0036] According to the above steps, the network traffic data generated by the business system of the financial institution within a preset time period is obtained; the scale of the financial institution is determined according to the number of backend servers of the business system, and the target rule for identifying the network traffic data is determined according to the scale of the financial institution; the first abnormal data in the network traffic data is determined according to the target rule; the network traffic data and the first abnormal data are input into the machine learning model to obtain the second abnormal data in the network traffic data; the first abnormal data and the second abnormal data are published, so as to achieve the purpose of fully considering the business characteristics of small and medium-sized financial institutions, thereby realizing the technical effect of improving the accuracy of identifying abnormal network traffic data generated in the business systems of small and medium-sized financial institutions.

[0037] The following Figure 1 The steps shown are exemplary and explanatory.

[0038] According to some optional embodiments of the present application, determining the first abnormal data in the network traffic data according to the target rule can be achieved by the following method: if the number of times the abnormal Internet Protocol IP address connects to the preset port of the business system within a preset time period is greater than a first preset threshold, the network traffic data generated by the connection process is determined to be the first abnormal data, wherein the abnormal IP address is an IP address that is not in the preset IP address set.

[0039] For example, a trust list of IP addresses within the normal service area of ​​small and medium-sized banks is established, which may include, for example, the bank's customer base, partners, and IP ranges of known friendly service providers. If an abnormal Internet Protocol (IP) address outside the preset IP address set (i.e., the trust list) is detected, and the number of times it connects to the preset port of the banking system (such as online banking port 443) within a preset time period (such as 10 minutes) is greater than a first preset threshold (e.g., 100 times), the network traffic data generated by the connection process is determined to be the first abnormal data. The setting of this threshold should be based on statistical analysis of the bank's daily business traffic to ensure that normal customer access will not be misjudged as abnormal.

[0040] According to other optional embodiments of the present application, determining the first abnormal data in the network traffic data according to the target rule can be achieved by the following method: if the number of times the target employee account accesses the first business module in the business system that the target employee account has no access to within a preset time period is greater than a second preset threshold, the network traffic data generated by the access process is determined to be the first abnormal data.

[0041] For example, network traffic monitoring tools (such as firewalls, intrusion detection systems, etc.) are used to record the network access behavior of all employee accounts in real time, including the target system, module, time, frequency and other information accessed. When the monitoring system detects that the target employee account has accessed the first business module that it has no permission to access for more than the second preset threshold within a preset time period (such as 1 hour), the system will automatically trigger an alarm and mark the relevant network traffic data as the first abnormal data. The alarm information will be submitted to the security administrator or the operator of the operation center for manual review. The operator needs to determine whether this behavior does constitute a security threat based on the alarm information and the employee's operation records, and take corresponding control measures, such as restricting the access rights of the employee account, or further investigating the employee's behavioral intentions.

[0042] In some optional embodiments of the present application, determining the first abnormal data in the network traffic data according to the target rule can be achieved by the following method: if it is detected that the target data in the second business module in the business system whose importance level is greater than the preset level is transmitted to the external device of the business system, and the capacity of the target data is greater than the third preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data; if the number of data transmissions between the business system and the external payment system and / or the external credit system is greater than the fourth preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data.

[0043] Specifically, first, define the second business module in the bank information system as a module involving sensitive information and key business processes, such as a credit approval system, a customer account management system, etc. The target data in these modules usually include customer personal information, transaction records, loan approval details, etc., and have a high level of importance. Set a preset level to distinguish the importance of data; at the same time, set a third preset threshold to measure whether the capacity of data output exceeds the normal range. For example, the preset level can be defined as the "confidential" level, and the third preset threshold is set to 50MB, which means that any data marked as "confidential" level will be considered abnormal if it is transferred in excess of 50MB at one time. When it is detected that the target data in the second business module (with an importance level greater than the preset level) is transmitted to an external device of the business system, and the capacity of the target data is greater than the third preset threshold, the system will immediately record this behavior and mark it as the first abnormal data. This can prevent data leakage or unauthorized data transmission, especially when a large amount of sensitive information is transmitted.

[0044] Furthermore, a fourth preset threshold is set as the upper limit of the number of data transmissions between the business system and the external payment system and / or the external credit system within a day or an hour. For example, more than 50 data transmissions to these external systems within a day can be set as abnormal. If the number of data transmissions between the bank information system and the external payment system or the external credit system exceeds the fourth preset threshold, the system will automatically detect this abnormality and mark the network traffic data generated during the transmission process as the first abnormal data. This helps to identify potential system abuse or data security threats, especially when the frequency of data transmission between these external systems and the bank's internal system increases abnormally.

[0045] As some optional embodiments of the present application, before inputting the network traffic data and the first abnormal data into the machine learning model to obtain the second abnormal data in the network traffic data, the following steps can also be performed: when it is determined according to the target rule that the amount of first abnormal data in the network traffic data is greater than a preset threshold, the machine learning model is deployed.

[0046] Specifically, first, the network traffic data is analyzed according to the preset target rules to identify the first abnormal data. The target rules may include but are not limited to traffic surges, abnormal source IPs, abnormal access by internal personnel, etc. Secondly, the number of first abnormal data is counted. If this number exceeds the preset threshold, it indicates that there are enough abnormal samples that can be used to train and verify the machine learning model. The setting of the preset threshold should be based on the needs of model training. For example, for a binary classification problem, more than 500 black sample data are required to ensure the effectiveness of model training. Thirdly, once it is confirmed that the number of first abnormal data meets the threshold requirements, the deployment process of the machine learning model will be started. The deployment of the model may include steps such as data preprocessing, feature engineering, model training and tuning to ensure that the model can effectively identify abnormal patterns in network traffic. Finally, the model is initially deployed, the performance of the model is continuously monitored, feedback information on the model's predicted results and actual results is collected, and the model is periodically retrained and parameter adjusted to cope with changes in the network environment and emerging threats.

[0047] In some optional embodiments of the present application, after publishing the first abnormal data and the second abnormal data, the following steps may also be performed: obtaining an authenticity label of the second abnormal data, determining a target loss function based on the difference between the second abnormal data and the authenticity label, and iteratively training a machine learning model based on the target loss function.

[0048] When the expert rule model or the preliminary machine learning model marks certain data as abnormal (i.e., second abnormal data), manual review or higher-level analysis methods are required to confirm whether these data are truly abnormal. In this step, the operators or professional security analysts of the operation center will conduct in-depth analysis and manual confirmation of the second abnormal data based on the system logs, network monitoring data, and other business context information, and finally assign an authenticity label to each second abnormal data, such as "real abnormality" or "false positive".

[0049] In machine learning, the loss function is a metric used to quantify the difference between the model's predicted value and the actual value. In this embodiment, the design of the target loss function needs to take into account the difference between the predicted label of the second anomaly data and the manually confirmed authenticity label. If the data predicted by the model as abnormal is actually labeled as normal (false positive), or the abnormal data that the model failed to correctly identify is labeled as abnormal (false negative), these will increase the value of the loss function. The loss function may include binary cross entropy loss (for binary classification problems) or other loss functions suitable for anomaly detection scenarios to ensure that the model can effectively distinguish between abnormal and normal data.

[0050] By calculating the value of the loss function, the performance of the model in the current training round can be evaluated. In subsequent training rounds, the algorithm will adjust the parameters of the model to reduce the value of this loss function, with the goal of making the model more accurately predict the authenticity when processing the second abnormal data. This process usually involves a backpropagation algorithm, which updates the model parameters according to the gradient of the loss function and gradually optimizes the model's ability to identify abnormal data. The number of iterative training and the specific parameters such as learning rate and optimizer will have a direct impact on the final accuracy and efficiency of the model.

[0051] Through the above steps, we can continuously optimize the machine learning model by using limited abnormal data and expert judgment, so that it has higher accuracy when dealing with network security threats specific to small and medium-sized banks. This feedback-based model training strategy helps to improve the autonomous protection capabilities of small and medium-sized banks in network security.

[0052] As other optional embodiments of the present application, after publishing the first exception data and the second exception data, the following steps can also be performed: combining the first exception data and the second exception data with a preset sentence to obtain a target sentence, and calling the large language model through the target sentence; obtaining the output result generated by the large language model in response to the target sentence, wherein the output result at least includes: a processing plan for the exception data.

[0053] Specifically, the first abnormal data (e.g., traffic surge, suspected malicious scanning) and the second abnormal data (e.g., abnormal access by internal personnel, suspected data leakage) identified are first converted into target sentences described in natural language. The purpose of this step is to enable the large language model to understand the background and specific details of the abnormal situation, so as to give more accurate suggestions. Example of target sentence: "The traffic of the bank's online banking server has surged to 120% of its normal bandwidth in the past 5 minutes, and it is suspected to be under DDoS attack. Please provide a response strategy." Secondly, the target sentence is used as input to call the pre-trained large language model. This step utilizes the knowledge and experience of the large language model in the field of network security and can generate a processing solution based on the description of abnormal data. Target sentence input: "Internal employees frequently access customer account information during non-working hours, suspected of abnormal access by internal personnel. Please provide processing suggestions." Finally, the output result generated by the large language model based on the target sentence is obtained, and the output result at least includes a processing solution for the abnormal data. In this step, the output of the large language model may include detailed response strategies, possible attack type analysis, recommended next actions, and other relevant suggestions. For DDoS attacks, it is recommended to immediately start the traffic cleaning service and increase the server bandwidth to relieve pressure. For abnormal access by internal personnel, it is recommended to immediately freeze the employee account, conduct a permission audit, and trace back their access records to assess the risk of data leakage.

[0054] Figure 2 is a flow chart of another data processing method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0055] Step S201, expert rule model design: When designing expert rules, the business characteristics and security requirements of small and medium-sized banks should be fully considered to ensure the pertinence and effectiveness of the rules. For example, the rules can be based on multi-dimensional features such as time, traffic size, source IP, port, protocol type, etc. The triggering conditions of the expert rule model should be carefully designed and tested to ensure that abnormal traffic can be captured without generating too many false alarms that affect the normal operation of the system and user experience.

[0056] Step S202, construction of black sample database: In the initial stage, the black sample database can be constructed through the abnormal traffic captured by the expert rule model. At the same time, the sample library can be enriched by referring to the security reports, attack and defense cases in the industry, and the bank's own historical security events. The database should be updated regularly, removing samples that are no longer applicable and adding newly discovered abnormal features to adapt to the ever-changing network security threats.

[0057] Step S203, training and iteration of machine learning model: When the black sample database accumulates to a certain number, the machine learning model can be trained. The choice of model should be based on the data type and the complexity of the problem, such as SVM, random forest, neural network, etc. The training process requires meticulous data preprocessing, including feature engineering, data cleaning, missing value processing, etc., to improve the training effect of the model. After the model is launched, it should be integrated with the data of the expert rule model, and continuously iterated and optimized through feedback from actual operation to improve the accuracy and adaptability of the model.

[0058] Step S204, manual confirmation process of the operation center: The operators of the operation center should have certain security analysis capabilities and knowledge to effectively identify and confirm abnormal data. The manual confirmation process should include data review, analysis, recording and feedback mechanisms to ensure that each confirmation provides valuable information for model optimization.

[0059] Step S205: The confirmed abnormal data should be immediately recorded in the black sample database to form a closed-loop data return mechanism. The updated black sample database is used to retrain the model regularly to adapt to new security threats and dynamic changes in banking services.

[0060] Step S206, after manually checking and confirming the abnormal situation, the execution process at the system operation level can be designed into the following key steps to ensure the effective implementation of network security measures: 1. Instruction generation and issuance: After confirming the abnormality, the operator will generate corresponding operation instructions through the system interface. These instructions may include but are not limited to isolating the network, adjusting firewall rules, restricting IP access, restarting system services, etc. Among them, these instructions need to describe in detail the operation object (such as a specific server, network device or IP address), operation type (such as closing a port, updating an access control list) and operation parameters (such as operation time, specific content of the new rule). 2. The system should have the ability to respond automatically, that is, when it receives an operation instruction, it can automatically perform the corresponding security operation. 3. When the system performs any operation, a detailed log should be recorded, including information such as operation time, operator, operation instruction content, and operation results.

[0061] Figure 3 is a structural diagram of a data processing device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0062] The acquisition module 31 is used to acquire the network traffic data generated by the business system of the financial institution within a preset time period.

[0063] The first determination module 32 is used to determine the scale of the financial institution according to the number of backend servers of the business system, and determine the target rules for identifying network traffic data according to the scale of the financial institution.

[0064] The second determination module 33 is used to determine the first abnormal data in the network traffic data according to the target rule.

[0065] The third determination module 34 is used to input the network traffic data and the first abnormal data into the machine learning model to obtain the second abnormal data in the network traffic data.

[0066] The publishing module 35 is used to publish the first abnormal data and the second abnormal data.

[0067] Optionally, the second determination module 33 is also used to perform the following steps: if the number of times the abnormal Internet Protocol IP address connects to the preset port of the business system within a preset time period is greater than a first preset threshold, the network traffic data generated by the connection process is determined to be first abnormal data, wherein the abnormal IP address is an IP address that is not in the preset IP address set.

[0068] Optionally, the second determination module 33 is also used to execute the following steps: if the number of times the target employee account accesses the first business module in the business system that the target employee account has no access authority to within a preset time period is greater than a second preset threshold, determine that the network traffic data generated by the access process is the first abnormal data.

[0069] Optionally, the second determination module 33 is also used to perform the following steps: if it is detected that the target data in the second business module in the business system whose importance level is greater than the preset level is transmitted to the external device of the business system, and the capacity of the target data is greater than a third preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data; if the number of data transmissions between the business system and the external payment system and / or the external credit system is greater than a fourth preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data.

[0070] Optionally, the data processing device is also used to perform the following steps before inputting the network traffic data and the first abnormal data into the machine learning model to obtain the second abnormal data in the network traffic data: deploying the machine learning model when it is determined according to the target rule that the number of first abnormal data in the network traffic data is greater than a preset threshold.

[0071] Optionally, the data processing device is also used to perform the following steps after publishing the first abnormal data and the second abnormal data: obtain the authenticity label of the second abnormal data, determine the target loss function based on the difference between the second abnormal data and the authenticity label, and iteratively train the machine learning model based on the target loss function.

[0072] Optionally, the data processing device is also used to perform the following steps after publishing the first exception data and the second exception data: combining the first exception data and the second exception data with a preset sentence to obtain a target sentence, and calling the large language model through the target sentence; obtaining an output result generated by the large language model in response to the target sentence, wherein the output result at least includes: a processing plan for the exception data.

[0073] It should be noted that the above Figure 3 The modules in the embodiment may be program modules (e.g., a set of program instructions for implementing a specific function) or hardware modules. For the latter, they may be expressed in the following forms, but are not limited thereto: the expression form of the above modules is a processor, or the functions of the above modules are implemented by a processor.

[0074] It should be noted that Figure 3 The preferred implementation of the illustrated embodiment can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0075] Figure 4 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a data processing method. Figure 4 As shown, the computer terminal 40 may include one or more (402a, 402b, ..., 402n are used to illustrate) processors 402 (the processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 404 for storing data, and a transmission module 406 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0076] It should be noted that the one or more processors 402 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 40. As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0077] The memory 404 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present application. The processor 402 executes various functional applications and data processing by running the software programs and modules stored in the memory 404, that is, realizing the above-mentioned data processing method. The memory 404 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 404 may further include a memory remotely arranged relative to the processor 402, and these remote memories may be connected to the computer terminal 40 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0078] The transmission module 406 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 40. In one example, the transmission module 406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0079] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 40 .

[0080] It should be noted that, in some optional embodiments, the above Figure 4 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 4 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0081] It should be noted that Figure 4 The computer terminal shown is used to execute Figure 1 The data processing method shown, therefore the relevant explanations in the execution method of the above command are also applicable to the electronic device and will not be repeated here.

[0082] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data processing method.

[0083] A program for executing the following functions on a non-volatile storage medium: obtaining network traffic data generated by a business system of a financial institution within a preset time period; determining the scale of the financial institution based on the number of backend servers of the business system, and determining a target rule for identifying the network traffic data based on the scale of the financial institution; determining first abnormal data in the network traffic data based on the target rule; inputting the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data; and publishing the first abnormal data and the second abnormal data.

[0084] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if the number of times the abnormal Internet Protocol IP address connects to the preset port of the business system within a preset time period is greater than a first preset threshold, the network traffic data generated by the connection process is determined to be the first abnormal data, wherein the abnormal IP address is an IP address that is not in the preset IP address set.

[0085] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if the number of times the target employee account accesses a first business module in a business system that the target employee account has no authority to access within a preset time period is greater than a second preset threshold, determining that the network traffic data generated by the access process is the first abnormal data.

[0086] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if it is detected that the target data in the second business module in the business system whose importance level is greater than the preset level is transmitted to the external device of the business system, and the capacity of the target data is greater than a third preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data; if the number of data transmissions between the business system and the external payment system and / or the external credit system is greater than a fourth preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data.

[0087] Optionally, before inputting the network traffic data and the first abnormal data into the machine learning model to obtain the second abnormal data in the network traffic data, the method also includes: deploying the machine learning model when it is determined according to the target rule that the amount of first abnormal data in the network traffic data is greater than a preset threshold.

[0088] Optionally, after publishing the first abnormal data and the second abnormal data, the method further includes: obtaining an authenticity label of the second abnormal data, determining a target loss function based on a difference between the second abnormal data and the authenticity label, and iteratively training the machine learning model based on the target loss function.

[0089] Optionally, after publishing the first exception data and the second exception data, the method further includes: combining the first exception data and the second exception data with a preset sentence to obtain a target sentence, and calling the large language model through the target sentence; obtaining an output result generated by the large language model in response to the target sentence, wherein the output result at least includes: a processing solution for the exception data.

[0090] An embodiment of the present application further provides an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above data processing method is executed when the program is run.

[0091] The processor is used to run a program that performs the following functions: obtaining network traffic data generated by the business system of the financial institution within a preset time period; determining the scale of the financial institution based on the number of backend servers of the business system, and determining a target rule for identifying the network traffic data based on the scale of the financial institution; determining first abnormal data in the network traffic data based on the target rule; inputting the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data; and publishing the first abnormal data and the second abnormal data.

[0092] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if the number of times the abnormal Internet Protocol IP address connects to the preset port of the business system within a preset time period is greater than a first preset threshold, the network traffic data generated by the connection process is determined to be the first abnormal data, wherein the abnormal IP address is an IP address that is not in the preset IP address set.

[0093] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if the number of times the target employee account accesses a first business module in a business system that the target employee account has no authority to access within a preset time period is greater than a second preset threshold, determining that the network traffic data generated by the access process is the first abnormal data.

[0094] Optionally, according to the target rule, the first abnormal data in the network traffic data is determined, including: if it is detected that the target data in the second business module in the business system whose importance level is greater than the preset level is transmitted to the external device of the business system, and the capacity of the target data is greater than a third preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data; if the number of data transmissions between the business system and the external payment system and / or the external credit system is greater than a fourth preset threshold, the network traffic data generated by the transmission process is determined to be the first abnormal data.

[0095] Optionally, before inputting the network traffic data and the first abnormal data into the machine learning model to obtain the second abnormal data in the network traffic data, the method also includes: deploying the machine learning model when it is determined according to the target rule that the amount of first abnormal data in the network traffic data is greater than a preset threshold.

[0096] Optionally, after publishing the first abnormal data and the second abnormal data, the method further includes: obtaining an authenticity label of the second abnormal data, determining a target loss function based on a difference between the second abnormal data and the authenticity label, and iteratively training the machine learning model based on the target loss function.

[0097] Optionally, after publishing the first exception data and the second exception data, the method further includes: combining the first exception data and the second exception data with a preset sentence to obtain a target sentence, and calling the large language model through the target sentence; obtaining an output result generated by the large language model in response to the target sentence, wherein the output result at least includes: a processing solution for the exception data.

[0098] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0099] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0100] In the above-mentioned embodiments of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary protection measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0105] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Obtain network traffic data generated by the financial institution's business system within a preset period of time; Determine the scale of the financial institution based on the number of backend servers of the business system, and determine target rules for identifying the network traffic data based on the scale of the financial institution; Determining first abnormal data in the network traffic data according to the target rule; Inputting the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data; The first abnormal data and the second abnormal data are published.

2. The method according to claim 1, characterized in that Determining first abnormal data in the network traffic data according to the target rule includes: If the abnormal Internet Protocol IP address connects to the preset port of the business system for a number of times greater than a first preset threshold within a preset time period, the network traffic data generated by the connection process is determined to be the first abnormal data, wherein the abnormal IP address is an IP address that is not in the preset IP address set.

3. The method according to claim 1 or 2, characterized in that: Determining first abnormal data in the network traffic data according to the target rule includes: If the number of times that the target employee account accesses the first business module in the business system that the target employee account has no permission to access within a preset time period is greater than a second preset threshold, it is determined that the network traffic data generated during the access process is the first abnormal data.

4. The method according to claim 3, characterized in that Determining first abnormal data in the network traffic data according to the target rule includes: If it is detected that target data in a second business module in the business system whose importance level is greater than a preset level is transmitted to an external device of the business system, and the capacity of the target data is greater than a third preset threshold, it is determined that the network traffic data generated during the transmission process is the first abnormal data; If the number of data transmissions between the business system and the external payment system and / or the external credit system is greater than a fourth preset threshold, it is determined that the network traffic data generated during the transmission process is the first abnormal data.

5. The method according to claim 1, characterized in that Before inputting the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data, the method further includes: When it is determined according to the target rule that the amount of first abnormal data in the network traffic data is greater than a preset threshold, the machine learning model is deployed.

6. The method according to claim 1, characterized in that After publishing the first abnormal data and the second abnormal data, the method further includes: Obtain an authenticity label of the second abnormal data, determine a target loss function according to a difference between the second abnormal data and the authenticity label, and iteratively train the machine learning model based on the target loss function.

7. The method according to claim 1, characterized in that After publishing the first abnormal data and the second abnormal data, the method further includes: Combining the first abnormal data and the second abnormal data with a preset sentence to obtain a target sentence, and calling a large language model through the target sentence; An output result generated by the large language model in response to the target sentence is obtained, wherein the output result at least includes: a processing solution for abnormal data.

8. A data processing device, characterized in that: include: An acquisition module, used to acquire network traffic data generated by a financial institution's business system within a preset period of time; A first determination module is used to determine the scale of the financial institution according to the number of backend servers of the business system, and determine a target rule for identifying the network traffic data according to the scale of the financial institution; A second determination module, configured to determine first abnormal data in the network traffic data according to the target rule; A third determination module, configured to input the network traffic data and the first abnormal data into a machine learning model to obtain second abnormal data in the network traffic data; A publishing module is used to publish the first abnormal data and the second abnormal data.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the data processing method according to any one of claims 1 to 7 when running.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented.

Images