A data processing method for software development application based on large AI models

Abstract

The present invention relates to the field of software development application data processing, and discloses a software development application data processing method based on an AI large model, including: deploying an anomaly detection model at the data access layer to perform dynamic anomaly detection and real-time cleaning on software development application data, optimizing the scoring model based on historical security event feedback and classifying the software development application data; performing anti-correlation desensitization training on the software development application data, performing homomorphic encryption, regularly conducting association attack simulation tests and adjusting parameters; managing keys in a distributed manner and performing audit tracking based on the blockchain; accessing a regulation database based on a dynamic compliance engine, parsing regulations to generate a rule library, deploying checkpoints at key data links for real-time detection and blocking, and performing secure erasure verification; dynamically scheduling resources in combination with large model prediction, and monitoring costs in real time through a monitoring system; deploying nodes on multiple platforms, encrypting and aggregating parameter update models, and regularly scanning for vulnerabilities.

Description

Technical Field

[0001] The present invention relates to the technical field of software development application data processing, and specifically relates to a software development application data processing method based on an AI large model. Background Art

[0002] In the prior art, the verification and error handling mechanisms in the preprocessing stage of software development application data cannot cover complex data error types, such as logical errors or semantic errors. In heterogeneous data sources, data format differences lead to low preprocessing efficiency; if the data volume surges, the serial processing of cleaning, encryption, and desensitization becomes a performance bottleneck, lacking the support of a streaming processing framework; the classification standard depends on preset sensitivity tags and does not introduce a dynamic evaluation mechanism, resulting in outdated classification. If machine learning models are relied on for classification, the deviation or update delay of model training data may affect classification accuracy, and there is a lack of a processing strategy for software development application data between sensitive and non-sensitive; the mixed use of object storage, block storage, and file storage leads to data fragmentation, increasing the difficulty of data migration and unified management across storage types; the key lifecycle management is missing. If the master key is leaked, there is a risk of full-disk data exposure; the data passes through multiple intermediate nodes during transmission. If end-to-end encryption is not implemented, there is a risk of being intercepted by a man-in-the-middle; the failover mechanism in task scheduling is not mentioned, and node downtime causes task interruption;

[0003] In view of this, a software development application data processing method based on an AI large model is needed. Summary of the Invention

[0004] The purpose of the present invention is to provide a software development application data processing method based on an AI large model. To solve the above-mentioned prior art problems, the present invention is realized through the following technical solutions:

[0005] A software development application data processing method based on an AI large model provided by an embodiment of the present invention specifically includes the following steps:

[0006] Deploy an anomaly detection model at the data access layer to perform dynamic anomaly detection and real-time cleaning on software development application data, optimize the scoring model based on historical security event feedback, and classify the software development application data;

[0007] Conduct anti-correlation desensitization training on software development application data, perform homomorphic encryption, regularly conduct correlation attack simulation tests and adjust parameters; manage keys in a distributed manner, and perform audit tracking based on the blockchain;

[0008] Access the regulatory database based on a dynamic compliance engine, parse the regulations to generate a rule library, deploy checkpoints at key data links for real-time detection and blocking, and perform secure erasure verification; combine large models to predict and dynamically schedule resources, and monitor costs in real time through a monitoring system; deploy nodes on multiple platforms, encrypt and aggregate parameter update models, and scan for vulnerabilities regularly.

[0009] Furthermore, the specific method for deploying the anomaly detection model is as follows:

[0010] Regularly extract software development application data tables from the database, and filter software development application data for a specific time period or specific conditions;

[0011] Copy the software development application data to a data collection server;

[0012] Configure the data source and target location, and set the extraction time interval;

[0013] Based on the extraction interval, use a script to connect to the database and query the required software development application data;

[0014] Configure the Kafka producer, introduce the Kafka client library in the application code, and write the producer code;

[0015] Configure the Kafka consumer, consume data from the Kafka topic and transfer it to the data access layer; also write the consumer code using the Kafka client library;

[0016] Audit the historical software development application data extracted from the database;

[0017] Furthermore, the specific method for performing dynamic anomaly detection is as follows:

[0018] Utilize the semantic understanding ability of the anomaly detection model to construct a logical rule library for historical software development application data, and update the rule weights through a dynamic Bayesian network;

[0019] Extract historical software development application data for cleaning; select features related to anomaly detection according to business requirements and data characteristics;

[0020] Convert non-numeric features to numeric features; utilize the semantic understanding ability of the anomaly detection model, combine historical security events and business knowledge, and define logical rules;

[0021] According to the dependency relationship between the logical rule library and the features, define the structure of the dynamic Bayesian network; the nodes in the network represent features or rules, and the edges represent the dependency relationship between the nodes;

[0022] Adopt the maximum likelihood estimation and Bayesian estimation methods to calculate and analyze the conditional probability distribution of each node in the network;

[0023] For each rule in the logic rule base, count the number of times the rule is triggered in abnormal events within a preset sampling period to obtain the trigger frequency ;

[0024] Calculate the rule weight under high-frequency triggering through the formula where is the initial weight, is the learning rate, is the high-frequency trigger frequency value;

[0025] Calculate the rule weight under low-frequency triggering through the formula where is the low-frequency trigger frequency value;

[0026] Input the real-time software development application data into the logic rule base for rule matching; if the data meets the conditions of the preset rule, the preset rule is triggered;

[0027] Furthermore, the specific method for grading the software development application data is as follows:

[0028] Set indicators in combination with the scope and degree of influence of abnormal events on software development applications;

[0029] Count the number of times abnormal events occur within a certain period of time, and refer to the rule weights updated by the dynamic Bayesian network; when a rule with a high weight is triggered, the corresponding abnormal data has a higher risk level;

[0030] Furthermore, based on the Numpy library, dynamically adjust the privacy budget ε according to the field type;

[0031] Set ε to 0.1 for numerical fields, determine the sensitivity as half of the difference between the maximum and minimum values of the numerical fields, and generate Laplace noise based on the sensitivity and the privacy budget;

[0032] Control the noise range within ±100 through the truncation strategy, and correct the boundary of the noise value that exceeds the reasonable range;

[0033] Set ε to 0.2 for text fields, and use generalization processing to blur the address to the city level;

[0034] Build an encryption service based on the PyPaillier library to implement summation and counting operations in the ciphertext state;

[0035] Based on the public dataset, screen structured data similar to the desensitized fields, and reconstruct the original record through the blurred city and the noised numerical value to verify the attack success rate;

[0036] The ratio of the number of successfully associated records to the total number of records is processed to obtain the attack success rate, and the complement of the attack success rate with respect to 1 is obtained to get the association risk indicator R;

[0037] Adjust the sensitivity according to the dynamic range of the field. If the sensitivity increases, then reduce ε to maintain the noise intensity;

[0038] Adjust ε according to the uniqueness after semantic generalization. If the uniqueness is high, i.e., it is easy to associate, then reduce ε;

[0039] Furthermore, the specific method for the associated attack simulation test and parameter adjustment is as follows:

[0040] Split the master key into M parts, create a secure container in the cloud platform to store the key shards, encrypt each shard, and set up a multi-factor authentication mechanism to ensure access security;

[0041] The old key is automatically destroyed after passing through the maximum retention period, and the rotation and destruction situations are regularly checked to reduce the leakage risk;

[0042] Furthermore, the specific method for the distributed management of the key is as follows:

[0043] Build a consortium blockchain, configure network parameters and node information;

[0044] Encode the data operation log into JSON format, add operation time, operator, and data version information and calculate the hash value, and store it on the chain through the blockchain client to ensure data immutability and traceability;

[0045] When unauthorized access to first-level data is detected, an alarm is automatically triggered and the event is recorded;

[0046] Deploy the smart contract to the consortium blockchain, configure multi-role audit permissions, and ensure audit transparency and compliance through on-chain permission management;

[0047] Furthermore, the specific method for parsing regulations to generate a rule library is as follows:

[0048] Establish a secure connection with the global regulations database, and based on the sampling period, obtain the regulations clause data through the API interface and store it in the local data storage system;

[0049] Build a model running environment, configure model parameters, model parameters;

[0050] Preprocess the obtained regulations clause data, input the preprocessed regulations clause data into the model, and perform semantic understanding and key information extraction on the regulations clauses;

[0051] Convert the extracted key information into structured compliance rules, store them in the compliance rule library, and establish an index for quick query;

[0052] Furthermore, the specific method for real-time detection and blocking of the deployment checkpoint is as follows:

[0053] Intercept data operation requests based on the checkpoint program and extract request-related information;

[0054] Match the extracted request information with the rules in the compliance rule library in real time;

[0055] Determine the storage media that needs to have data deleted. For cloud storage objects, obtain the object list through the API provided by the cloud storage service; for local storage media, obtain the storage device information through file system operations;

[0056] Scan the storage media through specific tools or programs to check whether there are unoverwritten areas or residual original data;

[0057] If any abnormal situation is found, perform the overwrite operation again until the verification passes;

[0058] Furthermore, the specific method for dynamically scheduling resources by combining large model prediction is as follows:

[0059] Collect historical load data through monitoring tools, normalize the collected historical load data, and organize the processed data into a training dataset according to the time series;

[0060] During the training process, adjust the training parameters of the model, evaluate the prediction accuracy of the model through the cross-validation method, and use the trained model to predict the future load peak to generate prediction results;

[0061] Set the scaling policy of HPA to dynamically adjust the number of computing nodes, with the goal of increasing the resource utilization rate to the minimum utilization threshold;

[0062] Configure data collection tasks, collect resource usage and cost-related data, set up visualization charts, and display resource usage and cost distribution in real time;

[0063] According to the budget control target of the enterprise, set the budget threshold, configure the alarm rule based on the budget threshold, and when the budget is lower than the budget threshold, trigger the alarm information and send the alarm information to the preset receiving channels to notify relevant personnel.

[0064] Advantages of the present invention:

[0065] 1. Cover the entire process of data processing, from data access to final use. Each module collaborates closely to form a complete data management system, ensuring the security, compliance, and efficient processing of data at all links. Both the anomaly detection model and the resource scheduling system adopt machine learning algorithms, dynamically adjusting according to data changes and business requirements to improve the detection accuracy and resource utilization efficiency. The dynamic compliance engine tracks regulatory changes in real-time and automatically updates compliance rules to ensure that data processing always meets the latest regulatory requirements.

[0066] 2. Multiple encryption technologies and key management mechanisms safeguard data privacy and security, reducing the risk of data leakage. Blockchain audit tracking and secure erasure verification ensure that data operations are traceable and deletions are thorough, enhancing the reliability of data management. Resource elastic scheduling and cost monitoring mechanisms effectively reduce resource waste. Cross-cloud federated learning and multi-technology fusion solutions enable the system to adapt to different cloud environments and business requirements, featuring high flexibility and scalability. The dynamic compliance engine ensures that data processing complies with global regulatory requirements, reducing the compliance risk for enterprises. The blockchain audit tracking function leaves a complete record of data operations throughout the process, facilitating auditing and supervision and improving the transparency and compliance of enterprise data management. Brief Description of the Drawings

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0068] Figure 1 It is a flowchart of the steps of a data processing method for software development applications based on an AI large model provided in Embodiment 1 of the present invention. Detailed Embodiments

[0069] To enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0070] Embodiment 1

[0071] As Figure 1 shown, a data processing method for software development applications based on an AI large model provided in the embodiment of the present invention specifically includes the following steps:

[0072] Step 1: Deploy an anomaly detection model at the data access layer to perform dynamic anomaly detection and real-time cleaning on software development application data, optimize the scoring model based on historical security event feedback, and classify the software development application data;

[0073] In a specific embodiment, the specific method for deploying the anomaly detection model is as follows:

[0074] Write an SQL script to regularly extract software development application data tables from the MySQL database and filter software development application data for a specific time period or specific conditions;

[0075] It should be noted that an SQL script refers to a text file containing one or more Structured Query Language (SQL) statements, recording instructions for operating on a database;

[0076] Use the built-in replication function of the database to copy software development application data to the data collection server to reduce the performance impact on the production database;

[0077] Use an ETL tool to automate the data extraction process, configure the data source and target location, and set the extraction time interval;

[0078] It should be noted that an ETL tool refers to a software for data processing, and its core function is to execute the ETL process. ETL stands for Extract, Transform, and Load;

[0079] Based on the extraction interval, write a script using the MongoDB driver to connect to the MongoDB database and query the required software development application data;

[0080] It should be noted that MongoDB is an open-source, document-oriented NoSQL database management system, which is applied in the field of modern data storage and processing;

[0081] Configure a Kafka producer to send software development application data generated by the software development application to a Kafka topic;

[0082] It should be noted that Kafka is a distributed stream processing platform. The Kafka producer is responsible for publishing data to Kafka topics. Kafka producers are various application programs, including but not limited to: log collectors and message senders, which send software development application data to the Kafka cluster in the form of messages;

[0083] Introduce the Kafka client library in the application code and write producer code;

[0084] Configure a Kafka consumer to consume data from a Kafka topic and transmit it to the data access layer, and write consumer code using the Kafka client library;

[0085] It should be noted that a Kafka consumer refers to reading data from a Kafka topic, subscribing to one or more topics, and processing the received messages in a preset order;

[0086] Audit the historical software development application data extracted from the database;

[0087] Specifically, design an audit form to record the basic information of the data, whether it is abnormal, and the type of abnormality, make markings according to the form, and summarize the data after the markings are completed;

[0088] Formulate audit rules. For SQL injection attacks, if the input data contains sensitive keywords such as but not limited to SELECT and DROP, and the analysis and judgment do not conform to the normal business logic, it is determined as an SQL injection exception;

[0089] Use a programming language to write a rule matching script, traverse the historical message queue to uniformly access multi-source data, and establish an anomaly detection model;

[0090] In a specific embodiment, the specific method for performing dynamic anomaly detection is as follows:

[0091] Utilize the semantic understanding ability of the anomaly detection model to construct a logical rule library for historical software development application data, and update the rule weights through a dynamic Bayesian network;

[0092] Specifically, clean the historical software development application data extracted from MySQL and MongoDB data sources, remove duplicate data, and process missing values and outliers;

[0093] Select features related to anomaly detection according to business requirements and data characteristics;

[0094] Exemplarily, for network request data, select the request IP address, request time, request method, and request parameters as features;

[0095] Convert non-numeric features to numeric features; for example, use one-hot encoding to process categorical features;

[0096] Utilize the semantic understanding ability of the anomaly detection model, combine historical security events and business knowledge, and define logical rules;

[0097] Exemplarily, for login data, if the IP address attempts to log in unsuccessfully multiple times in a short period of time, it is determined as an abnormal login behavior; for database operation data, if a user deletes sensitive data during non-working hours, it is determined as an abnormal operation;

[0098] Based on subsequent management and updates, represent the defined logical rules in a preset form. For example, store the rules in a database and files using conditional expressions and decision trees.

[0099] Define the structure of the dynamic Bayesian network according to the dependency relationship between the logical rule library and the features; the nodes in the network represent features or rules, and the edges represent the dependency relationships between the nodes.

[0100] Use historical software development application data to learn the parameters of the dynamic Bayesian network.

[0101] Adopt the maximum likelihood estimation and Bayesian estimation methods to calculate and analyze the conditional probability distribution of each node in the network.

[0102] Monitor software development application data in real time. When an abnormal event is detected, record the relevant feature and rule information.

[0103] Update the weights of each rule in the logical rule library according to the inference results of the dynamic Bayesian network.

[0104] If the triggering frequency of any rule in an abnormal event is higher than the high-frequency triggering threshold, increase the weight of this rule; if the triggering frequency of any rule in an abnormal event is lower than the low-frequency triggering threshold, reduce its weight.

[0105] Specifically, during the dynamic anomaly detection process, record the triggering situation of each rule in each abnormal event.

[0106] Use a table to store the triggering records. The columns of the table include but are not limited to: the identifier of the abnormal event, the identifier of the rule, and whether the rule is triggered.

[0107] Set an initial weight for each rule in the logical rule library, and preset the initial weights of all rules to the same value; the initial weights are set according to business experience and historical data.

[0108] For each rule in the logical rule library, count the number of times the rule is triggered in abnormal events within a preset sampling period to obtain the triggering frequency ;

[0109] Through the formula Calculate the rule weight under high-frequency triggering, where is the initial weight, is the learning rate, and the learning rate responds to rule changes according to business experience and historical data to avoid false alarm interference, is the high-frequency triggering frequency value;

[0110] Through the formula Calculate the rule weight under low-frequency triggering, where, is the low-frequency trigger frequency value;

[0111] Input the real-time software development application data into the logic rule library for rule matching; if the data meets the conditions of the preset rules, the preset rules are triggered.

[0112] In a specific embodiment, the specific method for grading the software development application data is as follows:

[0113] Set indicators in combination with the scope and degree of influence of abnormal events on software development applications.

[0114] For example, if an anomaly causes the system to completely collapse and a large amount of data is lost, it is marked as a severe anomaly; if some functions have minor faults and some data shows anomalies, it is marked as a general anomaly; if the impact on the user experience is small, such as a slightly slower page loading speed, it is marked as a mild anomaly.

[0115] Count the number of abnormal events occurring within a certain period of time, and refer to the rule weights updated by the dynamic Bayesian network; when a rule with a high weight is triggered, the corresponding abnormal data has a higher risk level.

[0116] Step 2: Perform anti-correlation desensitization training on the software development application data based on the risk level, and perform homomorphic encryption. Regularly conduct association attack simulation tests and adjust parameters; manage the keys in a distributed manner, and conduct audit tracking based on the blockchain.

[0117] In a specific embodiment, the specific method for the anti-correlation desensitization training is as follows:

[0118] Based on the Numpy library, dynamically adjust the privacy budget ε according to the field type.

[0119] It should be noted that the Numpy library represents an open-source numerical computing library that provides high-performance multi-dimensional array objects and various functions for operating on arrays.

[0120] Set ε to 0.1 for numerical fields, and determine the sensitivity as half of the difference between the maximum and minimum values of the numerical fields. Generate Laplace noise based on the sensitivity and the privacy budget.

[0121] Control the noise range within ±100 through a truncation strategy, and perform boundary correction on noise values outside the reasonable range.

[0122] Set ε to 0.2 for text fields, and use generalization processing to blur the address to the city level.

[0123] Build an encryption service based on the PyPaillier library to implement summation and counting operations in the ciphertext state.

[0124] It should be noted that the PyPaillier library is a Python library for implementing the Paillier homomorphic encryption algorithm;

[0125] Select data from the public dataset every week for the associated attack simulation test;

[0126] Specifically, based on the structured data screened from the public dataset that is similar to the desensitized fields, reconstruct the original records through the blurred city and the noiseified numerical values to verify the attack success rate;

[0127] Process the ratio of the number of successfully associated records to the total number of records to obtain the attack success rate, and obtain the complementary number of the attack success rate with respect to 1 to get the associated risk index R. If R is greater than or equal to the associated risk threshold, it is regarded as successful association, and adjust the differential privacy and homomorphic encryption parameters and strategies according to the results;

[0128] Adjust the sensitivity according to the dynamic range of the field. If the sensitivity increases, reduce ε to maintain the noise intensity;

[0129] Adjust ε according to the uniqueness after semantic generalization. If the uniqueness is high, that is, it is easy to associate, then reduce ε;

[0130] In a specific embodiment, the specific method for the associated attack simulation test and parameter adjustment is as follows:

[0131] Based on the Intel SGX secure enclave technology, split the master key into M parts, create a secure container on the cloud platform to store the key shards, encrypt each shard with AES-256, and set up a multi-factor authentication mechanism to ensure access security;

[0132] Develop a key management service to automatically generate a new working key at 0:00 every day and distribute it through the TLS 1.3 secure channel;

[0133] The old key is automatically destroyed after passing through the maximum retention period, and the rotation and destruction situations are regularly checked to reduce the leakage risk;

[0134] In a specific embodiment, the specific method for the distributed management of keys is as follows:

[0135] Build a consortium chain based on Hyperledger Fabric and configure network parameters and node information;

[0136] It should be noted that Hyperledger Fabric is an open-source blockchain framework that provides highly customizable, flexible, and secure blockchain solutions;

[0137] Encode the data operation log into JSON format, add operation time, operator, and data version information and calculate the hash value, and store it on the chain through the blockchain client to ensure data immutability and traceability;

[0138] Write an intelligent contract in Go language to implement access control logic, automatically trigger an alarm and record events when unauthorized access to first-level data is detected;

[0139] Deploy the intelligent contract to the consortium blockchain, configure multi-role audit permissions, and ensure audit transparency and compliance through on-chain permission management;

[0140] Step 3: Connect to the regulatory database based on the dynamic compliance engine, parse the regulations to generate a rule library, deploy checkpoints at key data links for real-time detection and blocking, and perform security erasure verification; Combine the large model to predict and dynamically schedule resources, and monitor the cost in real time through the monitoring system; Deploy nodes on multiple platforms, encrypt and aggregate parameter update models, and scan for vulnerabilities regularly;

[0141] In a specific embodiment, the specific method for parsing regulations to generate a rule library is as follows:

[0142] Establish a secure connection with the global regulatory database, obtain regulatory clause data based on the sampling period through the API interface, and store it in the local data storage system;

[0143] Build a running environment for the legal NLP model based on RoBERTa, configure model parameters, and the model parameters include but are not limited to: word vector dimension and number of hidden layers;

[0144] It should be noted that RoBERTa represents a pre-trained language model in the field of natural language processing;

[0145] Preprocess the obtained regulatory clause data, including but not limited to: removing special characters and word segmentation operations, and input the preprocessed regulatory clause data into the model;

[0146] Use the NLP model to perform semantic understanding and key information extraction on the regulatory clauses;

[0147] Exemplarily, extract the conditions and requirements related to the data minimization principle from the General Data Protection Regulation (GDPR), and extract the processing process and conditions for user data deletion requests from the CCPA;

[0148] Convert the extracted key information into structured compliance rules, store them in the compliance rule library, and establish an index for quick query;

[0149] In a specific embodiment, the specific method for deploying checkpoints for real-time detection and blocking is: deploy a compliance checkpoint program at the interface layer of data transmission and in the read-write operation module of the storage system;

[0150] Intercept data operation requests based on the checkpoint program and extract request-related information, including but not limited to: data source, operation type, and data fields involved;

[0151] Match the extracted request information with the rules in the compliance rule library in real time;

[0152] Exemplarily, when there is a data transfer operation, check whether the transferred data contains EU user data. If it does, check whether it complies with the data minimization principle of GDPR; detect whether there are any violations through conditional judgment and logical reasoning;

[0153] If a violation is detected, immediately and automatically block the current data operation request. At the same time, generate a detailed compliance report according to the preset report template. The report content includes but not limited to: violation time, relevant regulatory provisions, and details of the violation operation, and send the compliance report to relevant compliance management personnel and technical personnel for handling;

[0154] Determine the storage media that needs to be data-deleted. For cloud storage objects, obtain the object list through the API provided by the cloud storage service; for local storage media, obtain the storage device information through file system operations;

[0155] Scan the storage media through specific tools or programs to check whether there are unoverwritten areas or residual original data;

[0156] If any abnormal situation is found, perform the rewrite operation again until the verification passes;

[0157] In a specific embodiment, the specific method for dynamically scheduling resources by combining large model prediction is as follows:

[0158] Collect historical load data through monitoring tools, including but not limited to: CPU usage rate, memory usage rate, and number of requests; perform normalization processing on the collected historical load data, and organize the processed data into a training data set according to the time series;

[0159] Build a Transformer prediction model using the Python deep learning framework, and input the preprocessed load data into the model for training;

[0160] During the training process, adjust the training parameters of the model, including the number of attention heads and layers, evaluate the prediction accuracy of the model through cross-validation methods, and use the trained model to predict future load peaks to generate prediction results;

[0161] Based on Kubernetes' HPA and Custom Metrics API, the prediction results of the Transformer model are used as the scaling metrics for HPA;

[0162] It should be noted that Kubernetes represents an open-source system for automating the deployment, scaling, and management of containerized applications. HPA is an important feature in Kubernetes, used to automatically adjust the number of Pod replicas according to the load of the application, thereby achieving horizontal scaling of the application; Custom Metrics API is an important extension mechanism in Kubernetes, used to allow users to use custom metrics to drive operations such as auto-scaling;

[0163] Set the scaling policy of HPA. If the predicted load peak is greater than the load threshold, the number of computing nodes is automatically increased; if the load is in the idle period, the resources are released to the shared pool; dynamically adjust the number of computing nodes, with the goal of increasing the resource utilization rate to the minimum utilization threshold;

[0164] Based on the Prometheus + Grafana monitoring system, configure data collection tasks in Prometheus to collect resource usage and cost-related data, create dashboards in Grafana, set visualization charts, and display resource usage and cost distribution in real time;

[0165] It should be noted that Prometheus is a monitoring solution widely used in the cloud native and DevOps fields, helping enterprises effectively monitor the running status, performance metrics, and resource usage of the system;

[0166] According to the budget control target of the enterprise, set the budget threshold, configure alarm rules in Prometheus based on the budget threshold. When the budget is lower than the budget threshold, trigger alarm information and send the alarm information to the preset receiving channels to notify relevant personnel;

[0167] In a specific embodiment, the specific method for updating the encrypted aggregation parameter model is as follows:

[0168] On each federated learning node, configure the TLS1.3 + national cryptography SM4 encryption protocol;

[0169] After the node trains the model locally, serialize the parameter tensor into a byte stream, generate ciphertext through SM4 encryption, encapsulate the ciphertext into a TLS record layer protocol data unit, and transmit it to the aggregation server through the TLS1.3 channel. The aggregation server decrypts the parameters of each node using the SM4 key, executes the aggregation algorithm, and generates global model parameters; the aggregated parameters are re-encrypted to all nodes to trigger local model updates;

[0170] For the TLS 1.3 protocol, configure the certificate and key related parameters to ensure identity authentication and data encryption for communication between nodes;

[0171] For the SM4 encryption algorithm, set the encryption key and the initial vector to encrypt the data transmitted between nodes;

[0172] Capture packet analysis of the TLS 1.3 handshake process to verify whether there is a downgrade attack or weak cipher suites;

[0173] If any exist, trigger a vulnerability warning, automatically pull the official patch and perform hot deployment. If hot repair is not possible, isolate the affected nodes and switch to the backup cluster.

[0174] The technical solution of the embodiment of the present invention is as follows: Combine SQL scripts, database replication functions, ETL tools, and Kafka message queues to achieve automated data extraction and real-time transmission of multi-source databases, reduce the impact on the production system, and ensure the integrity and real-time nature of data collection; Build a logical rule library based on historical data, use a dynamic Bayesian network to update the rule weights to achieve dynamic monitoring of data anomalies, and perform preprocessing through data cleaning and feature engineering to improve the accuracy of model detection; Adopt differential privacy technology and homomorphic encryption to protect data privacy and security, regularly conduct correlation attack simulation tests to ensure data desensitization effects; Implement sharded storage of keys, combine multi-factor authentication and regular rotation to reduce the risk of key leakage, store the data operation logs on the chain, and use smart contracts to achieve access control and audit tracking to ensure the traceability and security of data operations; Parse global regulations through a legal NLP model to generate a compliance rule library, deploy checkpoints at key data links to achieve real-time detection and blocking of violations, and ensure that data processing complies with regulatory requirements; Adopt overwriting deletion technology and a deep learning residual detection model to ensure thorough data deletion, and ensure data consistency and security through a cross-cloud replica synchronization deletion mechanism; Use the model to predict the load peak, achieve elastic resource scheduling, conduct cost monitoring and automatic optimization to improve resource utilization; Regularly perform vulnerability scans to ensure the security and reliability of the system.

[0175] The above has described in detail one embodiment of the present invention, but the content described is only the preferred embodiment of the present invention and cannot be considered as limiting the implementation scope of the present invention; The above formulas are all dimensionless and take their numerical calculations. The formulas are obtained by software simulation of a large amount of collected data to obtain a formula closest to the real situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation and historical experience and can be adjusted according to the actual situation; The above is only the preferred embodiment of the present invention and does not limit the present invention. All equivalent changes and improvements made according to the scope of the present invention application should still fall within the patent coverage scope of the present invention.

Claims

1. A data processing method for software development application based on AI large model, characterized in that, It includes the following steps: Deploy an anomaly detection model at the data access layer to perform dynamic anomaly detection and real-time cleaning on software development application data, optimize the scoring model based on historical security event feedback, and classify the software development application data; Conduct anti-correlation desensitization training on software development application data, perform homomorphic encryption, regularly conduct association attack simulation tests and adjust parameters; manage keys in a distributed manner, and conduct audit tracking based on the blockchain; The specific method of the anti-correlation desensitization training is as follows: Based on the Numpy library, dynamically adjust the privacy budget ε according to the field type; For numerical fields, set ε to 0.1, determine the sensitivity as half of the difference between the maximum and minimum values of the numerical fields, and generate Laplace noise based on the sensitivity and privacy budget; Control the noise range within ±100 through a truncation strategy, and correct the boundary of noise values that exceed the reasonable range; For text fields, set ε to 0.2, and use generalization processing to blur the address to the city level; Build an encryption service based on the PyPaillier library to implement summation and counting operations in the ciphertext state; Screen structured data similar to the desensitized fields from the public dataset, reconstruct the original record through the blurred city and noised values, and verify the attack success rate; Process the ratio of the number of successfully associated records to the total number of records to obtain the attack success rate, and obtain the association risk indicator R by getting the complement of the attack success rate with respect to 1; Adjust the sensitivity according to the dynamic range of the field. If the sensitivity increases, reduce ε to maintain the noise intensity; Adjust ε according to the uniqueness after semantic generalization. If the uniqueness is high, i.e., it is easy to associate, reduce ε; Based on the dynamic compliance engine, access the regulation database, parse the regulations to generate a rule library, deploy checkpoints at key data links for real-time detection and blocking, and conduct security erasure verification; combine the large model prediction to dynamically schedule resources, and monitor the cost in real time through the monitoring system; deploy nodes on multiple platforms, encrypt and aggregate parameter update models, and regularly scan for vulnerabilities.

2. The method for processing software development application data based on the AI large model according to claim 1, wherein The specific method of deploying the anomaly detection model is as follows: Regularly extract software development application data tables from the database, and screen software development application data for a specific time period or specific conditions; Copy the software development application data to the data collection server; Configure the data source and target location, and set the extraction time interval; Based on the extraction interval, use a script to connect to the database and query the required software development application data; Configure the Kafka producer, introduce the Kafka client library in the application code, and write the producer code; Configure the Kafka consumer, consume data from the Kafka topic and transfer it to the data access layer; also write the consumer code using the Kafka client library; Audit the historical software development application data extracted from the database.

3. A data processing method for software development applications based on an AI large model according to claim 1, characterized in that, The specific method of performing dynamic anomaly detection is as follows: Utilize the semantic understanding ability of the anomaly detection model to build a logical rule library for historical software development application data, and update the rule weights through a dynamic Bayesian network; Extract historical software development application data for cleaning; select features related to anomaly detection according to business requirements and data characteristics; Convert non-numerical features into numerical features; Utilize the semantic understanding ability of the anomaly detection model, combine historical security events and business knowledge, and define logical rules; Define the structure of the dynamic Bayesian network according to the logical rule library and the dependency relationships between features; the nodes in the network represent features or rules, and the edges represent the dependency relationships between nodes; Adopt the maximum likelihood estimation and Bayesian estimation methods to calculate and analyze the conditional probability distributions of each node in the network; For each rule in the logical rule library, count the number of times the rule is triggered in abnormal events within a preset sampling period to obtain the trigger frequency ; Through the formula the rule weight under high-frequency triggering is calculated, where is the initial weight, is the learning rate, is the high-frequency triggering frequency value; Through the formula the rule weight under low-frequency triggering is calculated, where is the low-frequency triggering frequency value; Input the real-time software development application data into the logical rule library for rule matching; if the data meets the conditions of the preset rules, the preset rules will be triggered.

4. A data processing method for software development applications based on an AI large model according to claim 1, characterized in that, The specific method for grading the software development application data is as follows: Set indicators by combining the scope and degree of the impact of anomaly events on software development applications; Count the number of anomaly events occurring within a certain period of time, and refer to the updated rule weights of the dynamic Bayesian network; when a rule with a high weight is triggered, the corresponding anomaly data has a higher risk level.

5. A data processing method for software development application based on AI large model according to claim 1, characterized in that, The specific method for the associated attack simulation test and parameter adjustment is as follows: Split the master key into M parts, create a secure container in the cloud platform to store the key shards, encrypt each shard, and set up a multi-factor authentication mechanism to ensure access security; The old key is automatically destroyed after passing through the maximum retention period, and the rotation and destruction situations are regularly checked to reduce the leakage risk.

6. A method for processing software development application data based on an AI large model according to claim 1, characterized in that, The specific method for the distributed management of keys is as follows: Build a consortium chain, configure network parameters and node information; Encode the data operation logs into the JSON format, add operation time, operator, and data version information and calculate the hash value, and store them on the chain through the blockchain client to ensure data immutability and traceability; Automatically trigger an alarm and record the event when unauthorized access to first-level data is detected; Deploy the smart contract to the consortium chain, configure multi-role audit permissions, and ensure audit transparency and compliance through on-chain permission management.

7. A data processing method for software development application based on AI large model according to claim 1, characterized in that, The specific method for parsing regulations to generate a rule library is as follows: Establish a secure connection with the global regulations database, obtain regulation clause data based on the sampling period through the API interface, and store it in the local data storage system; Build a model running environment, configure model parameters, model parameters; Preprocess the obtained regulation clause data, input the preprocessed regulation clause data into the model, and perform semantic understanding and key information extraction on the regulation clauses; Convert the extracted key information into structured compliance rules, store them in the compliance rule library, and establish an index for quick query.

8. A method for processing software development application data based on an AI large model according to claim 1, characterized in that, The specific method for the deployment of checkpoint real-time detection and blocking is as follows: Intercept data operation requests based on the checkpoint program and extract the request-related information; Perform real-time matching of the extracted request information with the rules in the compliance rule library; Determine the storage media that needs to perform data deletion. For cloud storage objects, obtain the object list through the API provided by the cloud storage service; for local storage media, obtain the storage device information through file system operations; Scan the storage media through specific tools or programs to check whether there are unoverwritten areas or residual original data; If any abnormal situation is found, perform the overwrite operation again until the verification passes.

9. A data processing method for software development application based on AI large model according to claim 1, characterized in that, The specific method for combining the large model prediction for dynamic resource scheduling is as follows: Collect historical load data through a monitoring tool, normalize the collected historical load data, and organize the processed data into a training dataset according to the time series; During the training process, adjust the training parameters of the model, evaluate the prediction accuracy of the model through the cross-validation method, and use the trained model to predict the future load peak to generate a prediction result; Set the scaling policy of HPA to dynamically adjust the number of computing nodes, with the goal of increasing the resource utilization rate to the minimum utilization threshold; Configure a data collection task to collect resource usage and cost-related data, and set up visual charts to display the resource usage and cost distribution in real time; According to the budget control target of the enterprise, set a budget threshold, configure an alarm rule based on the budget threshold, trigger an alarm message when the budget is lower than the budget threshold, and send the alarm message to a preset receiving channel to notify relevant personnel.

Images