A data operation acceleration method and system for cloud data architecture

Abstract

This invention discloses a data processing acceleration method and system for cloud data architecture, relating to the field of cloud data technology. This method and system for accelerating data processing in cloud data architecture assesses the threat risk index and vulnerability risk index of the cloud data receiving terminal based on data integrity and security, which can greatly improve data security. It calculates the target data transmission efficiency of the shared data transmission channel based on the target second data and the maximum aggregated data volume of each aggregation node in the shared data transmission channel, which can greatly improve the efficiency and stability of data transmission. Based on the gradient reduction formula, it uses layer-by-layer forward feedback to form a backpropagation mechanism, which can optimize parameters. After training through forward propagation and backpropagation, it can optimize cloud data and improve the accuracy of data processing.

Description

Technical Field

[0001] This invention relates to the field of cloud data technology, specifically to a method and system for accelerating data processing in a cloud data architecture. Background Technology

[0002] Cloud data refers to the storage of data across multiple computers using a network.

[0003] Chinese patent CN110825544A discloses a computing node and its failure detection method, as well as a cloud data processing system. The system primarily involves each computing node self-checking the operational status and resource usage of the services it provides, and reporting the results to a management node. The computing nodes dynamically adjust the interval between their next reports based on the results and inform the management node of this interval. The management node then reviews the reported results at the specified intervals to determine if a computing node has failed. While this patent solves the problem of cloud data detection, the following issues remain in practical operation:

[0004] 1. When collecting data from the cloud, the inability to effectively manage data flow leads to excessively long data retrieval times, resulting in reduced security and stability of cloud data.

[0005] 2. When processing data in the cloud, the calculations are not based on the parameters in the data, resulting in poor accuracy and the inability to accelerate the calculations later.

[0006] 3. It only optimizes and accelerates cloud data during the computation process, but does not effectively store the accelerated data. As a result, all the completed data is stored in one memory, leading to excessive storage capacity and slowing down the initial computation process. Summary of the Invention

[0007] The purpose of this invention is to provide a data processing acceleration method and system for cloud data architecture. It assesses the threat risk index and vulnerability risk index of the cloud data receiving terminal based on data integrity and security, which can greatly improve data security. It calculates the target data transmission efficiency of the shared data transmission channel based on the target second data and the maximum aggregated data volume of each aggregation node in the shared data transmission channel, which can greatly improve the efficiency and stability of data transmission. Based on the gradient reduction formula, it uses layer-by-layer forward feedback to form a backpropagation mechanism, which can optimize parameters. After training through forward and backward propagation, it can optimize cloud data, improving the accuracy of data processing. It determines the remaining available space capacity of the target cache space in real time and indicates it through a preset cursor, ensuring the caching effect of centralized data in the dataset and improving the security of SMS data information, thus solving problems in the prior art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for accelerating data processing in a cloud-based data architecture includes the following steps:

[0010] S1: Cloud Data Acquisition: Used to retrieve data from the database and then collect the corresponding data.

[0011] The data in the database is categorized and stored according to prefix codes, and mapped to a fixed-dimensional vector space.

[0012] S2: Data Acquisition Monitoring: Used to monitor the number and flow of data acquisition channels when collecting data based on data in the database;

[0013] S3: Cloud Data Processing: Used to train and process multiple layers of cloud data through neural network operations, and to build a dynamic neural network model based on the processing results, and to analyze the model parameters.

[0014] S4: Data integration: Based on the data results of neural network operations, determine the standard data format for the edge computing gateway to perform calculations on the data according to the calculation rules of the neural network operation gateway, and standardize the monitoring data, coordinate data and time data according to the standard data format to obtain standard monitoring data, standard coordinate data and standard time data;

[0015] S5: Operation Integration Data Cache Encryption: Used to extract key information fragments from the operation data, cache the extracted data according to the operation data type, and encrypt the cached operation data.

[0016] This invention also discloses a data processing acceleration system for cloud data architecture, wherein the data processing acceleration method for cloud data architecture is characterized in that: the data processing acceleration system for cloud data architecture includes:

[0017] The cloud data management and control unit is used to: obtain the network layer traffic changes of each cloud data receiving terminal during data transmission;

[0018] The importance of each network layer is assessed based on the traffic changes of the network layer of each data receiving terminal during data transmission.

[0019] Statistical analysis is performed on the target network layers whose importance is greater than or equal to a preset threshold for each cloud data receiving terminal.

[0020] Acquire historical successful transmission data for each cloud data receiving terminal, analyze the historical successful transmission data to determine its integrity and security, and assess the threat risk index and vulnerability risk index of the cloud data receiving terminal based on the integrity and security of the data;

[0021] The security index of each cloud data receiving terminal is calculated using a preset risk assessment system based on the workload of the target network layer of each cloud data receiving terminal and the threat risk index and vulnerability risk index of that cloud data receiving terminal.

[0022] Preferably, the cloud data management and control unit is further configured to:

[0023] Secure and risky cloud data receiving terminals are selected based on the security index of each cloud data receiving terminal.

[0024] Receive the first data sent by the secure cloud data receiving terminal;

[0025] Obtain the configuration and network information of the risk cloud data receiving terminal;

[0026] Construct a shared data transmission channel for risk cloud data receiving terminals based on network information;

[0027] Obtain multiple aggregation nodes in the shared data transmission channel and determine the maximum amount of data that can be aggregated at a single time for each aggregation node;

[0028] Identify the target second data with the largest data volume in the second data received by the risk cloud data receiving terminal;

[0029] The target data transmission efficiency of the shared data transmission channel is calculated based on the target second data and the maximum aggregated data volume of each word at each aggregation node in the shared data transmission channel:

[0030] Set the data transmission efficiency of the shared data transmission channel to the target data transmission efficiency. After setting, use the shared data transmission channel to receive the second data sent by the risk cloud data receiving terminal.

[0031] Preferred options also include:

[0032] The cloud data processing unit is used to perform calculations on cloud data based on neural network operations, select results based on the calculation results of cloud data, and view the selected results according to the data grouping list after obtaining the selected results;

[0033] The cloud-based data processing unit includes:

[0034] The computation settings module is used to provide parameters for cloud data through neural network computation after acquiring cloud data. If the computation fails, the neural network parameters need to be adjusted, generally only the number of nodes and the number of hidden layers are adjusted.

[0035] The computational data output module is used to set the scheme decision for neural network computation before acquiring data from the cloud;

[0036] The computational data export module is used to export the scheme sample data and the corresponding neural network computation data, and then compare the two.

[0037] Preferably, the computational process of the neural network operation includes:

[0038] First, propagate the parameter data forward;

[0039] In this process, parameter data is propagated from lower to higher levels, and reverse propagation is performed when the propagated data does not match the expected results.

[0040] Backpropagation involves propagating errors from higher levels to lower levels during training.

[0041] The training process is as follows: First, the weights of the parameters are initialized. After the settings are completed, the parameter data is forward propagated through the convolutional layer, the downsampling layer, and the fully connected layer to obtain the output value. When the error is greater than the expected value, the error is propagated back into the network, and the errors of the fully connected layer, the downsampling layer, and the convolutional layer are calculated in sequence. The error of each layer is the total error of the network. When the error is equal to or less than the expected value, the training is complete.

[0042] Preferred options also include:

[0043] The computational data integration unit is used to acquire monitoring data, coordinate data, and time data from the computational data, and to standardize the monitoring data, coordinate data, and time data to obtain standard monitoring data, standard coordinate data, and standard time data.

[0044] This involves extracting important fields from standard monitoring data, performing data correlation analysis on these important data fields, identifying target important data fields with anomalies, and then extracting and removing the abnormal monitoring data corresponding to these target important data fields from the standard monitoring data.

[0045] The coordinate features of the standard coordinate data and the time features of the time data are obtained. Based on the coordinate features and time features, the coordinate patterns and time patterns of the standard coordinate data and standard time data are determined respectively. Abnormal coordinate data that does not meet the coordinate patterns are extracted from the standard coordinate data and removed. Abnormal time data that does not meet the book pattern are extracted from the standard time data and removed.

[0046] The monitoring data is labeled with distribution maps using different data integration rules. Based on these different rules, dynamic data integration instructions are established. These instructions dynamically integrate the confirmed standard monitoring data, standard coordinate data, and standard time data to obtain multiple sets of integrated data. A dataset is then generated based on these multiple sets of integrated data.

[0047] Preferred options also include:

[0048] The computational data caching and encryption unit is used to store the generated dataset separately, and then cache and encrypt the data according to the data type.

[0049] Preferably, the computational data caching encryption unit includes:

[0050] The dataset retrieval module is used to obtain data information from independently stored computation datasets, extract key information fragments from the data in the dataset, and determine the data type of the data information in the dataset based on the data code prefix of the key information fragments.

[0051] The space determination module is used to determine the target cache space corresponding to the data in the dataset based on the data type, extract the capacity information of the target cache space, and determine the first remaining available space capacity of the target cache space based on the capacity information.

[0052] The space determination module is used to obtain the data length of the data in the dataset, and when the first remaining available space capacity is greater than the data length, to cluster the data in the dataset to obtain a set of sub-data types corresponding to the data in the dataset, and to set a type identifier for each sub-data type.

[0053] The space partitioning module is used to divide the target cache space into first blocks based on the type identifier, add block identifiers to the partitioned sub-target cache spaces, and divide each sub-target cache space into second blocks to obtain the first storage entry and the second storage entry corresponding to each sub-target cache space, wherein the block identifier corresponds to the type identifier.

[0054] The data caching module is used to extract the target content of the central data of each sub-data type based on the type identifier, and cache the type identifier and the target content to the first storage entry and the second storage entry respectively.

[0055] The data update module is used to monitor the reading operations of the central data in the dataset in real time. When reading of the data is detected, it determines the real-time reading amount of the data based on the reading process and determines the data tail of the remaining central data in the target cache space based on the reading amount.

[0056] The space update module is used to move the preset cursor to the end of the data in the dataset in the target cache space, determine the second remaining available space capacity of the target cache space based on the moving result, and indicate the second remaining available space capacity of the target cache space based on the preset cursor.

[0057] The space capacity encryption module is used to divide the computational data in the cached datasets in the first and second available space capacities into multiple data nodes.

[0058] The dataset is encrypted based on multiple partitioned data nodes.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] 1. This invention provides a data processing acceleration method and system for cloud data architecture. Before data collection, the data in the database is first detected, and the prefix codes of the data in the database are tagged and stored, mapped to a fixed-dimensional vector space. Through efficient vector calculation, accurate and rapid classification and tagged storage of large amounts of data are completed. Data location is quickly located based on tags, reducing system retrieval time. Historical successful transmission data of each cloud data receiving terminal is obtained, and its integrity and security are determined by analyzing the historical successful transmission data. Based on the integrity and security of the data, the threat risk index and vulnerability risk index of the cloud data receiving terminal are evaluated, which can greatly improve data security. The target data transmission efficiency of the shared data transmission channel is calculated based on the target second data and the maximum aggregated data volume of each aggregation node in the shared data transmission channel, which can greatly improve the efficiency and stability of data transmission.

[0061] 2. This invention provides a data processing acceleration method and system for cloud data architecture. The method involves setting up a neural network through a processing configuration module. After the parameter data is set, the parameter weights are initialized first. This method can improve the accuracy of the computation results. Then, the parameter data is forward-propagated through convolutional layers, downsampling layers, and fully connected layers to obtain the output value. Forward propagation involves moving the parameter data from lower to higher levels. When the propagated data result does not match the expectation, backward propagation is performed. Backward propagation is used to correct errors. The training propagates from high-level to low-level layers. After training, when the error is greater than the expected value, the error is propagated back into the network to calculate the errors of the fully connected layer, downsampling layer, and convolutional layer in sequence. When the error is equal to or less than the expected value, the training is complete. When the parameter data is trained through forward propagation, it passes through each hidden layer, and the final loss data is obtained when it passes through the hidden layer. When the parameter data is trained through backward propagation, it is fed forward layer by layer according to the gradient decrease formula to form a backward propagation mechanism, which can optimize the parameters. After training through forward propagation and backward propagation, the cloud data can be optimized to improve the accuracy of data calculation.

[0062] 3. This invention provides a data processing acceleration method and system for cloud data architecture. By analyzing the data information of the processing dataset, it accurately and effectively confirms the data information types of the dataset, thus facilitating the determination of the target cache space for caching the data information. Secondly, by clustering the data information of the processing dataset and classifying the different data types contained in the dataset based on the clustering results, it divides the target cache space according to the classification results, facilitating the storage of different types of data in corresponding storage areas. Simultaneously, each sub-target storage space is further divided to ensure the caching effect and accuracy of each type of data content and type identifier. Finally, different types of data are cached in their corresponding sub-target cache spaces, and the system monitors the read operations of the processing data in the dataset in real time. After a read operation occurs, the remaining available space capacity of the target cache space is determined in real time and indicated by a preset cursor, ensuring the caching effect of the data in the dataset and improving the security of SMS data information. Furthermore, multiple space capacities allow for larger data storage capacity, reflecting in the early processing steps and making the processing faster. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the data processing acceleration method for the cloud data architecture of the present invention;

[0064] Figure 2This is a schematic diagram of the data processing acceleration system module of the cloud data architecture of the present invention;

[0065] Figure 3 This is a schematic diagram of the cloud data processing unit module of the present invention;

[0066] Figure 4 This is a schematic diagram of the computational data caching and encryption unit module of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] To address the issue in existing technologies where data traffic cannot be effectively controlled during cloud data collection, resulting in excessively long data retrieval times and reduced cloud data security and stability, please refer to [link to relevant documentation]. Figure 1 and Figure 2 This embodiment provides the following technical solution:

[0069] A method for accelerating data processing in a cloud-based data architecture includes the following steps:

[0070] S1: Cloud Data Acquisition: Used to retrieve data from the database and then collect the corresponding data.

[0071] The data in the database is categorized and stored according to prefix codes, and mapped to a fixed-dimensional vector space.

[0072] S2: Data Acquisition Monitoring: Used to monitor the number and flow of data acquisition channels when collecting data based on data in the database;

[0073] S3: Cloud Data Processing: Used to train and process multiple layers of cloud data through neural network operations, and to build a dynamic neural network model based on the processing results, and to analyze the model parameters.

[0074] S4: Data integration: Based on the data results of neural network operations, determine the standard data format for the edge computing gateway to perform calculations on the data according to the calculation rules of the neural network operation gateway, and standardize the monitoring data, coordinate data and time data according to the standard data format to obtain standard monitoring data, standard coordinate data and standard time data;

[0075] S5: Operation Integration Data Cache Encryption: Used to extract key information fragments from the operation data, cache the extracted data according to the operation data type, and encrypt the cached operation data.

[0076] A data processing acceleration system for cloud data architecture includes:

[0077] The cloud data management unit is used for: acquiring the traffic changes of the network layer of each cloud data receiving terminal during data transmission; assessing the importance of each network layer based on the traffic changes of the network layer of each data receiving terminal during data transmission; statistically analyzing the target network layers of each cloud data receiving terminal whose importance is greater than or equal to a preset threshold; acquiring historical successful transmission data of each cloud data receiving terminal, parsing the historical successful transmission data to determine its integrity and security, and assessing the threat risk index and vulnerability risk index of the cloud data receiving terminal based on the integrity and security of the data; calculating the security index of the cloud data receiving terminal using a preset risk assessment system based on the workload of the target network layer of each cloud data receiving terminal and the threat risk index and vulnerability risk index of the cloud data receiving terminal. The cloud data management unit is also used for: acquiring the traffic changes of the network layer of each cloud data receiving terminal during data transmission; assessing the importance of each network layer of each cloud data receiving terminal based on the traffic changes of the network layer during data transmission; statistically analyzing the target network layers of each cloud data receiving terminal during data transmission; acquiring historical successful transmission data of each cloud data receiving terminal, parsing the historical successful transmission data to determine its integrity and security, and assessing the threat risk index and vulnerability risk index of the cloud data receiving terminal based on the integrity and security of the data; and statistically analyzing the target network layers of each cloud data receiving terminal during data transmission, using a preset risk assessment system. The security index of the receiving terminal filters out secure cloud data receiving terminals and risky cloud data receiving terminals; it receives the first data sent by the secure cloud data receiving terminal; it obtains the configuration information and network information of the risky cloud data receiving terminal; it constructs a shared data transmission channel for the risky cloud data receiving terminal based on the network information; it obtains multiple aggregation nodes in the shared data transmission channel and determines the maximum single aggregation data volume of each aggregation node; it determines the target second data with the largest data volume in the second data of the risky cloud data receiving terminal; it calculates the target data transmission efficiency of the shared data transmission channel based on the target second data and the maximum single aggregation data volume of each aggregation node in the shared data transmission channel; it sets the data transmission efficiency of the shared data transmission channel to the target data transmission efficiency; after setting, it uses the shared data transmission channel to receive the second data sent by the risky cloud data receiving terminal.

[0078] Specifically, before data collection, the data in the database is first inspected, and the prefix codes of the data in the database are tagged and stored, mapped to a fixed-dimensional vector space. Through efficient vector computation, accurate and rapid classification and tagged storage of large amounts of data are completed. Data location is quickly located based on tags, reducing system retrieval time. After retrieval, the data is collected. During collection, the traffic of the data collection channel is controlled by the cloud data management unit. Historical successful transmission data of each cloud data receiving terminal is obtained, and its integrity and security are determined by analyzing the historical successful transmission data. Based on the integrity and security of the data, the threat risk index and vulnerability risk index of the cloud data receiving terminal are assessed, which can greatly improve data security. At the same time, multiple aggregation nodes in the shared data transmission channel are obtained, the maximum single aggregation data volume of each aggregation node is determined, and the target second data with the largest data volume in the second data of the risky cloud data receiving terminal is determined. Based on the target second data and the maximum aggregation data volume of each aggregation node in the shared data transmission channel, the target data transmission efficiency of the shared data transmission channel is calculated, which can greatly improve the efficiency and stability of data transmission.

[0079] To address the issue in existing technologies where calculations on cloud data are not based on parameters within the data itself, resulting in poor accuracy and hindering subsequent acceleration of computations, please refer to [link to relevant documentation]. Figure 3 This embodiment provides the following technical solution:

[0080] It also includes: a cloud data processing unit, used to perform calculations on cloud data based on neural network operations, select results based on the calculation results of the cloud data, and view the selected results according to the data grouping list; wherein, the cloud data processing unit includes: a calculation setting module, used to provide parameters for cloud data through neural network operations after acquiring cloud data, wherein if the calculation fails, the neural network parameters need to be adjusted, generally only the number of nodes and the number of hidden layers are adjusted; a calculation data output module, used to set the scheme decision for neural network operation before acquiring cloud data; and a calculation data export module, used to export the scheme sample data and the corresponding neural network operation data, and compare the two after export.

[0081] The computational flow of the neural network includes: first, forward propagation of parameter data; wherein, the parameter data is propagated from lower levels to higher levels, and backpropagation is performed when the propagated data result does not match the expectation; wherein, backpropagation propagates the error from higher levels to lower levels for training; wherein, the propagation training process is as follows: first, the weights of the parameters are initialized; after the settings are completed, the parameter data is forward propagated through convolutional layers, downsampling layers, and fully connected layers to obtain the output value; when the error is greater than the expected value, the error is propagated back into the network, and the errors of the fully connected layers, downsampling layers, and convolutional layers are calculated in sequence; wherein, the error of each layer is the total error of the network; when the error is equal to or less than the expected value, the training is complete.

[0082] Specifically, the collected cloud data is first used to configure the neural network through the computation setting module. After the parameter data is set, the weights of the parameters are initialized. This method can improve the accuracy of the computation results. Then, the parameter data is forward propagated through convolutional layers, downsampling layers, and fully connected layers to obtain the output value. Forward propagation propagates the parameter data from low to high levels. When the data result obtained by propagation does not match the expectation, backpropagation is performed. Backpropagation propagates the error from high to low levels for training. After training, when the error is greater than the expected value, the error is propagated back into the network to calculate the error of the fully connected layer, downsampling layer, and convolutional layer in sequence. When the error is equal to or less than the expected value, the training is complete. When the parameter data is trained through forward propagation, it passes through each hidden layer, and the final loss data is obtained when passing through the hidden layer. When the parameter data is backpropagated, according to the gradient decrease formula, it is fed forward layer by layer to form a backpropagation mechanism, which can optimize the parameters. After training through forward propagation and backpropagation, cloud data can be optimized to improve the accuracy of data computation.

[0083] To address the issue in existing technologies where the diversity of cloud data causes computational acceleration to be followed by slower computation processes, please refer to [link to relevant documentation]. Figure 2 This embodiment provides the following technical solution:

[0084] It also includes: a computational data integration unit, used to acquire monitoring data, coordinate data, and time data from the computational data, and to standardize the monitoring data, coordinate data, and time data, resulting in standard monitoring data, standard coordinate data, and standard time data; it extracts important fields from the standard monitoring data, performs data correlation analysis on these important data fields to identify target important data fields with anomalies, and removes the abnormal monitoring data corresponding to these target important data fields from the standard monitoring data; it acquires the coordinate features of the standard coordinate data and the time features of the time data, and determines the coordinate and time patterns of the standard coordinate and time data based on these features, removing abnormal coordinate data that does not meet the coordinate patterns from the standard coordinate data, and removing abnormal time data that does not meet the time patterns from the standard time data; it marks the monitoring data distribution map with multiple different data integration rules, and establishes dynamic data integration instructions based on these different rules. These dynamic data integration instructions dynamically integrate the confirmed standard monitoring data, standard coordinate data, and standard time data to obtain multiple sets of integrated data, and generate a dataset based on these multiple sets of integrated data.

[0085] Specifically, the system acquires monitoring data, coordinate data, and time data from the computational data, and standardizes these data. After standardization, it generates standard monitoring data, standard coordinate data, and standard time data. Abnormal data is then removed based on these standard monitoring data, standard coordinate data, and standard time data. A dynamic data integration instruction is established to dynamically integrate the confirmed standard monitoring data, standard coordinate data, and standard time data, resulting in multiple sets of integrated data. These multiple sets of integrated data are then used to generate a dataset, thus integrating diverse computational data into a dataset, which is more convenient for later caching.

[0086] To address the issue in existing technologies that merely optimize and accelerate cloud data during computation without effectively storing the accelerated data—resulting in excessive storage capacity and slowing down initial computation processes—please refer to [the relevant documentation / reference]. Figure 4 This embodiment provides the following technical solution:

[0087] It also includes: a computational data caching encryption unit, used to store the generated dataset separately, cache and encrypt the data according to its type after storage. The computational data caching encryption unit includes: a dataset retrieval module, used to retrieve data information from the independently stored computational dataset, extract key information fragments from the data in the dataset, and determine the data type of the data information in the dataset based on the data code prefix of the key information fragments; a space determination module, used to determine the target cache space corresponding to the data in the dataset based on the data type, extract the capacity information of the target cache space, and determine the first remaining available space capacity of the target cache space based on the capacity information; the space determination module is used to obtain the data length of the data in the dataset, and when the first remaining available space capacity is greater than the data length, cluster the data in the dataset to obtain a set of sub-data types corresponding to the data in the dataset, and set a type identifier for each sub-data type; a space partitioning module, used to partition the target cache space into first blocks based on the type identifiers, add block identifiers to the partitioned sub-target cache spaces, and simultaneously partition each sub-target cache space into a second block. The system is divided into two parts: a first storage entry and a second storage entry corresponding to each sub-target cache space, where the block identifier corresponds to the type identifier; a data caching module is used to extract the target content of the central data corresponding to each sub-data type based on the type identifier, and cache the type identifier and the target content to the first storage entry and the second storage entry respectively; a data update module is used to monitor the reading operation of the central data in real time, and when reading of the central data is detected, it determines the real-time reading volume of the central data based on the reading process, and determines the data tail of the remaining central data in the target cache space based on the reading volume; a space update module is used to move the preset cursor to the tail of the central data in the target cache space, and determine the second remaining available space capacity of the target cache space based on the moving result, and indicate the second remaining available space capacity of the target cache space based on the preset cursor; a space capacity encryption module is used to divide the central data in the cached central data in the first available space capacity and the second available space capacity into multiple data nodes; and encrypt the central data based on the multiple data nodes.

[0088] Specifically, by analyzing the data information of the computational dataset, the data types of the dataset can be accurately and effectively identified, facilitating the determination of the target cache space for caching the data information of the computational dataset. Secondly, by clustering the data information of the computational dataset, the different data types contained in the data information of the computational dataset are classified according to the clustering results, and the target cache space is divided according to the classification results, so that different types of data can be stored in the corresponding storage areas. At the same time, each sub-target storage space is further divided to ensure the caching effect and caching accuracy of each type of data content and type identifier. Finally, different types of data are cached in the corresponding sub-target cache space, and the read operations of the computational data in the dataset are monitored in real time. After a read operation occurs, the remaining available space capacity of the target cache space is determined in real time and indicated by a preset cursor, which ensures the caching effect of the data in the dataset and improves the security of SMS data information. At the same time, multiple space capacities can allow for larger data storage capacity, reflecting in the early computation steps, making the computation process faster.

[0089] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0090] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A data operation acceleration method for a cloud data architecture, characterized in that: Includes the following steps: S1: Cloud Data Acquisition: Used to retrieve data from the database and then collect the corresponding data. The data in the database is categorized and stored according to prefix codes, and mapped to a fixed-dimensional vector space. S2: Data Acquisition Monitoring: Used to monitor the number and flow of data acquisition channels when collecting data based on data from the database; S3: Cloud Data Processing: Used to train and process multiple layers of cloud data through neural network operations, and to build a dynamic neural network model based on the results, and analyze the model parameters; S4: Data Integration: Based on the data results of neural network operations, this function determines the standard data format for edge computing gateways to perform calculations according to the calculation rules of the neural network computing gateway. Then, it standardizes the monitoring data, coordinate data, and time data according to the standard data format to obtain standard monitoring data, standard coordinate data, and standard time data. S5: Operation Integration Data Cache Encryption: Used to extract operation data based on key information fragments in the operation data, cache the data according to the operation data type, and encrypt the cached operation data. The cloud-based data architecture's data processing acceleration system includes: The cloud data management and control unit is used to: obtain the network layer traffic changes of each cloud data receiving terminal during data transmission; The importance of each network layer is assessed based on the traffic changes of the network layer of each data receiving terminal during data transmission. Statistical analysis is performed on the target network layers whose importance is greater than or equal to a preset threshold for each cloud data receiving terminal. Acquire historical successful transmission data for each cloud data receiving terminal, analyze the historical successful transmission data to determine its integrity and security, and assess the threat risk index and vulnerability risk index of the cloud data receiving terminal based on the integrity and security of the data; The security index of each cloud data receiving terminal is calculated using a preset risk assessment system based on the working intensity of the target network layer of each cloud data receiving terminal and the threat risk index and vulnerability risk index of that cloud data receiving terminal. The cloud data processing unit is used to perform calculations on cloud data based on neural network operations, select results based on the calculation results of cloud data, and view the selected results according to the data grouping list after obtaining the selected results; The computational process of the neural network includes: First, perform forward propagation of the parameter data; In this process, parameter data is propagated from lower to higher levels, and reverse propagation is performed when the propagated data does not match the expected results. Backpropagation involves propagating errors from higher levels to lower levels during training. The training process is as follows: First, the weights of the parameters are initialized. After the settings are completed, the parameter data is forward propagated through the convolutional layer, the downsampling layer, and the fully connected layer to obtain the output value. When the error is greater than the expected value, the error is propagated back into the network, and the errors of the fully connected layer, the downsampling layer, and the convolutional layer are calculated in sequence. The error of each layer is the total error of the network. When the error is equal to or less than the expected value, the training is complete.

2. The data processing acceleration method for cloud data architecture according to claim 1, characterized in that: The cloud-based data management and control unit is also used for: Secure and risky cloud data receiving terminals are selected based on the security index of each cloud data receiving terminal. Receive the first data sent by the secure cloud data receiving terminal; Obtain the configuration and network information of the risk cloud data receiving terminal; Construct a shared data transmission channel for risk cloud data receiving terminals based on network information; Obtain multiple aggregation nodes in the shared data transmission channel and determine the maximum amount of data that can be aggregated at a single time for each aggregation node; Identify the target second data with the largest data volume in the second data received by the risk cloud data receiving terminal; The target data transmission efficiency of the shared data transmission channel is calculated based on the target second data and the maximum aggregated data volume of each word at each aggregation node in the shared data transmission channel: Set the data transmission efficiency of the shared data transmission channel to the target data transmission efficiency. After setting, use the shared data transmission channel to receive the second data sent by the risk cloud data receiving terminal.

3. The data processing acceleration method for cloud data architecture according to claim 2, characterized in that: The cloud-based data processing unit includes: The computation settings module is used to provide parameters for cloud data through neural network computation after acquiring cloud data. If the computation fails, the neural network parameters need to be adjusted, only the number of nodes and the number of hidden layers need to be adjusted. The computational data output module is used to set the scheme decision for neural network computation before acquiring data from the cloud; The computational data export module is used to export the scheme sample data and the corresponding neural network computation data, and then compare the two after export.

4. The data processing acceleration method for cloud data architecture according to claim 3, characterized in that: Also includes: The computational data integration unit is used to acquire monitoring data, coordinate data, and time data from the computational data, and to standardize the monitoring data, coordinate data, and time data to obtain standard monitoring data, standard coordinate data, and standard time data. This involves extracting important fields from standard monitoring data, performing data correlation analysis on these important data fields, identifying target important data fields with anomalies, and then extracting and removing the abnormal monitoring data corresponding to these target important data fields from the standard monitoring data. The coordinate features of the standard coordinate data and the time features of the time data are obtained. Based on the coordinate features and time features, the coordinate patterns and time patterns of the standard coordinate data and standard time data are determined respectively. Abnormal coordinate data that does not meet the coordinate patterns are extracted from the standard coordinate data and removed. Abnormal time data that does not meet the time patterns are extracted from the standard time data and removed. Based on the monitoring data labeling distribution map, various data integration rules are determined, and dynamic data integration instructions are established based on these different data integration rules. The dynamic data integration instructions dynamically integrate the confirmed standard monitoring data, standard coordinate data, and standard time data to obtain multiple sets of integrated data, and a dataset is generated based on these multiple sets of integrated data.

5. The data processing acceleration method for cloud data architecture according to claim 4, characterized in that: Also includes: The computational data caching and encryption unit is used to store the generated dataset separately, and then cache and encrypt the data according to the data type.

6. The data processing acceleration method for cloud data architecture according to claim 5, characterized in that: The computational data caching and encryption unit includes: The dataset retrieval module is used to obtain data information from independently stored computation datasets, extract key information fragments from the data in the dataset, and determine the data type of the data information in the dataset based on the data code prefix of the key information fragments. The space determination module is used to determine the target cache space corresponding to the data in the dataset based on the data type, extract the capacity information of the target cache space, and determine the first remaining available space capacity of the target cache space based on the capacity information.

7. The data processing acceleration method for cloud data architecture according to claim 6, characterized in that: The space determination module is also used for Obtain the data length of the data in the dataset. When the first remaining available space capacity is greater than the data length, cluster the data in the dataset to obtain a set of sub-data types corresponding to the data in the dataset, and set a type identifier for each sub-data type.

8. The data processing acceleration method for cloud data architecture according to claim 7, characterized in that: The computational data caching and encryption unit further includes: The space partitioning module is used to divide the target cache space into first blocks based on the type identifier, add block identifiers to the partitioned sub-target cache spaces, and divide each sub-target cache space into second blocks to obtain the first storage entry and the second storage entry corresponding to each sub-target cache space, wherein the block identifier corresponds to the type identifier. The data caching module is used to extract the target content of the central data of each sub-data type based on the type identifier, and cache the type identifier and the target content to the first storage entry and the second storage entry respectively. The data update module is used to monitor the reading operations of the central data in the dataset in real time. When reading of the data is detected, it determines the real-time reading amount of the data based on the reading process and determines the data tail of the remaining central data in the target cache space based on the reading amount. The space update module is used to move the preset cursor to the end of the data in the dataset in the target cache space, determine the second remaining available space capacity of the target cache space based on the moving result, and indicate the second remaining available space capacity of the target cache space based on the preset cursor. The space capacity encryption module is used to divide each piece of computational data in the cached dataset in the first available space capacity and the second available space capacity into multiple data nodes; The dataset is encrypted based on multiple partitioned data nodes.

Images