Information management system and method based on artificial intelligence
By adding SQL interception layer and transaction isolation levels to the information management system, the data identification model is trained, and data loss caused by the risk of database downtime is solved, real-time cloud backup and data protection of key data are realized.
Patent Information
- Application Number
- CN202411889109.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2044-12-20
AI Technical Summary
In the information management system, the database faces the risk of downtime in an environment with high concurrency, frequent data operations and complex system architecture, which affects the normal operation of the system and may lead to the loss of critical data.
Add an SQL interception layer between the application client and the database server, establish the association relationship between SQL data and the received message data of the interface layer, set the data operation transaction isolation level of important data information, filter SQL statements higher than or equal to the set isolation level, train the important information data identification model, identify through the model and perform cloud backup.
It realizes non-invasive real-time monitoring and intelligent identification of important data when the system is downtime, ensures the security of key data and the continuity of business, and improves the disaster recovery capabilities and data protection level of the database system.
Smart Images

Figure CN119759662B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information management, and in particular to an information management system and method based on artificial intelligence. Background Art
[0002] In information management systems, databases are core storage components used to store and manage large amounts of business data and critical information (such as financial transactions, orders, and inventory management). As business complexity and importance increase, ensuring the security and availability of this critical information becomes a key focus in system design. In environments with high concurrency, frequent data operations, and complex system architectures, databases can face the risk of downtime, impacting system operations and even leading to the loss of critical data or information.
[0003] Based on the above problems, in order to effectively distinguish and identify important data information and ensure that the data information that needs to be backed up can be quickly identified and backed up in the cloud when the system is at risk of downtime, a technical solution is proposed. This solution adds an SQL interception layer between the application client and the database server to capture and record all SQL operations. At the same time, it establishes an association between SQL data and the message data received by the interface layer. On this basis, the transaction isolation level of important data information is set, and the message data corresponding to all SQL operations that are higher than or equal to the set isolation level are filtered out. Then, the important information data recognition model is trained based on the information characteristics of the message data to perform important data information recognition. Summary of the Invention
[0004] In order to solve the above technical problems, an information management method based on artificial intelligence is provided. This technical solution solves the problems raised in the above background technology.
[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:
[0006] An artificial intelligence-based information management method includes: adding an SQL interception layer between an application client and a database server to establish an association between SQL data and message data received by the interface layer;
[0007] Set the transaction isolation level for data operations on important data information, and filter all SQL statements at or above the set transaction isolation level based on the SQL language type corresponding to the database used;
[0008] Extract key field features from SQL statements in training data and establish a key field feature information comparison library;
[0009] Obtain the message data received by the interface layer in the training data, extract the information features of the message data, and train the important information data recognition model;
[0010] When there is a risk of database downtime, the message data information of the interface layer is identified through the important information data identification model, and the important data information is backed up in the cloud based on the identification results.
[0011] Preferably, the step of adding an SQL interception layer between the application client and the database server and establishing an association relationship between SQL data and message data received by the interface layer specifically includes:
[0012] Configure the SQL interception layer between the application client and the database server as the middle layer for data requests;
[0013] Set a historical SQL statement collection time window to capture historical SQL statements executed within the set time window, and record the submission information of historical SQL statements. The submission information includes the SQL text, execution time, and thread ID of the thread that submitted the SQL statement;
[0014] Analyze the correspondence between SQL statements and message data in logs, and establish a historical sample database.
[0015] Preferably, analyzing the correspondence between SQL statements and message data in the logs and establishing a historical sample database specifically includes:
[0016] Use log analysis tools to retrieve and filter all log entries containing thread IDs for historical SQL statement submissions within a specified time window from system logs. Filter log data based on thread IDs and request time sequence to locate message data received by the interface layer within the time window.
[0017] The historical SQL data of the time window and the message data received by the interface layer are mapped through thread identifiers and stored in the historical sample database.
[0018] Preferably, the setting of the data operation transaction isolation level of important data information, based on the SQL language type corresponding to the database used, screening all SQL statements of the transaction isolation level and above, specifically includes:
[0019] Based on the target system's business type, set the data operation control transaction isolation level benchmark, and obtain the SQL language type corresponding to the database type used by the controlled target system;
[0020] According to the keywords of the transaction isolation level setting statement corresponding to the SQL language type, retrieve and filter the historical SQL data information in the historical sample database that enabled the setting of the data operation control transaction isolation level baseline and above when performing database operations, and obtain the corresponding mapping relationship to obtain the message data received by the interface layer;
[0021] The filtered historical SQL data information and the message data received by the interface layer are used as training data for the important information data recognition model.
[0022] Preferably, extracting key field features from SQL statements in the training data and establishing a key field feature information comparison library specifically includes:
[0023] Obtain the SQL statements corresponding to add, delete, and modify operations in the SQL data information in the training data and split the statements. Select the fields before the conditional statements and the operation data to be written corresponding to the fields to form field-data key-value pairs.
[0024] Perform frequency analysis on the field names in all field-data key-value pairs in the training data, filter out the field-data key-value pairs that appear more frequently than the preset frequency, and mark them as high-frequency field-data key-value pairs;
[0025] Obtain the storage information of high-frequency field-data key-value pairs in the database, delete the data portion of the key-value pairs whose storage fields are long text type, retain only their data length, and mark the processed key-value pairs as Class I key-value pairs. Meanwhile, do not process the key-value pairs of the remaining fields whose storage fields are not long text type and mark them as Class II key-value pairs.
[0026] Perform ASC encoding conversion and feature integration on the first-class key-value pairs and the second-class key-value pairs respectively, and integrate the encoding range of the data with the same field name;
[0027] A key field feature information comparison library is established based on the integrated key-value pair features.
[0028] Preferably, the acquiring of message data received by the interface layer in the training data, extracting information features of the message data, and training the important information data recognition model specifically includes:
[0029] Based on the mapping relationship between SQL data in the training data and the message data received by the interface layer, all message data information is obtained;
[0030] According to the message structure settings received by the application interface, select and use the message structure parsing framework to parse the message data content, mark the successfully parsed message data as structured message data, and mark the message data that failed to be parsed as unstructured message data;
[0031] Extract common features of structured and unstructured message data information, including time dimension sequence features and spatial dimension statistical features of message data;
[0032] After extracting field information from the message data, the message data is converted into a key-value pair format for feature extraction;
[0033] Perform correlation analysis on the extracted unstructured message data features and all the first-class key-value pair features in the key field feature information comparison library, and perform correlation analysis on the extracted structured message data features and all the second-class key-value pair features in the key field feature information comparison library;
[0034] Based on the dynamic programming algorithm, the longest number of consecutive identical features between the message data features and the key-value pair features is calculated to evaluate the field feature matching of the message data. The matching evaluation result is used as the field matching feature of the message data.
[0035] Integrate field matching features with common features to train important information data recognition models.
[0036] Preferably, the integration of field matching features and common features to train the important information data recognition model specifically includes:
[0037] Integrate the field matching features and common features of all structured message data to train a data recognition model for important information in structured message data;
[0038] Integrate the field matching features of all unstructured message data with common features to train an important information data recognition model for unstructured message data.
[0039] Preferably, when there is a risk of database downtime, identifying the message data information of the interface layer through the important information data identification model, and backing up the important data information in the cloud according to the identification result specifically includes:
[0040] Regularly monitor the database server's CPU usage, memory usage, disk I / O, number of connections, and query latency to comprehensively assess the risk of database downtime;
[0041] When the database is judged to be at risk of downtime, the important data information backup mechanism is activated, and the cloud server receives the latest message data information from the program client interface layer and classifies the message data types;
[0042] Identify the corresponding types of message data using the important information data identification model for structured message data and the important information data identification model for unstructured message data, and store the corresponding important information data in the cloud server cache based on the identification results;
[0043] When the risk of database downtime is detected to be eliminated, the important information cached by the cloud server will be sent back to the program client for processing and information synchronization.
[0044] Preferably, when the risk of database downtime is detected to be eliminated, the important information cached by the cloud server is sent back to the program client for processing, and the information synchronization specifically includes:
[0045] The database downtime risk is continuously monitored at the set frequency. When the database downtime risk is detected to be eliminated, the time when the important information data packets are switched to the cloud server is obtained, and the important information data packets after the switching time are sent back to the program client to re-enter the database.
[0046] Furthermore, an artificial intelligence-based information management system is proposed, which is used to implement the artificial intelligence-based information management method described above, comprising:
[0047] An SQL interception module adds an SQL interception layer between the application client and the database server, establishes a mapping relationship between the historical SQL data of a set time window and the message data received by the interface layer through thread identifiers, and stores them in a historical sample database;
[0048] A training data screening module, wherein the training data screening module retrieves and screens historical SQL data information in a historical sample database that has enabled the setting of the data operation control transaction isolation level benchmark and above when performing database operations, and simultaneously obtains the corresponding mapping relationship to obtain the message data received by the interface layer as training data;
[0049] A key field feature extraction module selects the fields preceding the conditional statement and the corresponding operation data to be written to form field-data key-value pairs. Based on the storage information of high-frequency field-data key-value pairs in the database, the key-value pairs are divided into first-class key-value pairs and second-class key-value pairs, and feature integration is performed. A key field feature information comparison library is established based on the integrated key-value pair features.
[0050] A recognition model training module, which extracts common features and field matching features of structured and unstructured message data information, and performs important information data recognition model training for structured and unstructured message data respectively;
[0051] The cloud backup module, when there is a risk of database downtime, identifies the message data information of the interface layer through the important information data recognition model, and performs cloud backup of the important data information based on the recognition result.
[0052] Compared with the prior art, the present invention has the following beneficial effects:
[0053] The solution combines SQL interception, transaction isolation level setting, feature extraction, and data recognition model training, without requiring architecture or code modifications to the managed system. It enables non-invasive real-time monitoring and intelligent identification of important data, and can promptly perform cloud-based backup of important data when there is a risk of system downtime, ensuring the security of critical data and business continuity, effectively improving the disaster recovery capabilities and data protection level of the database system. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 This is a flow chart of an information management method based on artificial intelligence of the present invention;
[0055] Figure 2 The present invention adds an SQL interception layer between the application client and the database server, and establishes a flow chart of the association relationship between SQL data and message data received by the interface layer;
[0056] Figure 3 For the data operation transaction isolation level of important data information set in the present invention, all SQL statement flow charts of the transaction isolation level and above are screened according to the SQL language type corresponding to the database used;
[0057] Figure 4 The invention is to extract key field features from SQL statements in training data and establish a flow chart of key field feature information comparison library;
[0058] Figure 5 The flow chart of the present invention for obtaining message data received by the interface layer in the training data, extracting information features of the message data, and training an important information data recognition model;
[0059] Figure 6 This is a flowchart of the present invention for identifying message data information of the interface layer through an important information data identification model when there is a risk of database downtime, and performing cloud backup of important data information based on the identification result;
[0060] Figure 7 This is a schematic diagram of the structure of an information management system based on artificial intelligence of the present invention. DETAILED DESCRIPTION
[0061] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.
[0062] Reference Figure 1 As shown, an information management method based on artificial intelligence includes:
[0063] Add an SQL interception layer between the application client and the database server to establish an association between SQL data and the message data received by the interface layer;
[0064] Set the transaction isolation level for data operations on important data information, and filter all SQL statements at or above the set transaction isolation level based on the SQL language type corresponding to the database used;
[0065] Extract key field features from SQL statements in training data and establish a key field feature information comparison library;
[0066] Obtain the message data received by the interface layer in the training data, extract the information features of the message data, and train the important information data recognition model;
[0067] When there is a risk of database downtime, the message data information of the interface layer is identified through the important information data identification model, and the important data information is backed up in the cloud based on the identification results.
[0068] Reference Figure 2 As shown, adding an SQL interception layer between the application client and the database server to establish an association between SQL data and message data received by the interface layer specifically includes:
[0069] Database proxy tools or other mature persistence layer frameworks (such as MyBatis and Hibernate) configure an SQL interception layer between the application client and the database server as an intermediate layer for data requests;
[0070] Set a historical SQL statement collection time window (for example, 10 seconds or 30 seconds) to capture SQL statements executed within the set time. This time window can be adjusted based on the amount of real-time data. Capture historical SQL statements executed within the set time window and record submission information for the historical SQL statements. The submission information includes the SQL text, execution time, and the thread ID of the thread that submitted the SQL statement.
[0071] Use log analysis tools to retrieve and filter all log entries containing historical SQL statement submission thread identifiers within a set time window from system logs. Filter log data based on thread identifiers and request time sequence, and locate message data received by the interface layer within the time window. The message data contains request content related to the captured SQL submission information. The message data received by the interface layer may include request bodies, request parameters, and other content for subsequent data association.
[0072] The historical SQL data of the time window and the message data received by the interface layer are mapped to each other through thread identifiers and stored in the historical sample database. The structure of the database table can include the following fields: auto-increment primary key, SQL statement text, SQL execution time, SQL submission thread identifier, message data received by the interface layer, and timestamp of message reception by the interface layer.
[0073] Reference Figure 3 As shown, set the data operation transaction isolation level for important data information. According to the SQL language type corresponding to the database used, filter all SQL statements with the transaction isolation level set or above, including:
[0074] Analyze the data consistency and security requirements of the target system based on its specific business type (e.g., finance, e-commerce, social platforms, etc.). The business type determines the transaction isolation requirements for database operations. For example, financial systems typically require higher transaction isolation levels to ensure strict consistency of transaction data. The higher the isolation level required for data operations, the more important the information and data. The important information and data in this solution refers to this type of information and data.
[0075] Based on the target system's business type, set a baseline for the transaction isolation level for data operation control. Also, obtain the SQL language type corresponding to the database type used by the target system being controlled. Based on the business requirements, set a baseline for the transaction isolation level for data operation control. Transaction isolation levels typically include READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ, and SERIALIZABLE. Each isolation level provides different data consistency guarantees. REPEATABLE READ is sufficient for most business scenarios. If the isolation level required by the business type is unknown, set REPEATABLE READ as the default baseline.
[0076] Set the statement keywords according to the transaction isolation level corresponding to the SQL language type, for example:
[0077] MySQL: SET TRANSACTION ISOLATION LEVEL REPEATABLE READ,
[0078] SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
[0079] Oracle: ALTER SESSION SET ISOLATION_LEVEL = SERIALIZABLE;
[0080] The keywords for MySQL retrieval are REPEATABLE READ and SERIALIZABLE. Oracle database does not support the REPEATABLE READ isolation level, so its retrieval and filtering keywords are only SERIALIZABLE.
[0081] Retrieve and filter historical SQL data information in the historical sample database that has enabled the transaction isolation level baseline and above when performing database operations, and obtain the corresponding mapping relationship to obtain the message data received by the interface layer;
[0082] The filtered historical SQL data information and the message data received by the interface layer are used as training data for the important information data recognition model.
[0083] Reference Figure 4 As shown, extracting key field features from SQL statements in the training data and establishing a key field feature information comparison library specifically include:
[0084] Obtain the SQL statements corresponding to the add, delete, and modify operations in the SQL data information in the training data and split the statements. Select the fields before the conditional statements and the operation data to be written corresponding to the fields to form field-data key-value pairs. For example:
[0085] The statement UPDATE price SET money = 1000 WHERE price_id = 12345 has a field-data key-value pair of "money: 1000".
[0086] Perform frequency analysis on the field names in all field-data key-value pairs in the training data, filter out the field-data key-value pairs that appear more frequently than the preset frequency, and mark them as high-frequency field-data key-value pairs. The preset frequency can be freely adjusted according to the sensitivity requirements of the recognition model;
[0087] Obtain the storage information of high-frequency field-data key-value pairs in the database, delete the data portion of the key-value pairs whose storage fields are long text type, retain only their data length, and mark the processed key-value pairs as Class I key-value pairs. Meanwhile, do not process the key-value pairs of the remaining fields whose storage fields are not long text type and mark them as Class II key-value pairs.
[0088] Perform ASC encoding conversion and feature integration on the first-class key-value pairs and the second-class key-value pairs respectively, and integrate the encoding range of the data with the same field name (the union after deduplication). The following is a specific example:
[0089] There are two key-value pairs: money: 1000 and money: 99. The field part money is converted to [109, 111, 110, 101, 121], and the data parts 1000 and 99 are converted to [49, 48, 48, 48] and [57, 57]. After range integration, it is [48, 49...57]. The final conversion result is [109, 111, 110, 101, 121]: [48, 49...57].
[0090] A key field feature information comparison library is established based on the integrated key-value pair features.
[0091] Reference Figure 5 As shown, obtaining the message data received by the interface layer in the training data, extracting the information features of the message data, and training the important information data recognition model specifically include:
[0092] Based on the mapping relationship between SQL data in the training data and the message data received by the interface layer, all message data information is obtained;
[0093] According to the received message structure settings specified by the application interface, such as JSON, XML, and CSV, the message structure parsing framework is selected and used to parse the message data content. The successfully parsed message data is marked as structured message data, and the message data that failed to be parsed is marked as unstructured message data.
[0094] Extract common features of structured and unstructured message data information. Common features include the time dimension sequence features and spatial dimension statistical features of message data. The time dimension sequence features can apply a sliding window on the time series to calculate the mean and variance of the message data in each window to capture short-term change trends. The spatial dimension statistical features can quantify the amount of information in a field or keyword through information entropy. In addition, the message data feature extraction method based on existing open source models is also applicable.
[0095] After extracting field information from the message data, the message data is converted into a key-value pair format for feature extraction;
[0096] Perform correlation analysis on the extracted unstructured message data features and all the first-class key-value pair features in the key field feature information comparison library, and perform correlation analysis on the extracted structured message data features and all the second-class key-value pair features in the key field feature information comparison library;
[0097] Based on the dynamic programming algorithm, the longest number of consecutive identical features between the message data features and the key-value pair features is calculated, and the ratio of the longest number of identical field features to the longer number of features is used as the field feature matching degree of the message data, and the matching degree evaluation result is used as the field matching feature of the message data.
[0098] Calculate the feature hit rate of the value part in the message data feature (it does not need to be continuous, as long as it exists in the key-value pair feature, it is considered a hit). Based on the calculation result, weight the field matching feature of the message data and fine-tune it. The following is a calculation example:
[0099] For the key fields money: 99 and money: 1000 and the message data monkey: 20, the longest consecutive number of identical features is 3 ([109, 111, 110]). The feature matching calculation result is 3 / 6=0.5. The feature hit rate of the value part in the message data features is 100% ([48, 50]). If the fine-tuning weight is set to 0.1, the feature matching calculation result after fine-tuning is 0.6.
[0100] Integrate the field matching features and common features of all structured message data to train an important information data recognition model for structured message data.
[0101] Integrate the field matching features of all unstructured message data with common features to train an important information data recognition model for unstructured message data.
[0102] Reference Figure 6 As shown in the figure, when there is a risk of database downtime, the message data information of the interface layer is identified through the important information data identification model, and the cloud backup of the important data information is performed according to the identification results. Specifically, the following are performed:
[0103] Regularly monitor the database server's CPU usage, memory usage, disk I / O, number of connections, and query latency to comprehensively assess the database's downtime risk. This can be done by performing weighted comprehensive calculations on the above server parameters or manually setting risk thresholds for each parameter for comparison to determine whether there is a downtime risk.
[0104] When the database is judged to be at risk of downtime, the important data information backup mechanism is activated, and the cloud server receives the latest message data information from the program client interface layer and classifies the message data types;
[0105] Identify the corresponding types of message data using the important information data identification model for structured message data and the important information data identification model for unstructured message data, and store the corresponding important information data in the cloud server cache based on the identification results;
[0106] When the risk of database downtime is detected to be eliminated, the important information cached by the cloud server will be sent back to the program client for processing and information synchronization.
[0107] Reference Figure 7 As shown, an information management system based on artificial intelligence includes:
[0108] An SQL interception module adds an SQL interception layer between the application client and the database server, establishes a mapping relationship between the historical SQL data of a set time window and the message data received by the interface layer through thread identifiers, and stores them in a historical sample database;
[0109] A training data screening module, wherein the training data screening module retrieves and screens historical SQL data information in a historical sample database that has enabled the setting of the data operation control transaction isolation level benchmark and above when performing database operations, and simultaneously obtains the corresponding mapping relationship to obtain the message data received by the interface layer as training data;
[0110] A key field feature extraction module selects the fields preceding the conditional statement and the corresponding operation data to be written to form field-data key-value pairs. Based on the storage information of high-frequency field-data key-value pairs in the database, the key-value pairs are divided into first-class key-value pairs and second-class key-value pairs, and feature integration is performed. A key field feature information comparison library is established based on the integrated key-value pair features.
[0111] A recognition model training module, which extracts common features and field matching features of structured and unstructured message data information, and performs important information data recognition model training for structured and unstructured message data respectively;
[0112] The cloud backup module, when there is a risk of database downtime, identifies the message data information of the interface layer through the important information data recognition model, and performs cloud backup of the important data information based on the recognition result.
[0113] Furthermore, the present solution also proposes an artificial intelligence-based information management method storage medium, on which a computer-readable program is stored. When the computer-readable program is called, the above-mentioned artificial intelligence-based information management method and system are executed.
[0114] It is understandable that the storage medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid state disk (SSD).
[0115] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.
Claims
1. An information management method based on artificial intelligence, characterized in that: include: Add an SQL interception layer between the application client and the database server to establish an association between SQL data and the message data received by the interface layer; Set the transaction isolation level for data operations on important data information, and filter all SQL statements at or above the set transaction isolation level based on the SQL language type corresponding to the database used; Extract key field features from SQL statements in training data and establish a key field feature information comparison library; Obtain the message data received by the interface layer in the training data, extract the information features of the message data, and train the important information data recognition model; When there is a risk of database downtime, the message data information of the interface layer is identified through the important information data identification model, and the important data information is backed up in the cloud based on the identification results; The acquiring of message data received by the interface layer in the training data, extracting information features of the message data, and training the important information data recognition model specifically includes: Based on the mapping relationship between SQL data in the training data and the message data received by the interface layer, all message data information is obtained; According to the message structure settings received by the application interface, select and use the message structure parsing framework to parse the message data content, mark the successfully parsed message data as structured message data, and mark the message data that failed to be parsed as unstructured message data; Extract common features of structured and unstructured message data information, including time dimension sequence features and spatial dimension statistical features of message data; After extracting field information from the message data, the message data is converted into a key-value pair format for feature extraction; Perform correlation analysis on the extracted unstructured message data features and all the first-class key-value pair features in the key field feature information comparison library, and perform correlation analysis on the extracted structured message data features and all the second-class key-value pair features in the key field feature information comparison library; Based on the dynamic programming algorithm, the longest number of consecutive identical features between the message data features and the key-value pair features is calculated to evaluate the field feature matching of the message data. The matching evaluation result is used as the field matching feature of the message data. Integrate field matching features with common features to train important information data recognition models.
2. The information management method based on artificial intelligence according to claim 1, characterized in that: The step of adding an SQL interception layer between the application client and the database server and establishing an association relationship between SQL data and message data received by the interface layer specifically includes: Configure the SQL interception layer between the application client and the database server as the middle layer for data requests; Set a historical SQL statement collection time window to capture historical SQL statements executed within the set time window, and record the submission information of historical SQL statements. The submission information includes the SQL text, execution time, and thread ID of the thread that submitted the SQL statement; Analyze the correspondence between SQL statements and message data in logs, and establish a historical sample database.
3. The information management method based on artificial intelligence according to claim 2, characterized in that: Analyzing the correspondence between SQL statements and message data in the logs and establishing a historical sample database specifically includes: Use log analysis tools to retrieve and filter all log entries containing the thread IDs of historical SQL statements submitted within a set time window from system logs. Filter log data based on thread IDs and request time sequence to locate message data received by the interface layer within the time window. The historical SQL data of the time window and the message data received by the interface layer are mapped through thread identifiers and stored in the historical sample database.
4. The information management method based on artificial intelligence according to claim 3, characterized in that: The data operation transaction isolation level for setting important data information is to filter all SQL statements of the transaction isolation level and above according to the SQL language type corresponding to the database used, specifically including: Based on the target system's business type, set the data operation control transaction isolation level benchmark, and obtain the SQL language type corresponding to the database type used by the controlled target system; According to the keywords of the transaction isolation level setting statement corresponding to the SQL language type, retrieve and filter the historical SQL data information in the historical sample database that enabled the setting of the data operation control transaction isolation level baseline and above when performing database operations, and obtain the corresponding mapping relationship to obtain the message data received by the interface layer; The filtered historical SQL data information and the message data received by the interface layer are used as training data for the important information data recognition model.
5. The information management method based on artificial intelligence according to claim 4, characterized in that: Extracting key field features from SQL statements in the training data and establishing a key field feature information comparison database specifically includes: Obtain the SQL statements corresponding to add, delete, and modify operations in the SQL data information in the training data and split the statements. Select the fields before the conditional statements and the operation data to be written corresponding to the fields to form field-data key-value pairs. Perform frequency analysis on the field names in all field-data key-value pairs in the training data, filter out the field-data key-value pairs that appear more frequently than the preset frequency, and mark them as high-frequency field-data key-value pairs; Obtain the storage information of high-frequency field-data key-value pairs in the database, delete the data portion of the key-value pairs whose storage fields are long text type, retain only their data length, and mark the processed key-value pairs as Class I key-value pairs. Meanwhile, do not process the key-value pairs of the remaining fields whose storage fields are not long text type and mark them as Class II key-value pairs. Perform ASC encoding conversion and feature integration on the first-class key-value pairs and the second-class key-value pairs respectively, and integrate the encoding range of the data with the same field name; A key field feature information comparison library is established based on the integrated key-value pair features.
6. The information management method based on artificial intelligence according to claim 5, characterized in that: The integration of field matching features and common features to train the important information data recognition model specifically includes: Integrate the field matching features and common features of all structured message data to train a data recognition model for important information in structured message data; Integrate the field matching features of all unstructured message data with common features to train an important information data recognition model for unstructured message data.
7. The information management method based on artificial intelligence according to claim 6, characterized in that: When there is a risk of database downtime, identifying the message data information of the interface layer through the important information data identification model and backing up the important data information in the cloud according to the identification result specifically include: Regularly monitor the database server's CPU usage, memory usage, disk I / O, number of connections, and query latency to comprehensively assess the risk of database downtime; When the database is judged to be at risk of downtime, the important data information backup mechanism is activated, and the cloud server receives the latest message data information from the program client interface layer and classifies the message data types; Identify the corresponding types of message data using the important information data identification model for structured message data and the important information data identification model for unstructured message data, and store the corresponding important information data in the cloud server cache based on the identification results; When the risk of database downtime is detected to be eliminated, the important information cached by the cloud server will be sent back to the program client for processing and information synchronization.
8. The information management method based on artificial intelligence according to claim 7, characterized in that: When the risk of database downtime is detected to be eliminated, the important information cached by the cloud server is sent back to the program client for processing. The information synchronization specifically includes: The database downtime risk is continuously monitored at the set frequency. When the database downtime risk is detected to be eliminated, the time when the important information data packets are switched to the cloud server is obtained, and the important information data packets after the switching time are sent back to the program client to re-enter the database.
9. An information management system based on artificial intelligence, used to implement an information management method based on artificial intelligence according to any one of claims 1 to 8, characterized in that: include: An SQL interception module adds an SQL interception layer between the application client and the database server, establishes a mapping relationship between the historical SQL data of a set time window and the message data received by the interface layer through thread identifiers, and stores them in a historical sample database; A training data screening module, wherein the training data screening module retrieves and screens historical SQL data information in a historical sample database that has enabled the setting of the data operation control transaction isolation level benchmark and above when performing database operations, and simultaneously obtains the corresponding mapping relationship to obtain the message data received by the interface layer as training data; A key field feature extraction module selects the fields preceding the conditional statement and the corresponding operation data to be written to form field-data key-value pairs. Based on the storage information of high-frequency field-data key-value pairs in the database, the key-value pairs are divided into first-class key-value pairs and second-class key-value pairs, and feature integration is performed. A key field feature information comparison library is established based on the integrated key-value pair features. A recognition model training module, which extracts common features and field matching features of structured and unstructured message data information, and performs important information data recognition model training for structured and unstructured message data respectively; The cloud backup module, when there is a risk of database downtime, identifies the message data information of the interface layer through the important information data recognition model, and performs cloud backup of the important data information based on the recognition result.
Citation Information
Patent Citations
Database processing method and database processing device
CN106293887A
A MySQL master-backup incremental synchronization method based on a Kubernetes system
CN109189860A