Database risk assessment method and system

Abstract

The application discloses a database risk assessment method and system, and relates to the technical fields of database management, software quality guarantee and the like. The database risk assessment method comprises the following steps: obtaining a field to be analyzed from a data table to be evaluated in a target database; performing data feature analysis on the field to be analyzed to obtain statistical fingerprint data, wherein the statistical fingerprint data comprises metadata, field length statistical features and field content statistical features; performing semantic analysis based on the field to be analyzed and the statistical fingerprint data to obtain a semantic analysis result; and performing risk assessment based on the statistical fingerprint data and the semantic analysis result to obtain a risk assessment result of the target database.

Description

Technical Field

[0001] This application relates to technical fields such as database management and software quality assurance, and in particular to a database risk assessment method and system. Background Technology

[0002] As business systems become increasingly complex, data out-of-bounds issues caused by improperly designed database field lengths are becoming more frequent. Traditional methods, relying on manual experience or static analysis, cannot adapt to dynamically changing business environments, necessitating an intelligent and automated risk warning mechanism.

[0003] One related technology assesses database risk based on the maximum length ratio, calculating the ratio of the current maximum field length to the defined length; an alarm is triggered if the ratio exceeds a threshold. However, this method cannot distinguish business semantics, resulting in a high false alarm rate. Another related technology evaluates the database based on a semantic rule base of regular expressions, matching field names against preset rules (e.g., fields containing "id (identification)" are considered fixed length). However, this method has poor generalization ability, high maintenance costs, and cannot handle complex comments. Summary of the Invention

[0004] The embodiments of this application aim to at least partially solve one of the technical problems in the related art. Therefore, the purpose of the embodiments of this application is to provide a database risk assessment method, system, device, and medium that improves the accuracy of database risk assessment.

[0005] This application provides a database risk assessment method, comprising: obtaining a field to be analyzed from a data table to be assessed in a target database; performing data feature analysis on the field to be analyzed to obtain statistical fingerprint data, wherein the statistical fingerprint data includes metadata, field length statistical features, and field content statistical features; performing semantic analysis based on the field to be analyzed and the statistical fingerprint data to obtain semantic analysis results; and performing risk assessment based on the statistical fingerprint data and the semantic analysis results to obtain a risk assessment result for the target database.

[0006] For example, the field to be analyzed includes multiple fields; data feature analysis is performed on the field to be analyzed to obtain statistical fingerprint data, including: performing data feature analysis on each field to obtain the field name, field data type, maximum field definition length, and field annotation data corresponding to each field, as the metadata corresponding to each field; performing data feature analysis on each field to obtain the maximum field usage length, average field length, field length standard deviation, and field length distribution data corresponding to each field, as the field length statistical features corresponding to each field; and performing data feature analysis on each field to obtain the field non-empty rate, field unique value ratio, and frequency of occurrence of specific characters in the field, as the field content statistical features corresponding to each field.

[0007] For example, semantic analysis is performed based on the field to be analyzed and statistical fingerprint data to obtain semantic analysis results, including: obtaining sample data corresponding to the field to be analyzed from the data table to be evaluated; obtaining prompt word data based on the sample data and statistical fingerprint data; performing semantic analysis on each field based on the prompt word data to obtain semantic type data, content pattern data, growth potential data and inference data corresponding to each field; and obtaining semantic analysis results based on the semantic type data, content pattern data, growth potential data and inference data.

[0008] For example, risk assessment based on statistical fingerprint data and semantic analysis results yields the risk assessment results for the target database, including: performing a risk imminent assessment on the field to be analyzed based on statistical fingerprint data to obtain a risk imminent assessment result; performing a growth potential assessment on the field to be analyzed based on semantic analysis results to obtain a growth potential assessment result; performing a pattern anomaly assessment on the field to be analyzed based on semantic analysis results and statistical fingerprint data to obtain a pattern anomaly assessment result; performing a distribution skew assessment on the field to be analyzed based on statistical fingerprint data to obtain a distribution skew assessment result; and weighting the risk imminent assessment result, growth potential assessment result, pattern anomaly assessment result, and distribution skew assessment result based on the growth potential coefficient, pattern anomaly coefficient, and distribution skew coefficient to obtain the risk assessment results for the target database.

[0009] For example, the statistical fingerprint data includes the maximum length of the field definition and the maximum length of the field used; based on the statistical fingerprint data, a risk imminent assessment is performed on the field to be analyzed to obtain the risk imminent assessment result, including: determining the initial risk imminent data based on the maximum length of the field definition and the maximum length of the field used; and performing nonlinear processing on the initial risk imminent data based on preset coefficients to obtain the risk imminent assessment result.

[0010] For example, the semantic analysis results include growth potential data; the growth potential of the field to be analyzed is evaluated based on the semantic analysis results to obtain the growth potential evaluation results, including: mapping the growth potential data with first preset mapping data to obtain the growth potential evaluation results, wherein the first preset mapping data includes the relationship between the growth potential data and a first preset value.

[0011] For example, the statistical fingerprint data includes field data types, and the semantic analysis results include content pattern data. Based on the semantic analysis results and the statistical fingerprint data, a pattern anomaly assessment is performed on the field to be analyzed to obtain a pattern anomaly assessment result, including: when the field data type is a variable length type, and the content pattern data includes any one of comma-separated data, structured data, and single-value data, a mapping process is performed based on a second preset mapping data to obtain a pattern anomaly assessment result, wherein the second preset mapping data includes the relationship between the content pattern data and a second preset value.

[0012] For example, the statistical fingerprint data includes field length distribution data and the maximum length of the field definition; the distribution skewness of the field to be analyzed is evaluated based on the statistical fingerprint data to obtain the distribution skewness evaluation result, including: determining the field skewness data based on the field length distribution data, the maximum length of the field definition, and a preset value; and obtaining the distribution skewness evaluation result based on the field skewness data, a preset threshold, and a third preset mapping data, wherein the third preset mapping data includes the relationship between the field skewness data, the preset threshold, and the third preset value.

[0013] For example, the database risk assessment method also includes generating a risk assessment report based on the risk assessment results, semantic analysis results, risk imminent assessment results, growth potential assessment results, pattern anomaly assessment results, and distribution skew assessment results.

[0014] Another embodiment of this application provides a database risk assessment system, which includes: an acquisition module for acquiring fields to be analyzed from a data table to be assessed in a target database; a first analysis module for performing data feature analysis on the fields to be analyzed to obtain statistical fingerprint data, wherein the statistical fingerprint data includes metadata, field length statistical features, and field content statistical features; a second analysis module for performing semantic analysis based on the fields to be analyzed and the statistical fingerprint data to obtain semantic analysis results; and an assessment module for performing risk assessment based on the statistical fingerprint data and the semantic analysis results to obtain a risk assessment result for the target database.

[0015] Another embodiment of this application provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0016] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0017] In the above embodiments, the database risk assessment method includes: obtaining the fields to be analyzed from the data table to be assessed in the target database; performing data feature analysis on the fields to be analyzed to obtain statistical fingerprint data, wherein the statistical fingerprint data includes metadata, field length statistical features, and field content statistical features; performing semantic analysis based on the fields to be analyzed and the statistical fingerprint data to obtain semantic analysis results; and performing risk assessment based on the statistical fingerprint data and the semantic analysis results to obtain the risk assessment results of the target database. By performing automated data feature analysis on the fields to be analyzed to generate statistical fingerprint data containing metadata, field length statistical features, and field content statistical features, the inherent structure and distribution characteristics of the data can be captured systematically and multidimensionally, providing rich and reliable quantitative basis for risk assessment. By combining the statistical fingerprint data with the semantic analysis results based on the fields and statistical fingerprints, a deep integration of quantitative statistical features and qualitative semantic information is achieved, enhancing the accuracy and reliability of risk assessment. Attached Figure Description

[0018] Figure 1 A flowchart of a database risk assessment method provided for embodiments of this application; Figure 2 A schematic diagram of JSON-formatted prompt word data provided for embodiments of this application; Figure 3 A schematic diagram of the field semantic analysis results provided for the implementation of this application; Figure 4 Another schematic diagram of semantic analysis results for an embodiment of this application; Figure 5 A schematic diagram of the risk assessment report provided for the implementation of this application; Figure 6 A block diagram of a database risk assessment system provided for another embodiment of this application; Figure 7 Flowchart of another database risk assessment method provided for the implementation of this application; Figure 8 Flowchart of another database risk assessment method provided for the implementation of this application; Figure 9 A block diagram of an electronic device provided for another embodiment of this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0020] As business systems become increasingly complex, data out-of-bounds issues caused by improperly designed database field lengths are becoming more frequent. Traditional methods, relying on manual experience or static analysis, cannot adapt to dynamically changing business environments, necessitating an intelligent and automated database risk warning mechanism.

[0021] One related technology assesses database risk based on the maximum length ratio, calculating the ratio of the current maximum field length to the defined length; an alarm is triggered if the ratio exceeds a threshold. However, this method cannot distinguish business semantics, resulting in a high false alarm rate. Another related technology evaluates the database based on a semantic rule base of regular expressions, matching field names against preset rules (e.g., fields containing "id" are considered fixed length). However, this method has poor generalization ability, high maintenance costs, and cannot handle complex comments.

[0022] The specific disadvantages of the relevant technologies are as follows: (1) The scanning tool only makes judgments based on the ratio of "current maximum length" to "defined maximum length". This results in a user_uuid field with a content of 32-bit UUID (Universally Unique Identifier) ​​and a defined length of 36, and a user_comment field with a content of user comments, a current maximum of 250 characters, and a defined length of 255, possibly being assessed as equally high risk. This is obviously unreasonable, because the former has a fixed length and no risk of exceeding the limit, while the latter has an extremely high risk. This "mechanical" judgment, which lacks an understanding of the business meaning of the fields, leads to a large number of false alarms and consumes the energy of developers. (2) Passive response and post-event remediation: Database operation and maintenance is usually passive. Problems are only discovered when the application fails to write data due to excessive length, generates runtime errors, or causes business logic errors due to silent truncation of data. At this time, online failures or data quality problems have often already occurred, the repair cost is high, and data loss may occur. (3) Reliance on human experience and lack of systematic approach: When designing table structures, database administrators (DBAs) or developers largely rely on personal experience and initial demand estimates when setting field lengths. During long-term system evolution and business changes, the lack of a systematic approach to periodically review and evaluate the rationality of these assumptions easily leads to "experience failure." (4) Limitations of static checks and inability to reflect real-world usage: Most code scanning or database structure checking tools can only perform static analysis, such as checking for "magic numbers" or unreasonable field type definitions. They cannot analyze the dynamic trends and content characteristics of the actual data stored in the fields, and therefore cannot anticipate potential risks arising from business growth.

[0023] The purpose of this application is to address the fundamental challenge of existing technologies failing to accurately assess risk by combining field business semantics, and to design a novel, intelligent database field risk prediction framework. This framework not only analyzes "how much data was used," but also uses a large AI (Artificial Intelligence) model to deeply analyze "what the field is used for," combining statistical and semantic features to establish a context-aware risk assessment model. Ultimately, this achieves accurate quantification, ranking, and early warning of out-of-bounds risks, helping technical teams focus on truly high-risk points, optimize the system in advance, and ensure data integrity and system stability.

[0024] In view of this, this application proposes a database risk assessment method. Through intelligent scanning and data feature analysis, it proactively predicts and quantifies the risk of database field content length exceeding the preset definition. By introducing AI large model to analyze field information, annotations and other metadata, it empowers the system to "understand" the purpose of the fields. Combined with actual data usage, it achieves a leapfrog transformation from "passive firefighting" and "experience judgment" to "intelligent prediction" and "proactive defense".

[0025] Figure 1 A flowchart illustrating the database risk assessment method provided for embodiments of this application.

[0026] like Figure 1 As shown, the database risk assessment method 100 provided in this application includes, for example, steps S110-S140.

[0027] Step S110: Obtain the fields to be analyzed from the data table to be evaluated in the target database.

[0028] For example, the data table to be evaluated may include one or more, and the fields to be analyzed may include one or more fields. Each field includes metadata and sample data. The metadata may include field name, maximum length of field definition, field comments, data type, etc., and the sample data may include one or more actual stored data corresponding to each field.

[0029] Step S120: Perform data feature analysis on the field to be analyzed to obtain statistical fingerprint data, which includes metadata, field length statistical features, and field content statistical features.

[0030] For example, metadata such as field name, maximum field definition length, field comments, data type, etc., and field length statistical features include maximum field length, average field length, field length standard deviation, and field length distribution data (e.g., the length distribution of sample data for each field). Field content statistical features include field non-empty rate (the proportion of non-empty values ​​in each field to the total sample data), field unique value proportion (the proportion of unique values ​​in each field to the non-empty values ​​in the field), and frequency of occurrence of specific characters in the field (specific characters such as commas, hyphens, etc.).

[0031] Step S130: Perform semantic analysis based on the field to be analyzed and statistical fingerprint data to obtain the semantic analysis results.

[0032] For example, semantic analysis is implemented based on a Large Language Model (LLM). Prompt word data is generated based on the field to be analyzed and statistical fingerprint data. The prompt word data is sent to the Large Language Model, which performs semantic analysis on the field to be analyzed and statistical fingerprint data based on the prompt word data to obtain semantic analysis results. The semantic analysis results include semantic type data (such as name, ID (identification), description, hierarchical path, etc.), content pattern data (such as single value, comma-separated list, JSON, etc.), growth potential data (fixed, low, medium, hierarchical, unbounded text, etc.), and inference data.

[0033] Step S140: Risk assessment is performed based on statistical fingerprint data and semantic analysis results to obtain the risk assessment results of the target database.

[0034] For example, the risk assessment includes risk imminent assessment, growth potential assessment, pattern anomaly assessment, and distribution skewness assessment. The risk assessment results of the target database are obtained by weighting the results of the risk imminent assessment, growth potential assessment, pattern anomaly assessment, and distribution skewness assessment. The risk assessment results include a comprehensive score.

[0035] In the above embodiments, by performing automated data feature analysis on the fields to be analyzed, statistical fingerprint data containing metadata, field length statistical features, and field content statistical features is generated. This can systematically and multidimensionally capture the inherent structure and distribution characteristics of the data, providing rich and reliable quantitative basis for risk assessment. By combining the statistical fingerprint data with the semantic analysis results based on fields and statistical fingerprints, a deep integration of quantitative statistical features and qualitative semantic information is achieved, enhancing the accuracy and reliability of risk assessment.

[0036] In one example, the fields to be analyzed include multiple fields. Data feature analysis is performed on the fields to be analyzed to obtain statistical fingerprint data, including: performing data feature analysis on each field to obtain the field name, field data type, maximum field definition length, and field annotation data for each field, which serve as the metadata for each field; performing data feature analysis on each field to obtain the maximum field usage length, average field length, field length standard deviation, and field length distribution data for each field, which serve as the statistical features of field length for each field; and performing data feature analysis on each field to obtain the field non-empty rate, field unique value ratio, and frequency of occurrence of specific characters for each field, which serve as the statistical features of field content for each field.

[0037] Specifically, all data tables to be evaluated are traversed, and a comprehensive "data profile" is created for each field (column). This prepares rich and accurate "ingredients" for AI analysis. For each field in the data tables to be evaluated in the target database, "statistical fingerprint data" is calculated and cached.

[0038] Perform data feature analysis on each field to obtain metadata, including: field name (e.g., full_name, leader_name, org_path); field data type (e.g., VARCHAR(255), INT); field definition maximum length (e.g., 255); and field comment (e.g., "organization full path name", "responsible person's name", "organization ID path").

[0039] Perform data feature analysis on each field to obtain field length statistics, including: Maximum used length (current_max_length): For example, analyzing the `full_name` field to find the longest value. Average length (avg_length): The average length of all non-empty data values. Standard deviation (length_stddev): The dispersion of lengths. A large standard deviation indicates highly uneven data length distribution. Length distribution histogram (length_distribution_histogram): Statistics on the distribution of all data lengths. For example, for the `full_name` field: 5% of lengths are between 10-20, 15% are between 20-30, 40% are between 30-40, 30% are between 40-50, and 10% are over 50. This distribution pattern itself is a strong signal.

[0040] Data feature analysis is performed on each field to obtain field content statistics, including: * **Not null ratio:** The proportion of non-null values ​​in the field. * **Uniqueness ratio:** COUNT(DISTINCT a) / COUNT(a), where 'a' represents the field, COUNT(a) represents the total number of rows in field 'a', DISTINCT a represents deduplication, and COUNT(DISTINCT a) represents the number of unique non-null values ​​in field 'a'. The ID field is close to 1, while the status field is very low. * **Special character frequency:** For example, the frequency of commas (,) in the leader_name field and the frequency of hyphens (-) in the org_path field.

[0041] In one example, semantic analysis is performed based on the field to be analyzed and statistical fingerprint data to obtain semantic analysis results, including: obtaining sample data corresponding to the field to be analyzed from the data table to be evaluated; obtaining prompt word data based on the sample data and statistical fingerprint data; performing semantic analysis on each field based on the prompt word data to obtain semantic type data, content pattern data, growth potential data and inference data corresponding to each field; and obtaining semantic analysis results based on semantic type data, content pattern data, growth potential data and inference data.

[0042] Specifically, all fields to be analyzed in the target database (including sample data `data_samples`) and statistical fingerprint data (`statistical_fingerprint`) are organized into a task queue. For each field in the queue, a structured Prompt is automatically assembled. The assembled Prompt data is then input into a large language model. The large language model infers from the Prompt data to obtain semantic type data `semantic_type`, content pattern data `content_pattern`, growth potential data `content_pattern`, and reasoning data for each field. The large language model can include large language models based on the Transformer architecture (such as GPT-4, ChatGLM, and Wenxin Yiyan), which have powerful natural language understanding and reasoning capabilities. Semantic type data `semantic_type` includes names, IDs (identification), descriptions, hierarchical paths, etc.; content pattern data `content_pattern` includes single values, comma-separated lists, JSON, etc.; and growth potential data `content_pattern` includes fixed, low, medium, hierarchical, and unbounded text, etc.

[0043] For example, such as Figure 2 As shown, taking the leader_name field (the field to be analyzed) of the org_manager table (the data table to be evaluated, table_manager) as an example, Figure 2 It's a JSON structure for inter-program communication, which is actually converted into natural language instructions when sent to the LLM. Figure 2 The structured information is converted into a natural language prompt that is friendly to large models and sent to the LLM service. This is no longer a manual operation but an internal automated process. The prompt data for the natural language is as follows: "You are a senior data architect. Please analyze the following target database fields... [Enter the field to be analyzed here]... Please determine its 'semantic type' (e.g., name, ID, description, hierarchical path, status code, amount, date, etc.), 'content pattern' (e.g., single value, comma-separated list, JSON, key-value pairs, free text, etc.), and 'length growth potential' (e.g., fixed, low, medium, high, hierarchical, unbounded text). Please return your analysis in JSON format and provide the reasoning process." The large language model is based on prompt word data for: (1) data preprocessing, cleaning field annotations, removing meaningless characters, and extracting key semantic information; (2) output verification: setting a confidence threshold (e.g., 0.8), and automatically marking "requires manual review" when the model output confidence is lower than the threshold; (3) batch processing optimization: using an asynchronous task queue to support concurrent processing, setting request frequency limits and retry mechanisms; (4) result (semantic analysis result) caching: caching the analysis results to avoid repeatedly analyzing the same fields.

[0044] For example, such as Figure 3 (Inference is made regarding leader_name) Figure 4 (For inference of org_path) As shown, the large language model performs inference based on prompt words to obtain semantic type data (semantic_type), content pattern data (content_pattern), growth potential data (growth_potential), and reasoning data (reasoning) for each field.

[0045] In one example, risk assessment is performed based on statistical fingerprint data and semantic analysis results to obtain the risk assessment results for the target database. This includes: performing a risk imminent assessment on the field to be analyzed based on statistical fingerprint data to obtain a risk imminent assessment result; performing a growth potential assessment on the field to be analyzed based on semantic analysis results to obtain a growth potential assessment result; performing a pattern anomaly assessment on the field to be analyzed based on semantic analysis results and statistical fingerprint data to obtain a pattern anomaly assessment result; performing a distribution skew assessment on the field to be analyzed based on statistical fingerprint data to obtain a distribution skew assessment result; and weighting the risk imminent assessment results, growth potential assessment results, pattern anomaly assessment results, and distribution skew assessment results based on the growth potential coefficient, pattern anomaly coefficient, and distribution skew coefficient to obtain the risk assessment results for the target database.

[0046] Specifically, the risk proximity assessment results include the initial risk proximity score (P_score) and the risk proximity score F(P_score), where F(P_score) represents the amplification of P_score using a nonlinear function F(x). The growth potential assessment results include the growth potential score (G_score), the pattern anomaly assessment results include the pattern anomaly score (A_score), the distribution skewness assessment results include the distribution skewness score (D_score), and the risk assessment results include the risk assessment score FinalRiskScore. The finalRiskScore of the target database is obtained by weighting the growth potential coefficient w_g, the pattern anomaly coefficient w_a, the distribution skewness coefficient w_d, the initial risk proximity score P_score, the growth potential score G_score, the pattern anomaly score A_score, and the distribution skewness score D_score, as shown in formula (1). FinalRiskScore = F(P_score) * (w_g * G_score + w_a * A_score + w_d *D_score) (1) Of these, w_g (growth potential weight) should be the highest (e.g., 0.5), as it fundamentally determines whether a field will "grow." w_a (pattern anomaly weight) should be the next highest (e.g., 0.3), as it reflects design flaws. w_d (distribution skew weight) serves as a supplement (e.g., 0.2), reflecting the "current state" of risk. These weights can be adjusted by the user in the system configuration according to their governance strategy.

[0047] The weighting design is based on historical data analysis and expert experience, and uses the Analytic Hierarchy Process (AHP) to determine relative importance: w_g = 0.5 (growth potential coefficient): Based on statistical analysis of 1000 real fields, growth potential is the most critical factor in risk prediction, contributing 48.7%; w_a = 0.3 (pattern anomaly coefficient): Pattern anomalies often indicate design flaws, accounting for 31.2% of the out-of-bounds incidents; w_d = 0.2 (distribution skewness coefficient): As a supplementary indicator, it reflects the urgency of the risk.

[0048] In one example, the statistical fingerprint data includes the maximum length of the field definition and the maximum length of the field in use. Based on the statistical fingerprint data, a risk imminent assessment is performed on the field to be analyzed to obtain the risk imminent assessment result, including: determining the initial risk imminent data based on the maximum length of the field definition and the maximum length of the field in use; and performing nonlinear processing on the initial risk imminent data based on preset coefficients to obtain the risk imminent assessment result.

[0049] Specifically, based on the maximum field length current_max_length and the maximum field definition length defined_max_length in the statistical fingerprint data, a division operation (current_max_length / defined_max_length) is performed to obtain the initial risk proximity data P_score. The initial risk proximity data P_score is a basic indicator. The nonlinear function F(x) is used to amplify the initial risk proximity data P_score, so that the field closer to 1 has a sharply increased risk score. The nonlinear function F(x) is shown in formula (2): F(x) = (2) Where b>1, such as b=3, b represents the nonlinear amplification factor (preset factor). F(x) adopts a power function amplification mechanism. When b=3, as P_score increases from 0.9 to 0.95, the risk sensitivity increases by about 1.5 times, which is consistent with the marginal effect characteristic of risk growth. This parameter was optimized through grid search and achieved the best F1 score on the validation set.

[0050] In one example, the semantic analysis results include growth potential data; based on the semantic analysis results, the growth potential of the field to be analyzed is evaluated to obtain the growth potential evaluation result, including: mapping the growth potential data with first preset mapping data to obtain the growth potential evaluation result, wherein the first preset mapping data includes the relationship between the growth potential data and a first preset value.

[0051] Specifically, mapping the growth potential data returned by AI (such as fixed, low, medium, hierarchical, and unbounded text) to numerical values ​​is a semantic-based prior risk assessment. For example, fixed values ​​(such as UUID (Universally Unique Identifier), ID card number): 0.1 (first preset value), low values ​​(such as name, status code): 0.3 (first preset value), medium values ​​(such as address, multi-value list): 0.7 (first preset value), hierarchical values ​​(such as org_path): 0.9 (first preset value), and unbounded text (such as user comments): 1.0 (first preset value).

[0052] In one example, statistical fingerprint data includes field data types, and semantic analysis results include content pattern data. Based on the semantic analysis results and statistical fingerprint data, pattern anomaly assessment is performed on the field to be analyzed to obtain pattern anomaly assessment results. This includes: when the field data type is a variable length type, and the content pattern data includes any one of comma-separated data, structured data, and single-value data, mapping processing is performed based on second preset mapping data to obtain pattern anomaly assessment results. The second preset mapping data includes the relationship between content pattern data and second preset values.

[0053] Specifically, the degree of mismatch between the content pattern data (content_pattern) identified by AI and the field data type (data_type). When the field data type is VARCHAR (variable length), and the content pattern data (content_pattern) includes any of the following: comma-separated list (comma-separated data), JSON (structured data), or single-value data, mapping processing is performed based on the second preset mapping data to obtain the pattern anomaly evaluation result. For example: comma-separated list stored in VARCHAR: high score (0.9, second preset value). This is a typical anti-paradigm design with extremely high risk. JSON stored in VARCHAR: high score (0.8, second preset value). Single-value data stored in VARCHAR: low score (0.1, second preset value).

[0054] In one example, the statistical fingerprint data includes field length distribution data and the maximum length defined for the field. Based on the statistical fingerprint data, the distribution skewness of the field to be analyzed is evaluated to obtain the distribution skewness evaluation result, including: determining the field skewness data based on the field length distribution data, the maximum length defined for the field, and a preset value; and obtaining the distribution skewness evaluation result based on the field skewness data, a preset threshold, and a third preset mapping data, wherein the third preset mapping data includes the relationship between the field skewness data, the preset threshold, and the third preset value.

[0055] Specifically, the length distribution histogram is analyzed. If the length distribution is severely "right-skewed," meaning a large amount of data is concentrated in the interval close to the maximum length, the score is high. For example, if the proportion of data with length > 0.8 * defined_max_length, where 0.8 represents the preset value) exceeds 20% (the preset threshold), then the D_score is high (0.9, the third preset value).

[0056] In one example, the database risk assessment method also includes generating a risk assessment report based on the risk assessment results, semantic analysis results, risk imminent assessment results, growth potential assessment results, pattern anomaly assessment results, and distribution skew assessment results.

[0057] Specifically, based on the risk assessment results (risk assessment scores), fields can be classified into risk levels to obtain the risk level corresponding to each field. For example, 80 points corresponds to medium risk. The FinalRiskScores of all fields to be analyzed are sorted to generate a detailed risk assessment report (e.g., ...). Figure 5 (As shown in Table 1). The risk assessment report not only includes a ranking list, but more importantly, it provides complete decision support information for each high-risk field. Table 1 Risk Assessment Report

[0058] In another example, semantic recognition based on regular expressions and rule bases is adopted, and a large rule base is predefined. For example, if the field name contains id, uuid, code and the data content conforms to a specific format, it is determined to be of the "fixed length" type; if the field name contains desc, comment, remark, it is determined to be of the "dynamic content" type. However, this approach has the following drawbacks: (1) High maintenance cost: The rule base needs to be continuously maintained and expanded manually, which cannot adapt to the new naming habits that emerge in business development. (2) Poor generalization ability: For names that do not conform to the preset rules (such as using pinyin or custom abbreviations), this solution is completely ineffective. (3) Lack of contextual understanding: It cannot handle complex or ambiguous comments, such as a comment that "stores the user's JSON configuration, including address and name", which is difficult for simple rules to determine its core growth point.

[0059] The AI ​​large-scale model solution adopted in this application has powerful generalization and contextual understanding capabilities. It does not require preset rules, can understand various natural language descriptions and irregular naming, and automatically perform induction and reasoning. Its robustness and intelligence level far exceed those of solutions based on fixed rules.

[0060] Figure 6 A block diagram of a database risk assessment system provided for another embodiment of this application, such as... Figure 6 As shown, the database risk assessment system 600 includes: an acquisition module 610 (data acquisition module), a first analysis module 620 (automated data analysis module), a second analysis module 630 (AI semantic analysis module), and an assessment module 640 (risk fusion assessment module and report generation module).

[0061] The acquisition module 610 (data acquisition module) is used to acquire the fields to be analyzed from the data table to be evaluated in the target database. The first analysis module 620 (automated data profiling module) is used to perform data feature analysis on the fields to be analyzed, obtaining statistical fingerprint data, which includes metadata, field length statistical features, and field content statistical features. The second analysis module 630 (AI semantic analysis module) is used to perform semantic analysis based on the fields to be analyzed and the statistical fingerprint data, obtaining semantic analysis results. The evaluation module 640 (risk fusion assessment module, report generation module) is used to perform risk assessment based on the statistical fingerprint data and the semantic analysis results, obtaining the risk assessment results for the target database.

[0062] Figure 7 Another database risk assessment method flowchart provided for the implementation of this application; such as Figure 7 As shown, the database risk assessment methods include S701-S705.

[0063] S701, data acquisition module.

[0064] For example, the data acquisition module connects to the target database configured by the user. It acquires and caches metadata and sample data for all tables to be scanned (data tables to be evaluated). The metadata includes: field name (column_name), field data type (data_type), maximum length of field definition (defined_max_length), and field comment data (column_comment). For each field, a certain amount of sample data is collected for statistical analysis.

[0065] S702, Automated Data Profiling Module.

[0066] For example, the automated data profiling module traverses all data tables to be evaluated and creates a comprehensive "data profile" for each field (column). This prepares rich and accurate "ingredients" for AI analysis. For each field in the data tables to be evaluated in the target database, "statistical fingerprint data" is calculated and cached.

[0067] S703, AI semantic analysis module.

[0068] For example, the AI ​​semantic analysis module, based on the generated "statistical fingerprint data", calls the large language model service to dynamically construct a rich context for each field to be analyzed in a programmatic manner, and calls the large model API (Application Programming Interface) to perform batch analysis to obtain semantic analysis results.

[0069] S704, Risk Integration Assessment Module.

[0070] For example, the risk fusion assessment module integrates "statistical fingerprint data" and "semantic analysis results" to calculate the final risk score (risk assessment result). The weighting design here must be logically sound.

[0071] S705, Report Generation Module.

[0072] For example, the report generation module sorts the FinalRiskScore of all fields to generate a detailed risk assessment report and visualizes the risk assessment report.

[0073] Figure 8 Another database risk assessment method flowchart provided for the implementation of this application; such as Figure 8 As shown, the database risk assessment methods include S801-S809.

[0074] S801, obtain metadata and sample data.

[0075] For example, the data acquisition module connects to the target database configured by the user. It acquires and caches metadata and sample data for all tables to be scanned (data tables to be evaluated). The metadata includes: field name (column_name), field data type (data_type), maximum length of field definition (defined_max_length), and field comment data (column_comment). For each field, a certain amount of sample data is collected for statistical analysis.

[0076] S802 calculates and statistically analyzes fingerprint data.

[0077] For example, the automated data profiling module traverses all data tables to be evaluated and creates a comprehensive "data profile" for each field (column). This prepares rich and accurate "ingredients" for AI analysis. For each field in the data tables to be evaluated in the target database, it calculates and caches "statistical fingerprint" data (including metadata, field length statistics, and field content statistics).

[0078] S803 generates batch tasks.

[0079] For example, the AI ​​semantic analysis module generates batch tasks based on the generated "statistical fingerprint data", calls the large language model service, dynamically constructs a rich context for each field to be analyzed in a programmatic manner, and calls the large model API to perform batch analysis to obtain semantic analysis results.

[0080] S804, Construct prompt word data.

[0081] For example, the AI ​​semantic analysis module automatically assembles a structured Prompt based on each field of the batch task, and inputs the assembled Prompt data into the large language model.

[0082] S805, Large Language Model Reasoning and Analysis.

[0083] For example, a large language model performs reasoning on prompt word data to obtain semantic analysis results.

[0084] S806 stores the semantic analysis results.

[0085] S807, Risk Integration Assessment.

[0086] For example, the risk fusion assessment module integrates "statistical fingerprint data" and "semantic analysis results" to conduct risk imminent assessment, growth potential assessment, pattern anomaly assessment, and distribution skewness assessment, and calculates the final risk score (risk assessment result) using weighted averages. The weighting design here must be logically sound.

[0087] S808, Risk Assessment Results.

[0088] For example, output the risk assessment results.

[0089] S809, Visualization of Risk Assessment Reports.

[0090] For example, the report generation module sorts the FinalRiskScore of all fields to generate a detailed risk assessment report and visualizes the risk assessment report.

[0091] The database risk assessment method and system proposed in this application adopt the following: (1) Automated process architecture: An end-to-end automated system from database connection, automatic profiling, AI analysis to risk assessment is proposed, rather than a fragmented method or tool. (2) "Statistical fingerprint" data profiling method: Instead of looking at the maximum length, it creatively introduces multi-dimensional statistical features such as length distribution histogram, unique value ratio, and specific character frequency, providing AI with richer and machine-understandable context. (3) Programmatic AI semantic feature engineering: The AI ​​large model is called as an internal "microservice" of the system. By programmatically and batch-dynamically constructing a Prompt containing statistical fingerprints, efficient and deep semantic analysis of the entire database is achieved. (4) Logically rigorous multi-factor risk fusion model: An assessment model containing four core dimensions including proximity, AI semantic growth potential, pattern anomaly, and data distribution is established, and a reasonable weight design logic is given, so that the risk assessment has both data-driven objectivity and knowledge-driven predictability.

[0092] Figure 9 A block diagram of an electronic device provided for another embodiment of this application.

[0093] Another embodiment of this application provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0094] like Figure 9 As shown, for ease of understanding, an embodiment of this application illustrates a specific electronic device 900.

[0095] Electronic device 900 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 900 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0096] like Figure 9As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. The RAM 903 may also store various programs and data required for the operation of the electronic device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0097] Multiple components in electronic device 900 are connected to input / output (I / O) interface 905. These components include: input unit 906, such as a keyboard or mouse; output unit 907, such as various types of displays or speakers; storage unit 908, such as a hard disk or optical disk; and communication unit 909, such as a network interface card (NIC), modem, or wireless transceiver. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods described above. For example, in some embodiments, any one or more of the various methods described above can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of any one or more of the various methods described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform any one or more of the various methods described above by any other suitable means (e.g., by means of firmware).

[0099] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0100] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0101] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0102] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0104] Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this application can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this application, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly and specifically defined in the embodiments.

[0105] In this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0106] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0107] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A database risk assessment method, characterized in that, The method includes: Obtain the fields to be analyzed from the data table to be evaluated in the target database; Data feature analysis is performed on the field to be analyzed to obtain statistical fingerprint data, wherein the statistical fingerprint data includes metadata, field length statistical features and field content statistical features; Semantic analysis is performed based on the field to be analyzed and the statistical fingerprint data to obtain the semantic analysis results; Risk assessment is performed based on the statistical fingerprint data and the semantic analysis results to obtain the risk assessment results for the target database.

2. The method according to claim 1, characterized in that, The field to be analyzed includes multiple fields; the step of performing data feature analysis on the field to be analyzed to obtain statistical fingerprint data includes: Perform data feature analysis on each field to obtain the field name, field data type, maximum field definition length, and field comment data corresponding to each field, which are used as the metadata corresponding to each field. Data feature analysis is performed on each field to obtain the maximum used length, average length, standard deviation, and distribution data of each field, which are used as the statistical features of the field length for each field. Data feature analysis is performed on each field to obtain the field non-empty rate, field unique value ratio, and field specific character occurrence frequency for each field, which serve as the statistical features of the field content for each field.

3. The method according to claim 1, characterized in that, The semantic analysis based on the field to be analyzed and the statistical fingerprint data, to obtain the semantic analysis results, includes: Obtain the sample data corresponding to the field to be analyzed from the data table to be evaluated; Based on the sample data and the statistical fingerprint data, prompt word data is obtained; Based on the prompt word data, semantic analysis is performed on each field to obtain semantic type data, content pattern data, growth potential data and inference data corresponding to each field; The semantic analysis results are obtained based on the semantic type data, the content pattern data, the growth potential data, and the reasoning data.

4. The method according to claim 1, characterized in that, The risk assessment based on the statistical fingerprint data and the semantic analysis results, to obtain the risk assessment results for the target database, includes: Based on the statistical fingerprint data, a risk proximity assessment is performed on the field to be analyzed to obtain the risk proximity assessment result; Based on the semantic analysis results, the growth potential of the field to be analyzed is evaluated to obtain the growth potential evaluation results. Based on the semantic analysis results and the statistical fingerprint data, the field to be analyzed is evaluated for pattern anomalies to obtain the pattern anomaly evaluation results. Based on the statistical fingerprint data, the distribution skewness of the field to be analyzed is evaluated to obtain the distribution skewness evaluation result. Based on the growth potential coefficient, pattern anomaly coefficient, and distribution skewness coefficient, the risk imminent assessment results, the growth potential assessment results, the pattern anomaly assessment results, and the distribution skewness assessment results are weighted to obtain the risk assessment results of the target database.

5. The method according to claim 4, characterized in that, The statistical fingerprint data includes the maximum length of the field definition and the maximum length of the field usage; the risk proximity assessment of the field to be analyzed based on the statistical fingerprint data, to obtain the risk proximity assessment result, includes: Based on the maximum length defined in the field and the maximum usage length of the field, determine the initial risk-approaching data; The initial risk proximity data is processed nonlinearly based on preset coefficients to obtain the risk proximity assessment result.

6. The method according to claim 4, characterized in that, The semantic analysis results include growth potential data; the growth potential assessment of the field to be analyzed based on the semantic analysis results, to obtain the growth potential assessment results, includes: The growth potential data is mapped to the first preset mapping data to obtain the growth potential assessment result. The first preset mapping data includes the relationship between the growth potential data and the first preset value.

7. The method according to claim 4, characterized in that, The statistical fingerprint data includes field data types, and the semantic analysis results include content pattern data; the pattern anomaly assessment of the field to be analyzed based on the semantic analysis results and the statistical fingerprint data, to obtain the pattern anomaly assessment results, includes: When the field data type is a variable length type, and the content pattern data includes any one of comma-separated data, structured data, and single-value data, mapping processing is performed based on the second preset mapping data to obtain the pattern anomaly evaluation result. The second preset mapping data includes the relationship between the content pattern data and the second preset value.

8. The method according to claim 4, characterized in that, The statistical fingerprint data includes field length distribution data and the maximum length defined for each field; the distribution skewness assessment of the field to be analyzed based on the statistical fingerprint data, to obtain the distribution skewness assessment result, includes: Based on the field length distribution data, the maximum defined length of the field, and the preset value, the field skew data is determined; Based on the field skew data, the preset threshold, and the third preset mapping data, the distribution skew evaluation result is obtained, wherein the third preset mapping data includes the relationship between the field skew data, the preset threshold, and the third preset value.

9. The method according to claim 4, characterized in that, The method further includes: A risk assessment report is generated based on the risk assessment results, the semantic analysis results, the risk imminent assessment results, the growth potential assessment results, the pattern anomaly assessment results, and the distribution skewness assessment results.

10. A database risk assessment system, characterized in that, The system includes: The acquisition module is used to obtain the fields to be analyzed from the data table to be evaluated in the target database; The first analysis module is used to perform data feature analysis on the field to be analyzed to obtain statistical fingerprint data, wherein the statistical fingerprint data includes metadata, field length statistical features and field content statistical features; The second analysis module is used to perform semantic analysis based on the field to be analyzed and the statistical fingerprint data to obtain semantic analysis results. The assessment module is used to perform risk assessment based on the statistical fingerprint data and the semantic analysis results to obtain the risk assessment results of the target database.

Images