Field grading method and device based on dynamic rule, equipment and storage medium

Abstract

The invention relates to the technical field of data processing and analysis and the technical field of financial science and technology, and discloses a field grading method based on a dynamic rule, and the method comprises the steps: obtaining field data and corresponding metadata, analyzing the metadata according to a preset matching rule, and screening out target metadata; performing logical operation processing on the target metadata to obtain a metadata processing result; performing rapid feature detection on the field data, setting a processing priority of a preset recognizer according to a detection result, processing the field data in sequence, and generating a field data recognition result; judging whether the recognition result is null or not, and if yes, generating a field grading result according to the metadata processing result; and if not, generating a field grading result according to the field data identification result. By screening the data, the processing data volume is reduced, and the processing rule can be adjusted according to the data scale and characteristics. Meanwhile, important data are processed preferentially, performance reduction is avoided, and therefore the overall data processing efficiency and precision are improved.

Description

Technical Field

[0001] The present invention relates to the fields of data processing and analysis technology and financial technology technology, and in particular to a field grading method, device, equipment and storage medium based on dynamic rules. Background Art

[0002] In the information and data era, the financial industry relies heavily on database systems to manage and process massive amounts of customer data, transaction data, financial records, etc. As the types and amounts of financial data continue to grow, the complexity of the data has also increased significantly. Therefore, it is particularly important to effectively classify and grade this data, which is directly related to the efficiency of business processes and the accuracy of decision-making.

[0003] Traditionally, financial institutions often use static rules to classify data. These static rules are usually set manually based on business needs and industry standards. For example, banks may classify customer data based on parameters such as account type and transaction amount. However, static rules are inefficient and inflexible when dealing with rapidly changing financial markets, especially when faced with complex and unstructured data (such as customer behavior data, market conditions, etc.). Such fixed rules are difficult to capture dynamic features in the data.

[0004] One of the core needs of the financial industry is to respond to market changes and avoid risks in real time. For example, financial institutions may need to dynamically adjust their risk control models or product recommendation strategies based on the latest market trends or customer behavior data. Using static rules cannot reflect these changes in a timely manner, which in turn affects the accuracy of decision-making.

[0005] Although dynamic rules have many advantages in financial data classification, they also face some challenges. Financial data is large and diverse, and frequent rule adjustments may increase the system load, especially when the entire data set needs to be recalculated. This will not only reduce the overall performance of the system, but also may affect the accuracy of the classification results. Especially when processing big data in real time, the pressure on computing resources may lead to a decrease in the efficiency and accuracy of classification. Summary of the invention

[0006] The main purpose of the present invention is to provide a field classification method, device, equipment and storage medium based on dynamic rules, aiming to solve the technical problems of low classification efficiency and insufficient accuracy of existing field classification methods.

[0007] To achieve the above object, the present invention provides a field classification method based on dynamic rules, and the field classification method based on dynamic rules comprises the following steps:

[0008] Acquire field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data;

[0009] Analyze the metadata according to preset matching rules, filter out target metadata, and perform logical operation processing on the target metadata according to preset logical processing rules to obtain metadata processing results;

[0010] Performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and using the preset identifiers to identify the field data in sequence according to the processing priority to obtain a field data identification result;

[0011] If the field data identification result is empty, generating a field classification result according to the metadata processing result;

[0012] If the field data recognition result is not empty, a field classification result is generated according to the field data recognition result.

[0013] Optionally, the field data is identified by using the preset identifiers in sequence according to the processing priority to obtain a field data identification result, including:

[0014] Using the preset identifiers in sequence to identify the field data according to the processing priority, obtaining an identification result corresponding to each preset identifier, wherein the preset identifiers include a group sensitive identifier, a sample rule identifier, an intelligent model identifier, and a regular rule identifier;

[0015] The proportion of the recognition results corresponding to each preset identifier is calculated by a proportion formula, and the recognition result with the highest proportion is used as the field data recognition result.

[0016] Optionally, the field data is identified using the preset identifiers in sequence according to the processing priority to obtain an identification result corresponding to each preset identifier, including:

[0017] Using the preset identifiers to identify the field data in sequence according to the processing priority;

[0018] When the sample rule identifier is used to identify the field data, a preset sample rule is loaded through the sample rule identifier;

[0019] The sample data set of the preset sample rule is traversed to perform a complete match judgment with the field data. If the complete match judgment is successful, the recognition task of the sample rule identifier is ended to obtain the recognition result of the sample rule identifier.

[0020] Optionally, the field data is identified using the preset identifiers in sequence according to the processing priority to obtain an identification result corresponding to each preset identifier, including:

[0021] Using the preset identifiers to identify the field data in sequence according to the processing priority;

[0022] When the intelligent model identifier is used to identify the field data, the intelligent model identifier uses a built-in preset naming recognition rule to identify the field data to obtain a recognition result of the intelligent model identifier.

[0023] Optionally, identifying the field data by applying a built-in preset naming recognition rule by the intelligent model identifier includes:

[0024] Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain a newly added recognition rule;

[0025] The field data is identified by using the preset naming identification rules and the newly added identification rules built into the intelligent identifier application.

[0026] Optionally, performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and sequentially using the preset identifiers to identify the field data according to the processing priority, includes:

[0027] Detecting whether the first preset number of characters of the field data have JSON text features;

[0028] If there is a JSON text feature, convert the field data into a JSON object, perform rapid feature detection on the JSON object, set a processing priority of a preset identifier according to the detection result, and use the preset identifier to identify the JSON object in sequence according to the processing priority;

[0029] If there is no JSON text feature, extract the first second preset number of phrases with the highest frequency in the field data, perform fast feature detection on the second preset number of phrases, set the processing priority of the preset identifier according to the detection result, and use the preset identifier to identify the second preset number of phrases in sequence according to the processing priority.

[0030] Optionally, after generating a field classification result according to the metadata processing result, the method further includes:

[0031] Acquire user feedback information, and adjust the preset matching rules and preset logical processing rules according to the user feedback information.

[0032] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a field grading device based on dynamic rules, and the field grading device based on dynamic rules comprises:

[0033] A data acquisition module, which acquires field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data;

[0034] The metadata analysis and processing module analyzes the metadata according to the preset matching rules, selects the target metadata, performs logical operation processing on the target metadata according to the preset logical processing rules, and obtains the metadata processing result;

[0035] A field data analysis module performs rapid feature detection on the field data, sets a processing priority of a preset identifier according to the detection result, and sequentially uses the preset identifier to identify the field data according to the processing priority to obtain a field data identification result;

[0036] The grading decision module generates a field grading result according to the metadata processing result if the field data identification result is empty; and generates a field grading result according to the field data identification result if the field data identification result is not empty.

[0037] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a dynamic rule-based field grading device, which includes a memory, a processor, and a dynamic rule-based field grading program stored in the memory and executable on the processor, and the dynamic rule-based field grading program, when executed by the processor, implements the steps of the dynamic rule-based field grading method as described above.

[0038] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which a dynamic rule-based field grading program is stored, and when the dynamic rule-based field grading program is executed by a processor, the steps of the dynamic rule-based field grading method as described above are implemented.

[0039] The present invention relates to a field classification method based on dynamic rules, which obtains field data and corresponding metadata, analyzes metadata according to preset matching rules, and screens out target metadata; performs logical operation processing on target metadata according to preset logical processing rules to obtain metadata processing results; performs rapid feature detection on field data, sets processing priorities of preset identifiers according to detection results, uses identifiers to process field data in sequence, and generates field data recognition results; determines whether the field data recognition results are empty, and if so, generates field classification results according to metadata processing results; if not, generates field classification results according to field data recognition results. The present invention reduces the amount of processed data by screening data, and can adjust processing rules according to data scale and characteristics. At the same time, important data is processed first to avoid performance degradation, thereby improving overall data processing efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a structural schematic diagram of a device hardware operating environment involved in an embodiment of a field grading device based on dynamic rules of the present invention;

[0041] Figure 2 This is a flow chart of a first embodiment of a field classification method based on dynamic rules according to the present invention;

[0042] Figure 3 It is a functional module diagram of a preferred embodiment of the field grading device based on dynamic rules of the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0045] It should be noted that in the information and data era, the financial industry is highly dependent on database systems to manage and process massive amounts of customer data, transaction data, financial records, etc. As the types and amounts of financial data continue to grow, the complexity of the data has also increased significantly. Therefore, it is particularly important to effectively classify and grade this data, which is directly related to the efficiency of business processes and the accuracy of decision-making.

[0046] Traditionally, financial institutions often use static rules to classify data. These static rules are usually set manually based on business needs and industry standards. For example, banks may classify customer data based on parameters such as account type and transaction amount. However, static rules are inefficient and inflexible when dealing with rapidly changing financial markets, especially when faced with complex and unstructured data (such as customer behavior data, market conditions, etc.). Such fixed rules are difficult to capture dynamic features in the data.

[0047] One of the core needs of the financial industry is to respond to market changes and avoid risks in real time. For example, financial institutions may need to dynamically adjust their risk control models or product recommendation strategies based on the latest market trends or customer behavior data. Using static rules cannot reflect these changes in a timely manner, which in turn affects the accuracy of decision-making.

[0048] Although dynamic rules have many advantages in financial data classification, they also face some challenges. Financial data is large and diverse, and frequent rule adjustments may increase the system load, especially when the entire data set needs to be recalculated. This will not only reduce the overall performance of the system, but also may affect the accuracy of the classification results. Especially when processing big data in real time, the pressure on computing resources may lead to a decrease in the efficiency and accuracy of classification.

[0049] Based on the above defects, the present invention provides a field classification device based on dynamic rules, referring to Figure 1 , Figure 1 It is a structural schematic diagram of the device hardware operating environment involved in the embodiment of the field grading device based on dynamic rules of the present invention.

[0050] like Figure 1 As shown, the field grading device based on dynamic rules may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1The hardware structure of the dynamic rule-based field grading device shown in the figure does not constitute a limitation on the dynamic rule-based field grading device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a field classification program based on dynamic rules. The operating system is a program for managing and controlling field classification devices and software resources based on dynamic rules, and supports the operation of the network communication module, the user interface module, the field classification program based on dynamic rules, and other programs or software; the network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0053] exist Figure 1 In the hardware structure of the field grading device based on dynamic rules shown in the figure, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the client and perform data communication with the client; the processor 1001 can call the field grading program based on dynamic rules stored in the memory 1005, and perform the following operations:

[0054] Acquire field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data;

[0055] Analyze the metadata according to preset matching rules, filter out target metadata, and perform logical operation processing on the target metadata according to preset logical processing rules to obtain metadata processing results;

[0056] Performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and using the preset identifiers to identify the field data in sequence according to the processing priority to obtain a field data identification result;

[0057] If the field data identification result is empty, generating a field classification result according to the metadata processing result;

[0058] If the field data recognition result is not empty, a field classification result is generated according to the field data recognition result.

[0059] Further, the field data is identified by using the preset identifiers in sequence according to the processing priority to obtain field data identification results, including:

[0060] Using the preset identifiers in sequence to identify the field data according to the processing priority, obtaining an identification result corresponding to each preset identifier, wherein the preset identifiers include a group sensitive identifier, a sample rule identifier, an intelligent model identifier, and a regular rule identifier;

[0061] The proportion of the recognition results corresponding to each preset identifier is calculated by a proportion formula, and the recognition result with the highest proportion is used as the field data recognition result.

[0062] Further, the field data is identified by using the preset identifiers in sequence according to the processing priority to obtain the identification result corresponding to each preset identifier, including:

[0063] Using the preset identifiers to identify the field data in sequence according to the processing priority;

[0064] When the sample rule identifier is used to identify the field data, a preset sample rule is loaded through the sample rule identifier;

[0065] The sample data set of the preset sample rule is traversed to perform a complete match judgment with the field data. If the complete match judgment is successful, the recognition task of the sample rule identifier is ended to obtain the recognition result of the sample rule identifier.

[0066] Further, the field data is identified by using the preset identifiers in sequence according to the processing priority to obtain the identification result corresponding to each preset identifier, including:

[0067] Using the preset identifiers to identify the field data in sequence according to the processing priority;

[0068] When the intelligent model identifier is used to identify the field data, the intelligent model identifier uses a built-in preset naming recognition rule to identify the field data to obtain a recognition result of the intelligent model identifier.

[0069] Furthermore, the field data is identified by applying a built-in preset naming identification rule by the intelligent model identifier, including:

[0070] Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain a newly added recognition rule;

[0071] The field data is identified by using the preset naming identification rules and the newly added identification rules built into the intelligent identifier application.

[0072] Further, performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and sequentially using the preset identifiers to identify the field data according to the processing priority, including:

[0073] Detecting whether the first preset number of characters of the field data have JSON text features;

[0074] If there is a JSON text feature, convert the field data into a JSON object, perform rapid feature detection on the JSON object, set a processing priority of a preset identifier according to the detection result, and use the preset identifier to identify the JSON object in sequence according to the processing priority;

[0075] If there is no JSON text feature, extract the first second preset number of phrases with the highest frequency in the field data, perform fast feature detection on the second preset number of phrases, set the processing priority of the preset identifier according to the detection result, and use the preset identifier to identify the second preset number of phrases in sequence according to the processing priority.

[0076] Furthermore, after generating the field classification result according to the metadata processing result, the method further includes:

[0077] Acquire user feedback information, and adjust the preset matching rules and preset logical processing rules according to the user feedback information.

[0078] The specific implementation of the dynamic rule-based field grading device of the present invention is basically the same as the various embodiments of the dynamic rule-based field grading method described below, and will not be repeated here.

[0079] The present invention also provides a dynamic rule-based field grading method based on the above-mentioned dynamic rule-based field grading device.

[0080] Reference Figure 2 , Figure 2 It is a flowchart diagram of the first embodiment of the field classification method based on dynamic rules of the present invention.

[0081] The embodiment of the present invention provides an embodiment of a field classification method based on dynamic rules. It should be noted that although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different from that shown here.

[0082] In various embodiments of the dynamic rule-based field classification method, the execution subject is a dynamic rule-based field classification device.

[0083] The field grading method based on dynamic rules is applied to a field grading device based on dynamic rules. The field grading method based on dynamic rules includes:

[0084] Step S10, obtaining field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data;

[0085] In this embodiment, two key information are obtained from the data source: field data and corresponding metadata. Field data refers to the actual stored value in a database or data table (for example, a specific number, string, or date), while metadata describes the properties and structure of field data, such as field name, data type, length, whether it is empty, etc. Metadata provides context and definition for field data, ensuring that the system can correctly interpret and process field data.

[0086] The system first extracts the actual field data from the database or other data storage location, and retrieves the metadata related to these fields. The system can obtain this data through SQL queries, API calls, etc. Metadata usually includes field name, field type, constraints (such as non-null, uniqueness, etc.), and provides a basis for subsequent data processing.

[0087] The role of metadata is to provide definitions for field data. It describes the structure of the field (such as whether it is a nested structure or a simple field), data type (such as string, integer, date, etc.), and constraints (such as whether the field is required, the maximum and minimum length of the field value, or uniqueness requirements). These definitions ensure that the storage, retrieval, and processing of field data in the system are compliant and consistent.

[0088] In the system, metadata is usually defined in advance and stored in a metadata table or metadata management system. When the system obtains field data, it automatically retrieves and matches the corresponding metadata to ensure that the field data conforms to the metadata definition. For example, if the metadata specifies that a field is an integer type, the system will perform type verification on the field during the data processing phase to ensure that its data is in integer format. In addition, the system will perform compliance checks on field data based on the constraints in the metadata (such as non-empty and uniqueness) to avoid the entry of illegal data.

[0089] By associating field data with corresponding metadata, the system can ensure that it has clear context information when processing field data, avoiding processing errors caused by data type mismatch or incorrect format. Metadata defines the structure, type, and constraints of field data, which helps improve the accuracy and consistency of data processing and ensure that data can be correctly interpreted and processed at all stages. At the same time, through the constraints of metadata, the system can realize automatic verification of data, improve data quality, and reduce the storage and transmission of erroneous data.

[0090] Step S20, analyzing the metadata according to a preset matching rule, filtering out target metadata, and performing logical operation processing on the target metadata according to a preset logical processing rule to obtain a metadata processing result;

[0091] In this embodiment, the system automatically analyzes metadata based on a set of predefined matching rules. These matching rules are defined based on business requirements and metadata attributes, such as whether the field name contains specific keywords, whether the data type meets expectations, etc. The matching rules can quickly filter out metadata that does not meet the requirements to improve processing efficiency.

[0092] First, the system extracts all relevant metadata from the metadata store. Then, the system applies pre-defined matching rules to analyze these metadata one by one. For example, the matching rule can be that the field name must contain the word "Date" or the field type must be an integer. The system analyzes the metadata one by one through string matching, type checking, and other methods to filter out metadata that meets the conditions.

[0093] After filtering out the target metadata, the system processes the data according to predefined logical processing rules. The logical processing rules can include logical operations such as "and", "or", and "not" to combine or exclude specific conditions and finally generate metadata processing results.

[0094] The target metadata processed by logical operations generates the final recognition result. This recognition result is used in the subsequent field classification process to determine how the field is classified and graded.

[0095] A specific implementation method may be: first, obtain the required field data and corresponding metadata from the database. The system performs a preliminary analysis of the metadata through the set matching rules, and filters out metadata that meets specific conditions. For example, the system may set a rule to filter out all metadata that contain "date" in the field name.

[0096] Once the target metadata is filtered out, the system will apply the preset logical processing rules to further calculate and process the data. Logical processing rules can be set according to business needs, such as combining multiple conditions for filtering or classification operations. Through this logical processing, the system will eventually generate a metadata processing result, which will be used to guide the classification and grading of fields.

[0097] It should be noted that matching rules and logical processing rules can be changed based on specific needs. For example, changes in business scenarios can affect matching rules and logical processing rules. For example, the priority, content requirements, or structure of certain data fields may change due to adjustments in business needs, resulting in rule changes. If a business requirement changes from "date field" to "timestamp field", matching rules and logical processing rules may need to be adjusted to recognize and process the new data format.

[0098] When the data scale changes significantly (for example, upgrading from small-scale data processing to big data processing), the rules may need to be changed to optimize system performance. Similarly, different data types will also lead to rule adjustments. For example, on large data sets, matching rules can be simplified or restricted to reduce processing time; and when processing structured or semi-structured data, logical processing rules may need to adapt to different formats (such as JSON, XML, etc.).

[0099] In order to improve the performance and response speed of the system, the rules may need to be dynamically adjusted or simplified. For example, when the system load is large, some logical processing rules can be simplified to reduce the impact of complex operations on performance. For example, under high load, the system may need to reduce complex multi-conditional logical operations and use simpler rules for processing to improve efficiency.

[0100] Example description:

[0101] Assume that there are multiple tables in the database, each containing many fields. The system first obtains the field data and metadata, such as the table name, field name, field type, and description. Then, the system uses the preset matching rules to analyze the metadata. For example, filter out all field data that contain "price" in the field name, and the data type of these fields is numeric.

[0102] After filtering out the target metadata that meets the conditions, the system further processes it according to the preset logical processing rules. Assume that the logical rule is: if the field name contains "price" and the field type is numeric, the system will identify these fields as "price fields" and give them specific labels or categories. In the final recognition results, the system identifies these fields as pricing-related fields, which facilitates subsequent data processing or analysis tasks.

[0103] Through the above steps, the automatic identification and classification of database fields are realized, which not only improves the efficiency of data processing, but also reduces the necessity of manual intervention and ensures the consistency and accuracy of field classification. The mechanism of processing data in stages and setting priorities can ensure that when dynamic rules are adjusted, the system will not immediately affect the overall performance due to rule changes. When dynamic rules are adjusted, the system will not immediately reprocess all data, but will initially screen out qualified data to reduce the processing volume, thereby ensuring that the overall performance of the system is not affected.

[0104] Step S30, performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and using the preset identifiers to identify the field data in sequence according to the processing priority to obtain a field data identification result;

[0105] In this embodiment, the purpose of fast feature detection on field data is to quickly analyze the features of field data to identify key attributes that affect data processing. These features may include data type (such as integer, string), data format (such as date, timestamp), data length, data content mode (such as whether it contains special characters), etc.

[0106] Based on the result of feature detection, the system will assign processing priorities to different preset recognizers. The priority determines the order in which the recognizers process field data. For example, some recognizers may be more suitable for processing numeric data, while others are more suitable for processing text data. The system will prioritize the most suitable recognizer based on the degree of feature matching.

[0107] According to the set priority order, different identifiers are called one by one to identify the field data. Each identifier will analyze and identify the field data based on its specific rules and algorithms, and generate corresponding recognition results.

[0108] After each identifier has identified the field data, the system will collect and analyze the generated identification results. If the results of all identifiers are empty or fail to identify valid information, the system will determine that the field data identification result is empty and may turn to the metadata-based results for classification. If the identification result is not empty, the system will directly use the field data identification result for classification.

[0109] A specific implementation method may be: first, feature detection is performed on the field data. For example, the system may detect whether the type of the field data is a numeric type, whether the content conforms to the date format, or whether the field content contains specific keywords. Based on the detection results, the system assigns priorities to different identifiers.

[0110] Assuming the system detects that the field data is in date format, the date recognizer is called first to recognize the field data. After the recognizer processes the field data, the system analyzes the generated recognition result. If the recognition result is empty, the system may re-evaluate the feature detection result or call other recognizers until a valid recognition result is generated.

[0111] Example description:

[0112] Assuming there is a field data "2023-08-13", the system first confirms that the field conforms to the date format through feature detection. Based on this result, the system sets the date recognizer to the highest priority and calls it first to process the data. If the date recognizer successfully recognizes that this is a date field, the system will generate a recognition result, such as "date field", and continue the classification process.

[0113] If the field data is "null" or the recognizer fails to generate a valid result, the system will determine that the recognition result is empty. At this time, the system may call other recognizers or perform field classification based on the recognition results of metadata.

[0114] Through the above steps, multiple identifiers are flexibly used to process and identify field data, thereby ensuring the accuracy and effectiveness of classification and grading. By processing data and logical operation rules in stages, the grading strategy can be flexibly adjusted according to the data scale, and important data can be processed first to ensure the efficiency and accuracy of grading in a big data environment. If the data scale is large, the system can be adjusted according to different data scales. For example, when the data scale is small, the full amount of data can be processed directly; when the data scale is large, the system can filter out the most important or high-priority data according to the matching rules for priority processing, and delay the processing of secondary data.

[0115] Step S40: if the field data identification result is empty, a field classification result is generated according to the metadata processing result; if the field data identification result is not empty, a field classification result is generated according to the field data identification result.

[0116] In this embodiment, if the recognition result of the field data is empty, the system will rely on the previous metadata processing results to generate the classification results of the field. The metadata processing results may be obtained based on the metadata attributes such as the name, type, description, etc. of the field through preset logical processing rules. If the field data cannot provide enough information for classification, the system will use these metadata processing results for final classification and classification.

[0117] If the field data recognition result is not empty, that is, the recognizer successfully recognizes valid information, then the system will directly use the recognition results of these field data to generate field classification results. This means that the field data itself provides enough information for classification and grading without relying on metadata.

[0118] A specific implementation method may be: first try to use the identifier to identify the field data and generate an identification result. If the field data identification result is empty (for example, the identifier cannot extract useful information from the field data), the system will use the identification result obtained by metadata analysis to generate the final field classification result.

[0119] If the field data recognition results are valid, the system will give priority to using these recognition results to generate grading results, because the field data directly reflects the actual content and characteristics of the data, which may be more accurate and reliable than relying solely on metadata analysis.

[0120] Example description:

[0121] Assume that the data of a field is empty or the content is ambiguous (such as "null"). In this case, the field data recognition result may be empty. The system will use metadata, such as the field name "birthdate", to identify and classify, and finally classify the field as a "date field".

[0122] On the contrary, if the field data is "2023-08-13" and the identifier successfully recognizes that this is a date field, the system will directly classify the field as a "date field" based on this result without referring to metadata.

[0123] Through the above steps, it is ensured that when the field data lacks information, it is still possible to rely on metadata for reasonable classification and grading. At the same time, when the field data is sufficient, the information of the data itself can be used first for more accurate classification.

[0124] The present invention relates to a field classification method based on dynamic rules, which obtains field data and corresponding metadata, analyzes metadata according to preset matching rules, and screens out target metadata; performs logical operation processing on target metadata according to preset logical processing rules to obtain metadata processing results; performs rapid feature detection on field data, sets processing priorities of preset identifiers according to detection results, uses identifiers to process field data in sequence, and generates field data recognition results; determines whether the field data recognition results are empty, and if so, generates field classification results according to metadata processing results; if not, generates field classification results according to field data recognition results. The present invention reduces the amount of processed data by screening data, and can adjust processing rules according to data scale and characteristics. At the same time, important data is processed first to avoid performance degradation, thereby improving overall data processing efficiency and accuracy.

[0125] Furthermore, a second embodiment of the field classification method based on dynamic rules of the present invention is proposed. In the above step S20, the field data is identified in sequence using the preset identifier according to the processing priority to obtain the field data identification result, including:

[0126] Step a1, using the preset identifiers to identify the field data in sequence according to the processing priority, and obtaining the identification result corresponding to each preset identifier, wherein the preset identifiers include group sensitive identifiers, sample rule identifiers, intelligent model identifiers, and regular rule identifiers;

[0127] Step a2, calculating the proportion of the recognition results corresponding to each preset identifier by a proportion formula, and taking the recognition result with the highest proportion as the field data recognition result.

[0128] In this embodiment, when identifying field data, different identifiers are called in sequence to identify the field data according to the processing priorities set in advance. Each identifier analyzes the field data according to its field of expertise (such as group sensitive data identification, sample rule identification, intelligent model identification, and regular rule identifier, etc.) and generates its own identification results. Finally, the system summarizes these identification results and determines which identification result is the most credible or representative.

[0129] Identifiers are modules or tools in the system that are specifically used to identify and process specific types of field data. The preset types of identifiers include:

[0130] Group Sensitive Identifiers: Used to identify and classify data involving sensitive information, such as personally identifiable information or financial data.

[0131] Sample rule identifier: matches and identifies fields based on pre-defined samples and rules, especially suitable for processing data of known categories.

[0132] Intelligent pattern identifier: Uses machine learning models or artificial intelligence algorithms to process complex or unforeseen data types with high adaptability.

[0133] Regular rule identifier: Use regular expressions to match field data patterns. It is suitable for identifying data in specific formats, such as email addresses, phone numbers, etc.

[0134] Calculate the "share" or "weight" of each recognizer's recognition results for the field data. These shares may be based on the confidence of the recognizer, the coverage of the recognition results, and other factors. The share formula usually calculates the ratio of the number of times a recognizer hits a field to the total number of hits. The system uses the recognition result with the highest share as the final field data recognition result because it is considered to be the most representative or accurate.

[0135] A specific implementation method may be: first, according to the feature detection result of the field data, determine the order of calling different identifiers. Assuming that the system detects that a field may contain sensitive information, it may preferentially call the group sensitive identifier. If the identifier successfully identifies the field as sensitive information, the system will record the result and continue to call other identifiers to obtain more information.

[0136] When all identifiers have finished processing, the system will calculate the percentage of each recognition result. For example, the group sensitive identifier may identify 50% of the fields as sensitive information, while the sample rule identifier and the intelligent model identifier may identify 30% and 20% respectively. In the end, the system will select the recognition result with the highest percentage (i.e. the result of the group sensitive identifier) ​​as the final classification result of the field.

[0137] Example description:

[0138] Assume that a field "salary" appears in a database table. The system first detects that this field may involve sensitive information, so it first calls the group sensitive identifier. The group sensitive identifier recognizes that the field is related to financial data and identifies it as a sensitive information field.

[0139] Next, the system calls the sample rule identifier and finds that the field name and content meet the preset "salary" field rules. The intelligent model identifier further confirms that the field does belong to the salary category through a machine learning model.

[0140] After all identifiers have completed processing, the system calculates the proportion of each recognition result and finds that the recognition results of the group sensitive identifier have the highest proportion, so the field is ultimately classified as a "sensitive information field."

[0141] This embodiment processes and classifies complex field data by using multiple recognizers to work together, and ensures the accuracy and reliability of the recognition results.

[0142] Furthermore, a third embodiment of the field classification method based on dynamic rules of the present invention is proposed. In the above step a1, the field data is identified by using the preset identifiers in sequence according to the processing priority, and the identification result corresponding to each preset identifier is obtained, including:

[0143] Step b1, using the preset identifiers to identify the field data in sequence according to the processing priority;

[0144] Step b2, when the sample rule identifier is used to identify the field data, a preset sample rule is loaded through the sample rule identifier;

[0145] Step b3, traversing the sample data set of the preset sample rule and performing a complete match judgment with the field data. If the complete match judgment is successful, the recognition task of the sample rule identifier is ended and the recognition result of the sample rule identifier is obtained.

[0146] In this embodiment, according to the previously set processing priority, four identifiers (group sensitive identifier, sample rule identifier, intelligent model identifier and regular rule identifier) ​​are called in sequence to identify field data. Each identifier analyzes and processes the field data based on its specific identification rules and algorithms to generate corresponding identification results.

[0147] When processing field data, if the current identifier is a sample rule identifier, the system will match and identify the field data according to the preset sample rules. The sample rule identifier will load predefined sample rules, which may include common data patterns, field name and content matching rules, etc. The sample rule identifier will go through the data set of sample rules one by one and compare each rule with the field data. If the field data completely matches a rule (complete match judgment is successful), the field data is identified as a category that meets the rule.

[0148] If the field data completely matches a preset sample rule, the recognition task will be terminated immediately, and the sample rule recognizer will consider that the field has been successfully recognized and will not continue to process other rules or data. This result will be recorded and used in the subsequent field classification process.

[0149] A specific implementation method may be: first determine the processing priority of the identifier. For example, if the system detects that the field data may involve group sensitive information, the group sensitive identifier is called first. After that, the system will call other identifiers in sequence, including the sample rule identifier, the intelligent model identifier, and the regular rule identifier.

[0150] When the sample rule identifier is called, the system loads all preset rules that may match the current field data. The sample rule identifier traverses these rules and makes a complete match judgment with the field data one by one. For example, if the field name and content are completely consistent with the requirements of a sample rule, the match is considered successful. At this point, the sample rule identifier's task ends and the matching result is used as the final recognition result.

[0151] Example description:

[0152] Assuming there is a field "customer_email", the system first determines that the field may be related to group sensitive information and uses the group sensitive identifier to make a preliminary judgment. Next, the sample rule identifier is called and loads a set of sample rules related to emails, such as "the field name contains 'email' and the content conforms to the email format."

[0153] The sample rule recognizer traverses all preset rules and finds that the "customer_email" field name and content exactly matches one of the rules. After the match is successful, the sample rule recognizer immediately terminates further processing and marks the recognition result as "email field". This result will be used for grading or classifying the field in the subsequent steps.

[0154] Through the above steps, this embodiment ensures that the field data can be classified quickly and accurately, especially when there are clear matching rules. By terminating the recognition task in advance, the system can save processing resources and improve overall efficiency.

[0155] Furthermore, a fourth embodiment of the field classification method based on dynamic rules of the present invention is proposed. In the above step a1, the field data is identified by using the preset identifiers in sequence according to the processing priority, and the identification result corresponding to each preset identifier is obtained, including:

[0156] Step c1, using the preset identifiers to identify the field data in sequence according to the processing priority;

[0157] Step c2, when using the intelligent model identifier to identify the field data, the intelligent model identifier applies the built-in preset naming recognition rules to identify the field data to obtain the recognition result of the intelligent model identifier.

[0158] In this embodiment, the group sensitive identifier, sample rule identifier, intelligent model identifier and regular rule identifier are called in sequence according to the set priority order. These identifiers will analyze the field data according to their own rules and algorithms to generate corresponding recognition results.

[0159] When processing field data, if the intelligent model recognizer is currently called, the recognizer will be activated and apply its unique intelligent algorithms and rules to recognize the field data.

[0160] The smart model recognizer will use its built-in preset naming recognition rules to recognize field data. These naming rules are usually models generated based on a large amount of training data to accurately identify the type or classification of the field. For example, the smart model recognizer may use these rules to identify whether a field belongs to a category such as "name", "address" or "organization name".

[0161] A specific implementation method may be: first, different identifiers are called in sequence according to the characteristics and processing priority of the field data. Assuming that the system determines that the intelligent model identifier is one of the identifiers with a higher priority, when it is called, the intelligent model identifier will load and apply its built-in preset naming recognition rules.

[0162] These rules may include analysis of field names and pattern recognition of field content. For example, the smart model recognizer may contain a preset rule library that can recognize field data in the format of "John Doe" as a name field. If the field data matches these rules, the smart model recognizer will generate a corresponding recognition result.

[0163] Example description:

[0164] Assume that when the system processes a field "company_name", according to the processing priority, the smart model recognizer is called. The smart model recognizer applies its built-in preset naming recognition rules, such as "the field name contains 'company' or 'organization'", and further analyzes whether the field content conforms to the specific pattern of the company or organization name.

[0165] If the intelligent model recognizer successfully recognizes that the content of the "company_name" field meets the criteria for a company name according to the rules, it will output a recognition result that classifies the field as a "company name field." This recognition result will be used for further classification and processing of the field.

[0166] Through the above steps, this embodiment uses the powerful analytical capabilities of the intelligent model identifier to effectively identify and classify complex field data, especially in scenarios that are difficult to cover with traditional rules.

[0167] Furthermore, a fifth embodiment of the field classification method based on dynamic rules of the present invention is proposed. In the above step c2, the field data is identified by the intelligent model identifier using a built-in preset naming identification rule, including:

[0168] Step d1, creating a new recognition rule, obtaining positive and negative sample data, and using the positive and negative sample data to train the new recognition rule to obtain a newly added recognition rule;

[0169] Step d2, identifying the field data by using the preset naming recognition rule and the newly added recognition rule built into the intelligent identifier.

[0170] In this embodiment, the intelligent model identifier is a module based on machine learning or artificial intelligence, which is used to analyze field data and identify its category or characteristics. The intelligent model identifier can automatically identify field data by applying built-in naming recognition rules, such as determining whether the field data represents a name, address, organization name, etc. The preset naming recognition rules are models built based on training data to identify common field types.

[0171] In some cases, the existing preset naming recognition rules may not be sufficient to recognize new or field-specific field data. In order to adapt to new needs, new recognition rules can be created. These rules can identify specific types of field data by defining new features or conditions.

[0172] Positive sample data is field data that meets the conditions of the new recognition rule, while negative sample data is data that does not meet the conditions. When training new recognition rules, the system needs to use these sample data to learn and optimize the rules to improve the accuracy of recognition.

[0173] Positive and negative sample data are input into the training algorithm to adjust and optimize the parameters of the new recognition rules. Through training, the system can learn how to accurately distinguish between field data that meets the new rules and new data that does not meet them.

[0174] Through the training process, the system generates and optimizes new recognition rules. These rules are now able to accurately identify new types or field data of specific fields. The intelligent model recognizer will apply both the built-in preset named recognition rules and the newly created recognition rules to process field data. Combining these two types of rules, the recognizer can more comprehensively and accurately identify the type or characteristics of field data.

[0175] A specific implementation method may be: first, use the built-in preset rules of the intelligent model identifier to identify the field data. If it is found that the existing rules cannot accurately identify certain fields, the system will enter the new rule creation process. First, the system administrator or developer will collect relevant positive and negative sample data, which are used to train new recognition rules.

[0176] The training process involves feeding this sample data into the machine learning algorithm, which generates and optimizes new recognition rules by analyzing the characteristics of the sample data. After training, the system combines these newly added recognition rules with the existing rules to form a more powerful recognizer. When processing field data, the recognizer will apply both built-in rules and newly added rules to ensure that all types of field data can be accurately recognized.

[0177] Example description:

[0178] Suppose the system is processing a new set of field data that does not fully match the existing recognition rules. For example, a new field "user_code" does not conform to the existing rules in both field name and content. To deal with this situation, the system creates a new recognition rule.

[0179] First, the system administrator collected some sample data, such as fields similar to "user_code" (positive samples) and fields unrelated to it (negative samples). Then, the system used these sample data for training and generated a new rule that can identify fields similar to "user_code". Finally, the system combined this new rule with the preset rules of the intelligent model identifier and applied it to all new field data. When processing "user_code" or similar fields again, the system is able to accurately identify and classify these fields because it not only relies on the preset rules, but also utilizes the new recognition rules created specifically for these fields.

[0180] Through the above steps, this embodiment ensures the flexibility and adaptability of the system, enabling it to cope with different data scenarios.

[0181] Furthermore, a sixth embodiment of the field classification method based on dynamic rules of the present invention is proposed. In the above step S30, the field data is subjected to rapid feature detection, a processing priority of a preset identifier is set according to the detection result, and the field data is identified by using the preset identifier in sequence according to the processing priority, including:

[0182] Step e1, detecting whether the first preset number of characters of the field data have JSON text features;

[0183] Step e2: if there is a JSON text feature, convert the field data into a JSON object, perform a rapid feature detection on the JSON object, set a processing priority of a preset identifier according to the detection result, and use the preset identifier to identify the JSON object in turn according to the processing priority;

[0184] Step e3: If there is no JSON text feature, extract the first second preset number of phrases with the highest frequency in the field data, perform rapid feature detection on the second preset number of phrases, set the processing priority of the preset identifier according to the detection result, and use the preset identifier to identify the second preset number of phrases in turn according to the processing priority.

[0185] In this embodiment, the characteristics of the field data are first quickly analyzed to determine whether it has a specific format or content structure. This feature detection may include checking whether the field data contains a specific type of characters, whether it conforms to a specific data format (such as JSON format), or the length and frequency distribution of the field data.

[0186] Based on the results of feature detection, the system assigns processing priorities to different preset recognizers. The priority determines the order in which the recognizers process field data, ensuring that the most suitable recognizer is processed first to obtain the best results.

[0187] The system will first check whether the first few characters of the field data (i.e., the "first preset number of characters") have the characteristics of the JSON format, such as whether the field data starts with "{" or "[". This is a quick preliminary test to determine whether the field data may be a JSON object. Generally speaking, this check is for large text field data.

[0188] If the field data is detected to have JSON text features, the system will parse it into a JSON object for further analysis and processing. This conversion helps the system better understand the structure and content of the field data. After converting the field data to a JSON object, the system performs another quick feature detection. This detection will analyze the data structure, key-value pairs, nesting levels, etc. inside the JSON object to determine the most suitable processing strategy.

[0189] If the field data does not have JSON text features, the system will extract several phrases that appear most frequently in the field data (such as the first second preset number of phrases) for further feature analysis. This method is used to process unstructured or semi-structured field data, and obtain the main content and features of the data through word frequency analysis. The system performs rapid feature detection on the extracted high-frequency phrases, and analyzes the types, structures, and distribution of these phrases in the field data. This step helps to identify potential categories or characteristics of field data.

[0190] Whether it is a JSON object or a high-frequency phrase, the system will assign processing priorities to the recognizers based on the detection results. The priority setting ensures that the most relevant recognizers can process the data first to improve the accuracy of recognition. Finally, the system calls different recognizers in the set priority order, processes the field data or the converted JSON object, and generates the recognition results.

[0191] A specific implementation method may be: first, perform a preliminary analysis on the field data. If the first few characters of the field data are detected to have JSON text features, the system will parse it into a JSON object and further analyze the object. By performing feature detection on the key-value pair structure and content of the JSON object, the system assigns priorities to subsequent recognizer processing steps.

[0192] If the field data does not have JSON text features, the system will extract the most frequently occurring phrases in the field data and perform feature detection on these phrases. For example, the system may detect whether these phrases are terms in a specific field, or their grammatical functions in the text. Based on these detection results, the system determines the order in which the recognizers are called to ensure that the data is processed most efficiently.

[0193] Example description:

[0194] Suppose the system is processing a field with the content "{'name':'Alice','age':30,'address':{'city':'NewYork','zip':'10001'}}". The system first detects the first few characters of the field data and finds that it conforms to the JSON format, so it parses it into a JSON object. Next, the system performs feature detection on the key-value pairs of the JSON object and finds that the field contains the common "name" and "address" structures, so it preferentially calls the intelligent model recognizer to process it.

[0195] If the field data is unstructured text, such as "some text content frequently contains certain keywords", the system will extract the most frequently appearing keywords and perform feature analysis on them. Assuming that "keyword 1" and "keyword 2" are the most frequently appearing phrases, the system will set the processing priority of the recognizer based on these features, and prioritize calling the recognizer that can recognize these phrases for further analysis.

[0196] Through the above steps, this embodiment dynamically adjusts the calling order of the identifiers according to the characteristics of the field data to achieve more efficient field recognition and classification.

[0197] Furthermore, a seventh embodiment of the field classification method based on dynamic rules of the present invention is proposed. In the above step S40, after generating the field classification result according to the metadata processing result, the method further includes:

[0198] Step f1, obtaining user feedback information, and adjusting the preset matching rules and preset logical processing rules according to the user feedback information.

[0199] In this embodiment, after generating the field classification results, user feedback information is further collected and analyzed. User feedback may include evaluation of the accuracy and applicability of the classification results, or user suggestions and improvement opinions on the system identification rules. Through this feedback information, the system can obtain problems in actual use and potential room for improvement.

[0200] According to user feedback, the existing preset matching rules and preset logical processing rules are adjusted and optimized. The preset matching rules are used to initially screen metadata, while the preset logical processing rules are used to perform further logical operations on the target metadata. By adjusting these rules, the system can improve the accuracy and consistency of the grading results and better adapt to user needs and usage environment.

[0201] A specific implementation method may be: after completing the field classification, the user's feedback information will be actively obtained. This feedback information can be obtained in various forms, such as evaluations directly entered by users, analysis of system usage logs, or manual adjustment records of users to specific classification results. After the system collects this feedback information, it is used to analyze the effectiveness of existing preset matching rules and logical processing rules.

[0202] Based on the analysis results, the system may find that some matching rules are too strict or loose, or that the logic processing rules do not adequately capture the characteristics of the field data. In this case, the system will adjust these rules. For example, the system may modify the matching rules to better suit a specific type of data, or adjust the priority of the logic processing rules to improve the accuracy of the overall classification.

[0203] Example description:

[0204] Assume that when the system processes a newly added field "transaction_code", it classifies it as a "normal text field" according to the existing rules. However, in actual use, the user finds that the field should be classified as a "sensitive data field" because it involves transaction information. The user submits feedback through the system interface to point out this error.

[0205] After receiving the feedback, the system analyzed the existing matching rules and logical processing rules and found that the existing rules failed to accurately identify "transaction_code" as sensitive information. Based on the feedback information, the system adjusted the matching rules to make it easier to identify fields containing the word "transaction" and increased the priority of related rules. After the adjustment, the system will more accurately classify similar fields as "sensitive data fields" the next time it processes similar fields.

[0206] Through the above steps, this embodiment can continuously optimize its recognition and classification capabilities to ensure that the field classification results can better meet the actual needs of users.

[0207] The present invention also provides a field grading device based on dynamic rules.

[0208] Reference Figure 3 , Figure 3 This is a functional module diagram of the first embodiment of the field grading device based on dynamic rules of the present invention. The field grading device based on dynamic rules includes:

[0209] A data acquisition module 10, which acquires field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data;

[0210] The metadata analysis and processing module 20 analyzes the metadata according to a preset matching rule, selects target metadata, and performs logical operation processing on the target metadata according to a preset logical processing rule to obtain a metadata processing result;

[0211] The field data analysis module 30 performs rapid feature detection on the field data, sets a processing priority of a preset identifier according to the detection result, and sequentially uses the preset identifier to identify the field data according to the processing priority to obtain a field data identification result;

[0212] The grading decision module 40 generates a field grading result according to the metadata processing result if the field data recognition result is empty; and generates a field grading result according to the field data recognition result if the field data recognition result is not empty.

[0213] In addition, an embodiment of the present invention further provides a computer storage medium.

[0214] The storage medium stores a field grading program based on dynamic rules. When the field grading program based on dynamic rules is executed by the processor, the steps of the field grading method based on dynamic rules as described above are implemented.

[0215] The specific implementation of the storage medium of the present invention is basically the same as the above-mentioned embodiments of the field classification method based on dynamic rules, and will not be repeated here.

[0216] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

[0217] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0218] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk.

[0219] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0220] The above are only various implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims. It should be noted that if software tools or components other than our company appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use.

Claims

1. A field classification method based on dynamic rules, It is characterized in that The following steps are involved: Acquire field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data; Analyze the metadata according to preset matching rules, filter out target metadata, and perform logical operation processing on the target metadata according to preset logical processing rules to obtain metadata processing results; Performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and using the preset identifiers to identify the field data in sequence according to the processing priority to obtain a field data identification result; If the field data identification result is empty, generating a field classification result according to the metadata processing result; If the field data recognition result is not empty, a field classification result is generated according to the field data recognition result.

2. The field classification method based on dynamic rules as claimed in claim 1, It is characterized in that The field data is identified by using the preset identifiers in sequence according to the processing priority to obtain a field data identification result, including: Using the preset identifiers in sequence to identify the field data according to the processing priority, obtaining an identification result corresponding to each preset identifier, wherein the preset identifiers include a group sensitive identifier, a sample rule identifier, an intelligent model identifier, and a regular rule identifier; The proportion of the recognition results corresponding to each preset identifier is calculated by a proportion formula, and the recognition result with the highest proportion is used as the field data recognition result.

3. The field classification method based on dynamic rules as claimed in claim 2, It is characterized in that The field data is identified by using the preset identifiers in sequence according to the processing priority, and an identification result corresponding to each preset identifier is obtained, including: Using the preset identifiers to identify the field data in sequence according to the processing priority; When the sample rule identifier is used to identify the field data, a preset sample rule is loaded through the sample rule identifier; The sample data set of the preset sample rule is traversed to perform a complete match judgment with the field data. If the complete match judgment is successful, the recognition task of the sample rule identifier is ended to obtain the recognition result of the sample rule identifier.

4. The field classification method based on dynamic rules as claimed in claim 2, It is characterized in that The field data is identified by using the preset identifiers in sequence according to the processing priority, and an identification result corresponding to each preset identifier is obtained, including: Using the preset identifiers to identify the field data in sequence according to the processing priority; When the intelligent model identifier is used to identify the field data, the intelligent model identifier uses a built-in preset naming recognition rule to identify the field data to obtain a recognition result of the intelligent model identifier.

5. The field classification method based on dynamic rules as claimed in claim 4, It is characterized in that The field data is identified by applying a built-in preset naming identification rule by the intelligent model identifier, including: Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain a newly added recognition rule; The field data is identified by using the preset naming identification rules and the newly added identification rules built into the intelligent identifier application.

6. The field classification method based on dynamic rules as claimed in claim 1, It is characterized in that Performing rapid feature detection on the field data, setting a processing priority of a preset identifier according to the detection result, and sequentially using the preset identifiers to identify the field data according to the processing priority, including: Detecting whether the first preset number of characters of the field data have JSON text features; If there is a JSON text feature, convert the field data into a JSON object, perform rapid feature detection on the JSON object, set a processing priority of a preset identifier according to the detection result, and use the preset identifier to identify the JSON object in sequence according to the processing priority; If there is no JSON text feature, extract the first second preset number of phrases with the highest frequency in the field data, perform fast feature detection on the second preset number of phrases, set the processing priority of the preset identifier according to the detection result, and use the preset identifier to identify the second preset number of phrases in sequence according to the processing priority.

7. The field classification method based on dynamic rules as described in claim 1, It is characterized in that After generating the field classification result according to the metadata processing result, the method further includes: Acquire user feedback information, and adjust the preset matching rules and preset logical processing rules according to the user feedback information.

8. A field grading device based on dynamic rules, It is characterized in that The field grading device based on dynamic rules includes: A data acquisition module, which acquires field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraint conditions of the corresponding field data; The metadata analysis and processing module analyzes the metadata according to the preset matching rules, selects the target metadata, performs logical operation processing on the target metadata according to the preset logical processing rules, and obtains the metadata processing result; A field data analysis module performs rapid feature detection on the field data, sets a processing priority of a preset identifier according to the detection result, and sequentially uses the preset identifier to identify the field data according to the processing priority to obtain a field data identification result; The grading decision module generates a field grading result according to the metadata processing result if the field data identification result is empty; and generates a field grading result according to the field data identification result if the field data identification result is not empty.

9. A field grading device based on dynamic rules, It is characterized in that The dynamic rule-based field grading device includes a memory, a processor, and a dynamic rule-based field grading program stored in the memory and executable on the processor. When the dynamic rule-based field grading program is executed by the processor, the steps of the dynamic rule-based field grading method as described in any one of claims 1 to 7 are implemented.

10. A computer storage medium, It is characterized in that The storage medium stores a dynamic rule-based field grading program, which, when executed by a processor, implements the steps of the dynamic rule-based field grading method as described in any one of claims 1 to 7.

Images