Dynamic rule-based field grading method, device, equipment and storage medium

Abstract

The application relates to the technical field of data processing and analysis and the technical field of financial technology, and discloses a field grading method based on dynamic rules, which comprises the following steps: acquiring field data and corresponding metadata, analyzing the metadata according to a preset matching rule, and screening out target metadata; performing logical operation processing on the target metadata to obtain a metadata processing result; performing rapid feature detection on the field data, setting a processing priority of a preset recognizer according to a detection result, sequentially processing the field data, and generating a field data recognition result; judging whether the recognition result is empty, and if the recognition result is empty, generating a field grading result according to the metadata processing result; and if the recognition result is not empty, generating a field grading result according to the field data recognition result. According to the application, the data is screened, the amount of processed data is reduced, and the processing rule can be adjusted according to the data scale and characteristics. Meanwhile, important data is preferentially processed, performance degradation is avoided, and thus the overall data processing efficiency and precision are improved.

Description

Technical Field

[0001] This invention relates to the fields of data processing and analysis technology and financial technology technology, and in particular to a field classification method, apparatus, device and storage medium based on dynamic rules. Background Technology

[0002] In the information and data age, the financial industry relies heavily on database systems to manage and process massive amounts of customer data, transaction data, and financial records. As the types and quantities of financial data continue to grow, their complexity also increases significantly. Therefore, effective classification and grading of this data is crucial, directly impacting the efficiency of business processes and the accuracy of decision-making.

[0003] Traditionally, financial institutions often use static rules to classify data. These static rules are usually manually set based on business needs and industry standards. For example, a bank might classify customer data based on parameters such as account type and transaction amount. However, static rules are inefficient and inflexible when dealing with rapidly changing financial markets, especially when faced with complex, unstructured data (such as customer behavior data and market conditions), where these fixed rules struggle to capture the dynamic characteristics of the data.

[0004] One of the core needs of the financial industry is to respond to market changes and mitigate risks in real time. For example, financial institutions may need to dynamically adjust their risk control models or product recommendation strategies based on the latest market trends or customer behavior data. Static rules cannot reflect these changes in a timely manner, thus affecting the accuracy of decision-making.

[0005] While dynamic rules offer numerous advantages for financial data classification, they also face several challenges. Financial data is vast and diverse, and frequent rule adjustments can increase system load, especially when the entire dataset needs to be recalculated. This not only degrades overall system performance but can also affect the accuracy of classification results. Particularly when processing large datasets in real time, the strain on computing resources can lead to decreased efficiency and accuracy in classification. Summary of the Invention

[0006] The main objective of this invention is to provide a field classification method, apparatus, device, and storage medium based on dynamic rules, aiming to solve the technical problems of low classification efficiency and insufficient accuracy in existing field classification methods.

[0007] To achieve the above objectives, the present invention provides a field hierarchical method based on dynamic rules, which includes the following steps:

[0008] Obtain the field data and the corresponding metadata, whereby the metadata is used to define the structure, type, and constraints of the corresponding field data;

[0009] The metadata is analyzed according to preset matching rules, target metadata is filtered out, and logical operations are performed on the target metadata according to preset logical processing rules to obtain metadata processing results.

[0010] The field data is subjected to rapid feature detection. Based on the detection results, the processing priority of the preset recognizer is set. The preset recognizer is used to recognize the field data in sequence according to the processing priority to obtain the field data recognition result.

[0011] If the field data identification result is empty, then a field classification result is generated based on the metadata processing result;

[0012] If the field data recognition result is not empty, then a field classification result is generated based on the field data recognition result.

[0013] Optionally, the preset recognizer is used sequentially according to the processing priority to recognize the field data, and the field data recognition result is obtained, including:

[0014] According to the processing priority, the preset recognizers are used sequentially to recognize the field data to obtain the recognition result corresponding to each preset recognizer. The preset recognizers include a group sensitive recognizer, a sample rule recognizer, an intelligent model recognizer, and a regular expression rule recognizer.

[0015] The proportion of recognition results corresponding to each preset recognizer is calculated using a proportion formula, and the recognition result with the highest proportion is taken as the recognition result of the field data.

[0016] Optionally, the field data is sequentially identified using the preset recognizers according to the processing priority, to obtain the recognition result corresponding to each preset recognizer, including:

[0017] The preset recognizer is used to recognize the field data sequentially according to the processing priority;

[0018] When the sample rule recognizer is used to recognize the field data, a preset sample rule is loaded through the sample rule recognizer.

[0019] The sample dataset of the preset sample rules is traversed and a complete match is performed with the field data. If the complete match is successful, the recognition task of the sample rule recognizer ends and the recognition result of the sample rule recognizer is obtained.

[0020] Optionally, the field data is sequentially identified using the preset recognizers according to the processing priority, to obtain the recognition result corresponding to each preset recognizer, including:

[0021] The preset recognizer is used to recognize the field data sequentially according to the processing priority;

[0022] When the intelligent model recognizer is used to recognize the field data, the intelligent model recognizer applies built-in preset naming recognition rules to recognize the field data, and obtains the recognition result of the intelligent model recognizer.

[0023] Optionally, the field data is identified by applying built-in preset naming recognition rules through the intelligent model recognizer, including:

[0024] Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain the new recognition rule;

[0025] The intelligent recognizer uses the built-in preset naming recognition rules and the newly added recognition rules to recognize the field data.

[0026] Optionally, the field data undergoes rapid feature detection, and a processing priority for a preset recognizer is set based on the detection results. The preset recognizer is then used sequentially to identify the field data according to the processing priority, including:

[0027] Detect whether the first preset number of characters of the field data have JSON text characteristics;

[0028] If the data has JSON text characteristics, the field data is converted into a JSON object, the JSON object is subjected to fast feature detection, the processing priority of the preset recognizer is set according to the detection results, and the preset recognizer is used to recognize the JSON object in sequence according to the processing priority.

[0029] If there are no JSON text features, extract the second preset number of most frequent word groups from the field data, perform fast feature detection on the second preset number of word groups, set the processing priority of the preset recognizer according to the detection results, and use the preset recognizer to recognize the second preset number of word groups in sequence according to the processing priority.

[0030] Optionally, after generating the field classification result based on the metadata processing result, the method further includes:

[0031] Obtain user feedback information and adjust the preset matching rules and preset logic processing rules based on the user feedback information.

[0032] Furthermore, to achieve the above objectives, the present invention also provides a field classification device based on dynamic rules, the field classification device based on dynamic rules comprising:

[0033] The data acquisition module acquires field data and the metadata corresponding to the field data. The metadata is used to define the structure, type, and constraints of the corresponding field data.

[0034] The metadata analysis and processing module analyzes the metadata according to preset matching rules, filters out target metadata, and performs logical operations on the target metadata according to preset logical processing rules to obtain the metadata processing result.

[0035] The field data analysis module performs rapid feature detection on the field data, sets the processing priority of the preset recognizer based on the detection results, and uses the preset recognizer to identify the field data in sequence according to the processing priority to obtain the field data recognition result.

[0036] The hierarchical decision module generates a field hierarchical result based on the metadata processing result if the field data identification result is empty; otherwise, it generates a field hierarchical result based on the field data identification result.

[0037] Furthermore, to achieve the above objectives, the present invention also provides a field classification device based on dynamic rules, the field classification device based on dynamic rules including a memory, a processor, and a field classification program based on dynamic rules stored in the memory and executable on the processor, the field classification program based on dynamic rules implementing the steps of the field classification method based on dynamic rules as described above when executed by the processor.

[0038] Furthermore, to achieve the above objectives, the present invention also provides a computer storage medium storing a field classification program based on dynamic rules, wherein when the field classification program based on dynamic rules is executed by a processor, it implements the steps of the field classification method based on dynamic rules as described above.

[0039] This invention relates to a field classification method based on dynamic rules. It involves acquiring field data and corresponding metadata, analyzing the metadata according to preset matching rules to filter target metadata, performing logical operations on the target metadata according to preset logical processing rules to obtain metadata processing results, performing rapid feature detection on the field data, setting the processing priority of preset recognizers based on the detection results, and using the recognizers sequentially to process the field data to generate field data recognition results, and determining whether the field data recognition results are empty. If empty, a field classification result is generated based on the metadata processing results; otherwise, a field classification result is generated based on the field data recognition results. This invention reduces the amount of data processed by filtering data and allows for adjustments to processing rules based on data scale and characteristics. Simultaneously, it prioritizes important data processing to avoid performance degradation, thereby improving overall data processing efficiency and accuracy. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the hardware operating environment of the device involved in the field hierarchical device embodiment of the present invention based on dynamic rules;

[0041] Figure 2 This is a flowchart illustrating the first embodiment of the field hierarchical method based on dynamic rules of the present invention;

[0042] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the field classification device based on dynamic rules of the present invention.

[0043] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0045] It's important to note that in the information and data age, the financial industry heavily relies on database systems to manage and process massive amounts of customer data, transaction data, and financial records. As the types and quantities of financial data continue to grow, their complexity also increases significantly. Therefore, effective classification and grading of this data is crucial, directly impacting the efficiency of business processes and the accuracy of decision-making.

[0046] Traditionally, financial institutions often use static rules to classify data. These static rules are usually manually set based on business needs and industry standards. For example, a bank might classify customer data based on parameters such as account type and transaction amount. However, static rules are inefficient and inflexible when dealing with rapidly changing financial markets, especially when faced with complex, unstructured data (such as customer behavior data and market conditions), where these fixed rules struggle to capture the dynamic characteristics of the data.

[0047] One of the core needs of the financial industry is to respond to market changes and mitigate risks in real time. For example, financial institutions may need to dynamically adjust their risk control models or product recommendation strategies based on the latest market trends or customer behavior data. Static rules cannot reflect these changes in a timely manner, thus affecting the accuracy of decision-making.

[0048] While dynamic rules offer numerous advantages for financial data classification, they also face several challenges. Financial data is vast and diverse, and frequent rule adjustments can increase system load, especially when the entire dataset needs to be recalculated. This not only degrades overall system performance but can also affect the accuracy of classification results. Particularly when processing large datasets in real time, the strain on computing resources can lead to decreased efficiency and accuracy in classification.

[0049] To address the aforementioned shortcomings, this invention provides a field-leveling device based on dynamic rules, referring to... Figure 1 , Figure 1 This is a schematic diagram of the hardware operating environment of the device involved in the field-level device embodiment of the present invention based on dynamic rules.

[0050] like Figure 1 As shown, the field-level device based on dynamic rules may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1The hardware structure of the field-level device based on dynamic rules shown in the figure does not constitute a limitation on the field-level device based on dynamic rules. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] like Figure 1 As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and a field hierarchical program based on dynamic rules. The operating system is a program that manages and controls the field hierarchical device and software resources based on dynamic rules, supporting the operation of the network communication module, the user interface module, the field hierarchical program based on dynamic rules, and other programs or software. The network communication module manages and controls the network interface 1004; the user interface module manages and controls the user interface 1003.

[0053] exist Figure 1 In the hardware structure of the field classification device based on dynamic rules shown, the network interface 1004 is mainly used to connect to the backend server and communicate with it; the user interface 1003 is mainly used to connect to the client and communicate with it; the processor 1001 can call the field classification program based on dynamic rules stored in the memory 1005 and perform the following operations:

[0054] Obtain the field data and the corresponding metadata, whereby the metadata is used to define the structure, type, and constraints of the corresponding field data;

[0055] The metadata is analyzed according to preset matching rules, target metadata is filtered out, and logical operations are performed on the target metadata according to preset logical processing rules to obtain metadata processing results.

[0056] The field data is subjected to rapid feature detection. Based on the detection results, the processing priority of the preset recognizer is set. The preset recognizer is used to recognize the field data in sequence according to the processing priority to obtain the field data recognition result.

[0057] If the field data identification result is empty, then a field classification result is generated based on the metadata processing result;

[0058] If the field data recognition result is not empty, then a field classification result is generated based on the field data recognition result.

[0059] Further, the preset recognizer is used sequentially according to the processing priority to recognize the field data, and the field data recognition result is obtained, including:

[0060] According to the processing priority, the preset recognizers are used sequentially to recognize the field data to obtain the recognition result corresponding to each preset recognizer. The preset recognizers include a group sensitive recognizer, a sample rule recognizer, an intelligent model recognizer, and a regular expression rule recognizer.

[0061] The proportion of recognition results corresponding to each preset recognizer is calculated using a proportion formula, and the recognition result with the highest proportion is taken as the recognition result of the field data.

[0062] Further, the field data is sequentially identified using the preset recognizers according to the processing priority, resulting in a recognition result corresponding to each preset recognizer, including:

[0063] The preset recognizer is used to recognize the field data sequentially according to the processing priority;

[0064] When the sample rule recognizer is used to recognize the field data, a preset sample rule is loaded through the sample rule recognizer.

[0065] The sample dataset of the preset sample rules is traversed and a complete match is performed with the field data. If the complete match is successful, the recognition task of the sample rule recognizer ends and the recognition result of the sample rule recognizer is obtained.

[0066] Further, the field data is sequentially identified using the preset recognizers according to the processing priority, resulting in a recognition result corresponding to each preset recognizer, including:

[0067] The preset recognizer is used to recognize the field data sequentially according to the processing priority;

[0068] When the intelligent model recognizer is used to recognize the field data, the intelligent model recognizer applies built-in preset naming recognition rules to recognize the field data, and obtains the recognition result of the intelligent model recognizer.

[0069] Furthermore, the field data is identified by applying built-in preset naming recognition rules through the intelligent model recognizer, including:

[0070] Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain the new recognition rule;

[0071] The intelligent recognizer uses the built-in preset naming recognition rules and the newly added recognition rules to recognize the field data.

[0072] Furthermore, rapid feature detection is performed on the field data, and a processing priority of a preset recognizer is set according to the detection results. The preset recognizer is then used sequentially to recognize the field data according to the processing priority, including:

[0073] Detect whether the first preset number of characters of the field data have JSON text characteristics;

[0074] If the data has JSON text characteristics, the field data is converted into a JSON object, the JSON object is subjected to fast feature detection, the processing priority of the preset recognizer is set according to the detection results, and the preset recognizer is used to recognize the JSON object in sequence according to the processing priority.

[0075] If there are no JSON text features, extract the second preset number of most frequent word groups from the field data, perform fast feature detection on the second preset number of word groups, set the processing priority of the preset recognizer according to the detection results, and use the preset recognizer to recognize the second preset number of word groups in sequence according to the processing priority.

[0076] Furthermore, after generating the field classification results based on the metadata processing results, the method further includes:

[0077] Obtain user feedback information and adjust the preset matching rules and preset logic processing rules based on the user feedback information.

[0078] The specific implementation of the field classification device based on dynamic rules of the present invention is basically the same as the various embodiments of the field classification method based on dynamic rules described below, and will not be repeated here.

[0079] The present invention also provides a field classification method based on dynamic rules based on the above-mentioned field classification device based on dynamic rules.

[0080] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the field hierarchical method based on dynamic rules of the present invention.

[0081] This invention provides an embodiment of a field hierarchical method based on dynamic rules. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0082] In various embodiments of the field classification method based on dynamic rules, the execution entity is a field classification device based on dynamic rules.

[0083] The dynamic rule-based field classification method is applied to a dynamic rule-based field classification device, and the dynamic rule-based field classification method includes:

[0084] Step S10: Obtain the metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraints of the corresponding field data;

[0085] In this embodiment, two key pieces of information are obtained from the data source: field data and corresponding metadata. Field data refers to the actual stored values ​​in the database or data table (e.g., specific numbers, strings, or dates), while metadata describes the attributes and structure of the field data, such as field name, data type, length, and whether it is nullable. Metadata provides context and definition for the field data, ensuring that the system can correctly interpret and process the field data.

[0086] The system first extracts the actual field data from the database or other data storage location, while simultaneously retrieving metadata related to these fields. The system can obtain this data through SQL queries, API calls, or other methods. Metadata typically includes field names, field types, constraints (such as NOT NULL, uniqueness, etc.), and provides the foundation for subsequent data processing.

[0087] Metadata serves to define field data, describing its structure (e.g., whether it's a nested structure or a simple field), data type (e.g., string, integer, date), and constraints (e.g., whether a field is required, maximum or minimum value length, or uniqueness requirements). These definitions ensure that the storage, retrieval, and processing of field data within the system are compliant and consistent.

[0088] In the system, metadata is typically predefined and stored in a metadata table or metadata management system. When the system retrieves field data, it automatically searches and matches the corresponding metadata to ensure that the field data conforms to the metadata definition. For example, if the metadata specifies that a field is of integer type, the system will perform type validation on that field during the data processing stage to ensure that its data is in integer format. In addition, the system will perform compliance checks on field data based on constraints in the metadata (such as non-null and uniqueness) to prevent the entry of non-compliant data.

[0089] By associating field data with corresponding metadata, the system can ensure that it has clear contextual information when processing field data, avoiding processing errors caused by data type mismatches or incorrect formats. Metadata defines the structure, type, and constraints of field data, helping to improve the accuracy and consistency of data processing and ensuring that data is correctly interpreted and processed at each stage. At the same time, through metadata constraints, the system can automate data validation, improving data quality and reducing the storage and transmission of erroneous data.

[0090] Step S20: Analyze the metadata according to the preset matching rules, filter out the target metadata, and perform logical operations on the target metadata according to the preset logical processing rules to obtain the metadata processing result;

[0091] In this embodiment, the system automatically analyzes metadata based on a set of predefined matching rules. These matching rules are defined based on business requirements and metadata attributes, such as whether field names contain specific keywords and whether data types meet expectations. The matching rules can quickly filter out metadata that does not meet the requirements, thereby improving processing efficiency.

[0092] First, the system retrieves all relevant metadata from the metadata store. Then, it applies predefined matching rules to analyze each piece of metadata. For example, a matching rule might be that the field name must contain the word "Date," or that the field type must be integer. The system analyzes the metadata piece by piece using methods such as string matching and type checking, filtering out metadata that meets the criteria.

[0093] After filtering out the target metadata, the system processes this data according to predefined logical processing rules. The logical processing rules can include logical operations such as "AND", "OR", and "NOT" to combine or exclude specific conditions, and finally generate the metadata processing results.

[0094] The final identification result is generated by processing the target metadata through logical operations. This identification result is used in the subsequent field classification process to determine how fields are classified and classified.

[0095] A specific implementation method could be as follows: First, retrieve the required field data and corresponding metadata from the database. The system then performs a preliminary analysis of this metadata using predefined matching rules, filtering out metadata that meets specific conditions. For example, the system could set a rule to filter out all metadata whose field names contain "date".

[0096] Once the target metadata is selected, the system will apply preset logical processing rules to further process this data. These rules can be customized according to business needs, such as combining multiple conditions for filtering or categorization. Through this logical processing, the system ultimately generates a metadata processing result, which will guide the classification and grading of fields.

[0097] It's important to note that matching rules and logical processing rules can be modified based on specific needs. Changes in business scenarios can affect these rules. For instance, the priority, content requirements, or structure of certain data fields may be adjusted due to business needs, leading to rule changes. If a business requirement changes from a "date field" to a "timestamp field," the matching rules and logical processing rules may need to be adjusted to recognize and process the new data format.

[0098] When the scale of data changes significantly (e.g., upgrading from small-scale data processing to big data processing), rules may need to be changed to optimize system performance. Similarly, different data types will also lead to rule adjustments. For example, on large datasets, matching rules can be simplified or restricted to reduce processing time; while when processing structured or semi-structured data, logical processing rules may need to be adapted to different formats (such as JSON, XML, etc.).

[0099] To improve system performance and responsiveness, rules may need to be dynamically adjusted or simplified. For example, when the system is under heavy load, certain logical processing rules can be simplified to reduce the impact of complex calculations on performance. For instance, under high load, the system may need to reduce complex multi-condition logical operations and instead use simpler rules to improve efficiency.

[0100] Example explanation:

[0101] Suppose the database contains multiple tables with numerous fields. The system first retrieves the field data and metadata, such as table name, field name, field type, and description. Then, the system uses preset matching rules to analyze this metadata. For example, it might filter out all field data whose names contain "price" and whose data type is numeric.

[0102] After filtering out the target metadata that meets the criteria, the system further processes it according to preset logical rules. Assuming the logical rule is: if the field name contains "price" and the field type is numeric, the system will identify these fields as "price fields" and assign them specific tags or categories. In the final identification results, the system will recognize these fields as pricing-related fields, facilitating subsequent data processing or analysis tasks.

[0103] Through the above steps, automated identification and classification of database fields are achieved, which not only improves data processing efficiency but also reduces the need for manual intervention, ensuring the consistency and accuracy of field classification. The phased data processing and priority setting mechanism ensures that when dynamic rules are adjusted, the system's overall performance is not immediately affected by rule changes. When dynamic rules are adjusted, the system does not immediately reprocess all data; instead, it initially filters out data that meets the criteria, reducing the processing load and thus ensuring that the overall system performance is not affected.

[0104] Step S30: Perform fast feature detection on the field data, set the processing priority of the preset recognizer according to the detection result, and use the preset recognizer to recognize the field data in sequence according to the processing priority to obtain the field data recognition result;

[0105] In this embodiment, the purpose of rapid feature detection on field data is to quickly analyze the characteristics of the field data in order to identify key attributes that affect data processing. These characteristics may include data type (such as integers, strings), data format (such as dates, timestamps), data length, data content pattern (such as whether it contains special characters), etc.

[0106] Based on the feature detection results, the system will assign processing priorities to different preset recognizers. The priority determines the order in which the recognizers process field data. For example, some recognizers may be better suited for processing numerical data, while others are better suited for processing text data. The system will prioritize calling the most suitable recognizer based on the degree of feature matching.

[0107] Following a set priority order, different recognizers are invoked one by one to identify the field data. Each recognizer analyzes and identifies the field data based on its specific rules and algorithms, generating corresponding recognition results.

[0108] After each recognizer completes its recognition of the field data, the system collects and analyzes the generated recognition results. If all recognizer results are empty or fail to recognize valid information, the system will determine that the field data recognition result is empty and may switch to hierarchical processing based on metadata results. If the recognition result is not empty, the system will directly use the field data recognition result for hierarchical processing.

[0109] A specific implementation method could be as follows: First, perform feature detection on the field data. For example, the system might detect whether the field data is numeric, whether the content conforms to a date format, or whether the field content contains specific keywords. Based on the detection results, the system assigns priorities to different recognizers.

[0110] If the system detects that the field data is in date format, it will first call the date recognizer to recognize the field data. After the recognizer processes the field data, the system will analyze the generated recognition result. If the recognition result is empty, the system may re-evaluate the feature detection result or call other recognizers until a valid recognition result is generated.

[0111] Example explanation:

[0112] Suppose we have a field containing the data "2023-08-13". The system first confirms that the field conforms to a date format through feature detection. Based on this result, the system sets the date recognizer to the highest priority and calls it first to process the data. If the date recognizer successfully identifies this as a date field, the system will generate a recognition result, such as "Date Field", and continue the hierarchical process.

[0113] If the field data is "null" or the recognizer fails to generate a valid result, the system will determine that the recognition result is empty. In this case, the system may call other recognizers or classify the field based on the recognition results of metadata.

[0114] By employing the steps described above, multiple identifiers are flexibly used to process and identify field data, ensuring the accuracy and effectiveness of classification and grading. Through phased data processing and logical operation rules, the grading strategy can be flexibly adjusted according to the data scale, prioritizing important data and ensuring efficiency and accuracy in big data environments. If the data scale is large, the system can adjust accordingly. For example, with small data sets, the entire dataset can be processed directly; while with large data sets, the system can filter out the most important or high-priority data based on matching rules for priority processing, delaying the processing of less important data.

[0115] Step S40: If the field data identification result is empty, then generate a field classification result based on the metadata processing result; if the field data identification result is not empty, then generate a field classification result based on the field data identification result.

[0116] In this embodiment, if the identification result of the field data is empty, the system will rely on the previous metadata processing results to generate the field's classification result. The metadata processing result may be obtained based on metadata attributes such as the field's name, type, and description, through preset logical processing rules. If the field data cannot provide enough information for classification, the system will use these metadata processing results for the final classification and classification.

[0117] If the field data recognition result is not empty, meaning the recognizer has successfully identified valid information, the system will directly use these field data recognition results to generate the field classification result. This means that the field data itself provides sufficient information for classification and classification, without relying on metadata.

[0118] A specific implementation method could be as follows: First, attempt to use a recognizer to identify the field data and generate identification results. If the field data identification result is empty (for example, the recognizer cannot extract useful information from the field data), the system will instead use the identification results previously obtained through metadata analysis to generate the final field classification result.

[0119] If the field data identification results are valid, the system will prioritize using these results to generate hierarchical results, because the field data directly reflects the actual content and characteristics of the data, which may be more accurate and reliable than relying solely on metadata analysis.

[0120] Example explanation:

[0121] If a field contains empty or ambiguous data (such as "null"), the field identification result may be empty. The system will use metadata, such as the field name "birthdate", for identification and classification, ultimately classifying the field as a "date field".

[0122] Conversely, if the field data is "2023-08-13" and the recognizer successfully identifies it as a date field, the system will directly classify the field as a "date field" based on this result, without referring to metadata.

[0123] The above steps ensure that even when field data lacks information, reasonable classification and grading can still be achieved by relying on metadata, while when field data is sufficient, more accurate grading can be achieved by prioritizing the information from the data itself.

[0124] This invention relates to a field classification method based on dynamic rules. It involves acquiring field data and corresponding metadata, analyzing the metadata according to preset matching rules to filter target metadata, performing logical operations on the target metadata according to preset logical processing rules to obtain metadata processing results, performing rapid feature detection on the field data, setting the processing priority of preset recognizers based on the detection results, and using the recognizers sequentially to process the field data to generate field data recognition results, and determining whether the field data recognition results are empty. If empty, a field classification result is generated based on the metadata processing results; otherwise, a field classification result is generated based on the field data recognition results. This invention reduces the amount of data processed by filtering data and allows for adjustments to processing rules based on data scale and characteristics. Simultaneously, it prioritizes important data processing to avoid performance degradation, thereby improving overall data processing efficiency and accuracy.

[0125] Furthermore, a second embodiment of the field classification method based on dynamic rules of the present invention is proposed. In step S20 above, the field data is identified sequentially using the preset recognizer according to the processing priority to obtain the field data identification result, including:

[0126] Step a1: According to the processing priority, the preset recognizers are used sequentially to recognize the field data to obtain the recognition result corresponding to each preset recognizer. The preset recognizers include a group sensitive recognizer, a sample rule recognizer, an intelligent model recognizer, and a regular expression rule recognizer.

[0127] Step a2: Calculate the proportion of the recognition result corresponding to each preset recognizer using the proportion formula, and take the recognition result with the highest proportion as the recognition result of the field data.

[0128] In this embodiment, when identifying field data, different recognizers are sequentially invoked according to a pre-set processing priority to identify the field data. Each recognizer analyzes the field data based on its area of ​​expertise (such as sensitive group data identification, sample rule identification, intelligent model identification, and regular rule recognizer) and generates its own identification result. Finally, the system summarizes these identification results and determines which identification result is the most reliable or representative.

[0129] A recognizer is a module or tool in a system specifically designed to identify and process specific types of field data. Preset recognizer types include:

[0130] Group Sensitive Identifier: Used to identify and classify data involving sensitive information, such as personally identifiable information or financial data.

[0131] Sample rule recognizer: Matches and identifies fields based on predefined samples and rules, and is particularly suitable for processing data with known categories.

[0132] Intelligent model recognizer: It uses machine learning models or artificial intelligence algorithms to process complex or unforeseen data types and has a high degree of adaptability.

[0133] Regular expression rule recognizer: It performs pattern matching on field data using regular expressions, and is suitable for recognizing data in specific formats, such as email addresses and phone numbers.

[0134] The system calculates the "percentage" or "weight" of each recognizer's field data recognition results. These percentages may be based on factors such as the recognizer's confidence level and the coverage of the recognition results. The percentage formula typically calculates the proportion of times a particular recognizer hits a field out of the total number of hits. The system uses the recognition result with the highest percentage as the final field data recognition result, as it is considered the most representative or accurate.

[0135] A specific implementation method could be as follows: First, based on the feature detection results of the field data, determine the order in which different recognizers are invoked. For example, if the system detects that a field may contain sensitive information, it might prioritize invoking the group's sensitive information recognizer. If the recognizer successfully identifies the field as sensitive information, the system will record the result and continue to invoke other recognizers to obtain more information.

[0136] After all the recognizers have finished processing, the system calculates the percentage of each recognition result. For example, the group sensitivity recognizer might identify 50% of the fields as sensitive information, while the sample rule recognizer and the intelligent model recognizer might have percentages of 30% and 20%, respectively. Ultimately, the system selects the recognition result with the highest percentage (i.e., the result from the group sensitivity recognizer) as the final classification result for the field.

[0137] Example explanation:

[0138] Suppose a field "salary" appears in a database table. The system first detects that this field may involve sensitive information, so it prioritizes calling the group's sensitivity identifier. The group's sensitivity identifier identifies that the field is related to financial data and classifies it as a sensitive information field.

[0139] Next, the system invokes the sample rule recognizer, which finds that the field name and content match the preset "salary" category field rules. The intelligent model recognizer further confirms, through a machine learning model, that the field does indeed belong to the salary category.

[0140] After all the identifiers have completed their processing, the system calculates the percentage of each identification result and finds that the identification result of the group's sensitive identifier has the highest percentage. Therefore, the field is ultimately classified as a "sensitive information field".

[0141] This embodiment uses multiple recognizers to work together to process and classify complex field data, and ensures the accuracy and reliability of the recognition results.

[0142] Furthermore, a third embodiment of the field classification method based on dynamic rules of the present invention is proposed. In step a1 above, the field data is identified sequentially using the preset recognizers according to the processing priority to obtain the recognition result corresponding to each preset recognizer, including:

[0143] Step b1: According to the processing priority, the preset recognizer is used to recognize the field data in sequence;

[0144] Step b2: When the field data is identified using the sample rule recognizer, a preset sample rule is loaded through the sample rule recognizer;

[0145] Step b3: Traverse the sample dataset of the preset sample rules and perform a complete match judgment with the field data. If the complete match judgment is successful, end the recognition task of the sample rule recognizer and obtain the recognition result of the sample rule recognizer.

[0146] In this embodiment, four recognizers (group sensitivity recognizer, sample rule recognizer, intelligent model recognizer, and regularity rule recognizer) are sequentially invoked according to the previously set processing priority to identify field data. Each recognizer analyzes and processes the field data based on its specific recognition rules and algorithms to generate corresponding recognition results.

[0147] During the processing of field data, if the current recognizer is a sample rule recognizer, the system will match and recognize the field data according to preset sample rules. The sample rule recognizer loads predefined sample rules, which may include common data patterns, field name and content matching rules, etc. The sample rule recognizer traverses the dataset of sample rules one by one and compares each rule with the field data. If the field data matches a rule exactly (a complete match is successful), the field data is recognized as belonging to the category that conforms to that rule.

[0148] If a field's data perfectly matches a preset sample rule, the recognition task will terminate immediately. The sample rule recognizer considers the field successfully recognized and will not process any other rules or data. This result will be recorded and used in subsequent field classification processes.

[0149] A specific implementation method could be as follows: First, determine the processing priority of the recognizers. For example, if the system detects that field data may involve sensitive group information, the group sensitivity recognizer is invoked first. After that, the system will invoke other recognizers in sequence, including the sample rule recognizer, the intelligent model recognizer, and the regular expression rule recognizer.

[0150] When the sample rule recognizer is invoked, the system loads all preset rules that may match the current field data. The sample rule recognizer iterates through these rules and performs a complete match check against the field data one by one. For example, if the field name and content exactly match the requirements of a sample rule, the match is considered successful. At this point, the sample rule recognizer's task ends, and the matching result is taken as the final recognition result.

[0151] Example explanation:

[0152] Suppose there is a field "customer_email". The system first determines that this field may be related to sensitive corporate information and uses a corporate sensitivity identifier for preliminary judgment. Next, the sample rule identifier is invoked and a set of sample rules related to email are loaded, such as "the field name contains 'email' and the content conforms to the email format".

[0153] The sample rule recognizer iterates through all preset rules and finds that the "customer_email" field name and content exactly match one of the rules. Upon successful matching, the sample rule recognizer immediately terminates further processing and marks the recognition result as an "email field". This result will be used in subsequent steps for field classification or categorization.

[0154] This embodiment ensures that field data can be classified quickly and accurately through the above steps, especially when there are explicit matching rules. By terminating the recognition task early, the system can save processing resources and improve overall efficiency.

[0155] Furthermore, a fourth embodiment of the field classification method based on dynamic rules of the present invention is proposed. In step a1 above, the field data is identified sequentially using the preset recognizers according to the processing priority to obtain the recognition result corresponding to each preset recognizer, including:

[0156] Step c1: According to the processing priority, the preset recognizer is used to recognize the field data in sequence;

[0157] Step c2: When the intelligent model recognizer is used to recognize the field data, the intelligent model recognizer applies the built-in preset naming recognition rules to recognize the field data and obtain the recognition result of the intelligent model recognizer.

[0158] In this embodiment, the group sensitivity identifier, sample rule identifier, intelligent model identifier, and regular expression rule identifier are invoked sequentially according to a set priority order. These identifiers will analyze the field data according to their respective rules and algorithms to generate corresponding identification results.

[0159] When processing field data, if the current call is to the intelligent model recognizer, the recognizer will be activated and its unique intelligent algorithms and rules will be applied to recognize the field data.

[0160] The intelligent model recognizer will use its built-in preset naming rules to identify field data. These naming rules are typically based on models generated from large amounts of training data and are used to accurately identify the type or category of a field. For example, the intelligent model recognizer might use these rules to identify whether a field belongs to a category such as "name," "address," or "organization name."

[0161] A specific implementation method could be as follows: First, based on the characteristics and processing priority of the field data, different recognizers are called sequentially. Assuming the system determines that the intelligent model recognizer is one of the higher-priority recognizers, then when it is called, the intelligent model recognizer will load and apply its built-in preset naming recognition rules.

[0162] These rules may include analysis of field names and pattern recognition of field content. For example, an intelligent model recognizer might contain a pre-defined rule base that can identify field data appearing in the format "John Doe" as a name field. If the field data matches these rules, the intelligent model recognizer will generate the corresponding recognition result.

[0163] Example explanation:

[0164] Suppose the system is processing a field "company_name". Based on processing priority, the intelligent model recognizer is invoked. The intelligent model recognizer applies its built-in preset naming recognition rules, such as "field name contains 'company' or 'organization'", and further analyzes whether the field content conforms to a specific pattern of company or organization names.

[0165] If the intelligent model recognizer successfully identifies that the content of the "company_name" field matches the criteria for a company name according to the rules, it will output the recognition result, classifying the field as a "company name field". This recognition result will be used for further classification and processing of the field.

[0166] This embodiment utilizes the powerful analytical capabilities of the intelligent model recognizer through the above steps to effectively identify and classify complex field data, especially in scenarios where traditional rules are difficult to cover.

[0167] Furthermore, a fifth embodiment of the field classification method based on dynamic rules of the present invention is proposed. In step c2 above, the field data is identified by applying the built-in preset naming recognition rules through the intelligent model recognizer, including:

[0168] Step d1: Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain the new recognition rule;

[0169] Step d2: The field data is identified by the smart recognizer using the built-in preset naming recognition rules and the newly added recognition rules.

[0170] In this embodiment, the intelligent model recognizer is a machine learning or artificial intelligence-based module used to analyze field data and identify its category or features. By applying built-in naming recognition rules, the intelligent model recognizer can automatically identify field data, such as determining whether field data represents a name, address, organization name, etc. The preset naming recognition rules are models built based on training data and are used to identify common field types.

[0171] In some cases, existing pre-defined naming and recognition rules may be insufficient to identify new or domain-specific field data. To adapt to new needs, new recognition rules can be created. These rules can identify specific types of field data by defining new features or conditions.

[0172] Positive sample data consists of field data that meets the conditions of the new recognition rule, while negative sample data consists of data that does not meet the conditions. When training the new recognition rule, the system needs to use this sample data to learn and optimize the rule, thereby improving the accuracy of the recognition.

[0173] Positive and negative sample data are input into the training algorithm to adjust and optimize the parameters of the new recognition rules. Through training, the system can learn how to accurately distinguish between field data that conforms to the new rules and new data that does not.

[0174] Through the training process, the system generates and optimizes new recognition rules. These rules can now accurately identify new types or domain-specific field data. The intelligent model recognizer will simultaneously apply both built-in preset naming recognition rules and newly created recognition rules to process field data. Combining these two types of rules, the recognizer can more comprehensively and accurately identify the type or characteristics of field data.

[0175] A specific implementation method could be as follows: First, the field data is identified using the built-in preset rules of the intelligent model recognizer. If it is found that the existing rules cannot accurately identify certain fields, the system will enter the new rule creation process. First, the system administrator or developers will collect relevant positive and negative sample data, which will be used to train the new recognition rules.

[0176] The training process involves feeding these sample data into a machine learning algorithm. The algorithm analyzes the features of the sample data to generate and optimize new recognition rules. After training, the system combines these new recognition rules with existing rules to form a more powerful recognizer. When processing field data, the recognizer will apply both built-in and new rules simultaneously to ensure accurate recognition of all types of field data.

[0177] Example explanation:

[0178] Suppose the system is processing a new set of field data that doesn't perfectly match existing recognition rules. For example, a new field, "user_code," doesn't conform to existing rules in either its name or content. To address this, the system creates a new recognition rule.

[0179] First, the system administrator collected sample data, including fields similar to "user_code" (positive samples) and unrelated fields (negative samples). Next, the system used this sample data for training, generating a new rule capable of recognizing fields like "user_code". Finally, the system combined this new rule with the preset rules of the intelligent model recognizer and applied it to all new field data. When processing "user_code" or similar fields again, the system can accurately identify and classify these fields because it relies not only on the preset rules but also utilizes the new recognition rules specifically created for these fields.

[0180] This embodiment ensures the system's flexibility and adaptability through the above steps, enabling it to cope with different data scenarios.

[0181] Furthermore, a sixth embodiment of the field classification method based on dynamic rules of the present invention is proposed. In step S30 above, rapid feature detection is performed on the field data, and the processing priority of the preset recognizer is set according to the detection result. The preset recognizer is then used to identify the field data sequentially according to the processing priority, including:

[0182] Step e1: Detect whether the first preset number of characters of the field data have JSON text characteristics;

[0183] Step e2: If the field data has JSON text features, convert the field data into a JSON object, perform fast feature detection on the JSON object, set the processing priority of the preset recognizer according to the detection results, and use the preset recognizer to recognize the JSON object in sequence according to the processing priority.

[0184] Step e3: If there are no JSON text features, extract the second preset number of most frequent word groups from the field data, perform fast feature detection on the second preset number of word groups, set the processing priority of the preset recognizer according to the detection results, and use the preset recognizer to recognize the second preset number of word groups in sequence according to the processing priority.

[0185] In this embodiment, the characteristics of the field data are first quickly analyzed to determine whether it has a specific format or content structure. This feature detection may include checking whether the field data contains specific types of characters, whether it conforms to a specific data format (such as JSON format), or the length and frequency distribution of the field data.

[0186] Based on the feature detection results, the system assigns processing priorities to different preset recognizers. The priority determines the order in which the recognizers process the field data, ensuring that the most suitable recognizer is processed first to obtain the best results.

[0187] The system first checks if the first few characters of the field data (i.e., the "first preset number of characters") have characteristics of JSON format, such as whether the field data begins with "{" or "[". This is a quick preliminary check to determine if the field data is likely a JSON object. Generally, this check is for large text field data.

[0188] If the system detects that the field data has JSON text characteristics, it will parse it into a JSON object for further analysis and processing. This conversion helps the system better understand the structure and content of the field data. After converting the field data into a JSON object, the system performs another rapid feature detection. This detection analyzes the internal data structure, key-value pairs, nesting levels, etc., of the JSON object to determine the most suitable processing strategy.

[0189] If the field data does not possess JSON text characteristics, the system extracts the most frequently occurring word groups (such as the first two preset number of word groups) for further feature analysis. This method is used to process unstructured or semi-structured field data, obtaining the main content and features of the data through word frequency analysis. The system performs rapid feature detection on the extracted high-frequency word groups, analyzing their type, structure, and distribution within the field data. This step helps identify the potential categories or characteristics of the field data.

[0190] Whether targeting JSON objects or high-frequency phrases, the system assigns processing priorities to the recognizers based on the detection results. This prioritization ensures that the most relevant recognizers are processed first, improving recognition accuracy. Ultimately, the system sequentially calls different recognizers according to the set priority order to process the field data or converted JSON objects and generate the recognition results.

[0191] A specific implementation method could be as follows: First, a preliminary analysis of the field data is performed. If the first few characters of the field data are detected to have JSON text characteristics, the system will parse it into a JSON object and perform further analysis on that object. By performing feature detection on the key-value pair structure and content of the JSON object, the system assigns priorities to subsequent recognizer processing steps.

[0192] If the field data does not possess JSON text characteristics, the system will extract the most frequently occurring phrases from the field data and perform feature detection on these phrases. For example, the system might detect whether these phrases belong to a specific domain of terminology or their grammatical function in the text. Based on these detection results, the system determines the order in which the recognizers are invoked to ensure that the data is processed most efficiently.

[0193] Example explanation:

[0194] Suppose the system is processing a field with the content "{'name':'Alice','age':30,'address':{'city':'NewYork','zip':'10001'}}". The system first checks the first few characters of the field data, finding it conforms to JSON format, and therefore parses it into a JSON object. Next, the system performs feature detection on the key-value pairs of the JSON object, discovering that the field contains the common "name" and "address" structures, and therefore prioritizes calling the intelligent model recognizer to process it.

[0195] If the field data is unstructured text, such as "some text content frequently contains certain keywords," the system will extract the most frequently occurring keywords and perform feature analysis on them. Assuming "keyword 1" and "keyword 2" are the most frequently occurring phrases, the system will set the processing priority of the recognizer based on these features, prioritizing the recognition function that can identify these phrases for further analysis.

[0196] This embodiment dynamically adjusts the calling order of the recognizer based on the characteristics of the field data through the above steps, so as to achieve more efficient field recognition and classification.

[0197] Furthermore, a seventh embodiment of the field classification method based on dynamic rules of the present invention is proposed. In step S40 above, after generating the field classification result based on the metadata processing result, the method further includes:

[0198] Step f1: Obtain user feedback information and adjust the preset matching rules and preset logical processing rules based on the user feedback information.

[0199] In this embodiment, after generating the field classification results, user feedback is further collected and analyzed. User feedback may include evaluations of the accuracy and applicability of the classification results, or suggestions and opinions from users regarding the system's identification rules. Through this feedback, the system can identify problems encountered in actual use and potential areas for improvement.

[0200] Based on user feedback, existing preset matching rules and preset logical processing rules have been adjusted and optimized. Preset matching rules are used for initial screening of metadata, while preset logical processing rules are used for further logical operations on the target metadata. By adjusting these rules, the system can improve the accuracy and consistency of the classification results and better adapt to user needs and usage environments.

[0201] A specific implementation method could be as follows: After completing the field classification, the system proactively acquires user feedback. This feedback can be obtained in various forms, such as user-inputted evaluations, analysis of system usage logs, or records of users manually adjusting specific classification results. After collecting this feedback, the system uses it to analyze the effectiveness of existing preset matching rules and logical processing rules.

[0202] Based on the analysis results, the system may find that some matching rules are too strict or too lenient, or that the logical processing rules fail to adequately capture the characteristics of the field data. In this case, the system will adjust these rules. For example, the system may modify the matching rules to better suit specific types of data, or adjust the priority of the logical processing rules to improve the overall accuracy of the classification.

[0203] Example explanation:

[0204] Suppose that when the system processes a newly added field "transaction_code", it classifies it as a "plain text field" according to existing rules. However, in actual use, the user discovers that this field should be classified as a "sensitive data field" because it involves transaction information. The user submits feedback through the system interface pointing out this error.

[0205] After receiving feedback, the system analyzed existing matching and logical processing rules and found that the existing rules failed to accurately identify "transaction_code" as sensitive information. Based on the feedback, the system adjusted the matching rules to make them more likely to identify fields containing the word "transaction" and increased the priority of related rules. After the adjustment, the system will more accurately classify similar fields as "sensitive data fields" the next time it processes them.

[0206] Through the above steps, this embodiment can continuously optimize its recognition and classification capabilities, ensuring that the field classification results can better meet the actual needs of users.

[0207] The present invention also provides a field hierarchical device based on dynamic rules.

[0208] Reference Figure 3 , Figure 3 This is a functional module diagram of the first embodiment of the field classification device based on dynamic rules of the present invention. The field classification device based on dynamic rules includes:

[0209] Data acquisition module 10 acquires field data and metadata corresponding to the field data, wherein the metadata is used to define the structure, type and constraints of the corresponding field data;

[0210] The metadata analysis and processing module 20 analyzes the metadata according to preset matching rules, filters out target metadata, and performs logical operations on the target metadata according to preset logical processing rules to obtain metadata processing results.

[0211] The field data analysis module 30 performs rapid feature detection on the field data, sets the processing priority of the preset recognizer according to the detection results, and uses the preset recognizer to recognize the field data in sequence according to the processing priority to obtain the field data recognition result.

[0212] The hierarchical decision module 40 generates a field hierarchical result based on the metadata processing result if the field data identification result is empty; and generates a field hierarchical result based on the field data identification result if the field data identification result is not empty.

[0213] Furthermore, embodiments of the present invention also propose a computer storage medium.

[0214] The storage medium stores a field classification program based on dynamic rules. When the processor executes the field classification program based on dynamic rules, it implements the steps of the field classification method based on dynamic rules as described above.

[0215] The specific implementation of the storage medium of the present invention is basically the same as the above-described embodiments of the field hierarchical method based on dynamic rules, and will not be repeated here.

[0216] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many modifications under the guidance of the present invention without departing from the spirit and scope of the claims. All equivalent structural or procedural transformations made using the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are within the protection scope of the present invention.

[0217] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0218] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0219] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0220] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use.

Claims

1. A field hierarchical method based on dynamic rules, characterized in that, Includes the following steps: Obtain the field data and the corresponding metadata, whereby the metadata is used to define the structure, type, and constraints of the corresponding field data; The metadata is analyzed according to preset matching rules, target metadata is filtered out, and logical operations are performed on the target metadata according to preset logical processing rules to obtain metadata processing results. The process involves: performing rapid feature detection on the field data; setting the processing priority of a preset recognizer based on the detection results; and sequentially using the preset recognizer to recognize the field data according to the processing priority to obtain the field data recognition result. This includes: detecting whether the first preset number of characters in the field data have JSON text features; if they have JSON text features, converting the field data into a JSON object; performing rapid feature detection on the JSON object; setting the processing priority of the preset recognizer based on the detection results; and sequentially using the preset recognizer to recognize the JSON object according to the processing priority; if they do not have JSON text features, extracting the second preset number of most frequent word groups from the field data; performing rapid feature detection on the second preset number of word groups; setting the processing priority of the preset recognizer based on the detection results; and sequentially using the preset recognizer to recognize the second preset number of word groups according to the processing priority. If the field data identification result is empty, then a field classification result is generated based on the metadata processing result; If the field data recognition result is not empty, then a field classification result is generated based on the field data recognition result.

2. The field hierarchical method based on dynamic rules as described in claim 1, characterized in that, According to the processing priority, the preset recognizer is used sequentially to recognize the field data, and the field data recognition result is obtained, including: According to the processing priority, the preset recognizers are used sequentially to recognize the field data to obtain the recognition result corresponding to each preset recognizer. The preset recognizers include a group sensitive recognizer, a sample rule recognizer, an intelligent model recognizer, and a regular expression rule recognizer. The proportion of recognition results corresponding to each preset recognizer is calculated using a proportion formula, and the recognition result with the highest proportion is taken as the recognition result of the field data.

3. The field hierarchical method based on dynamic rules as described in claim 2, characterized in that, According to the processing priority, the preset recognizers are used sequentially to recognize the field data, and the recognition results corresponding to each preset recognizer are obtained, including: The preset recognizer is used to recognize the field data sequentially according to the processing priority; When the sample rule recognizer is used to recognize the field data, a preset sample rule is loaded through the sample rule recognizer. The sample dataset of the preset sample rules is traversed and a complete match is performed with the field data. If the complete match is successful, the recognition task of the sample rule recognizer ends and the recognition result of the sample rule recognizer is obtained.

4. The field hierarchical method based on dynamic rules as described in claim 2, characterized in that, According to the processing priority, the preset recognizers are used sequentially to recognize the field data, and the recognition results corresponding to each preset recognizer are obtained, including: The preset recognizer is used to recognize the field data sequentially according to the processing priority; When the intelligent model recognizer is used to recognize the field data, the intelligent model recognizer applies built-in preset naming recognition rules to recognize the field data, and obtains the recognition result of the intelligent model recognizer.

5. The field hierarchical method based on dynamic rules as described in claim 4, characterized in that, The intelligent model recognizer uses built-in preset naming recognition rules to recognize the field data, including: Create a new recognition rule, obtain positive and negative sample data, and use the positive and negative sample data to train the new recognition rule to obtain the new recognition rule; The intelligent model recognizer uses the built-in preset naming recognition rules and the newly added recognition rules to recognize the field data.

6. The field hierarchical method based on dynamic rules as described in claim 1, characterized in that, After generating the field classification results based on the metadata processing results, the process also includes: Obtain user feedback information and adjust the preset matching rules and preset logic processing rules based on the user feedback information.

7. A field hierarchical device based on dynamic rules, characterized in that, The field classification device based on dynamic rules includes: The data acquisition module acquires field data and the metadata corresponding to the field data. The metadata is used to define the structure, type, and constraints of the corresponding field data. The metadata analysis and processing module analyzes the metadata according to preset matching rules, filters out target metadata, and performs logical operations on the target metadata according to preset logical processing rules to obtain the metadata processing result. The field data analysis module performs rapid feature detection on the field data, sets the processing priority of a preset recognizer based on the detection results, and sequentially uses the preset recognizer to recognize the field data according to the processing priority to obtain field data recognition results. This includes: detecting whether the first preset number of characters in the field data have JSON text features; if they have JSON text features, converting the field data into a JSON object, performing rapid feature detection on the JSON object, setting the processing priority of the preset recognizer based on the detection results, and sequentially using the preset recognizer to recognize the JSON object according to the processing priority; if they do not have JSON text features, extracting the second preset number of most frequent word groups from the field data, performing rapid feature detection on the second preset number of word groups, setting the processing priority of the preset recognizer based on the detection results, and sequentially using the preset recognizer to recognize the second preset number of word groups according to the processing priority. The hierarchical decision module generates a field hierarchical result based on the metadata processing result if the field data identification result is empty; otherwise, it generates a field hierarchical result based on the field data identification result.

8. A field hierarchical device based on dynamic rules, characterized in that, The dynamic rule-based field classification device includes a memory, a processor, and a dynamic rule-based field classification program stored in the memory and executable on the processor. When the dynamic rule-based field classification program is executed by the processor, it implements the steps of the dynamic rule-based field classification method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The storage medium stores a field classification program based on dynamic rules, which, when executed by a processor, implements the steps of the field classification method based on dynamic rules as described in any one of claims 1-6.

Images