Data classification and processing dynamic optimization method and related products applicable to OA systems

Abstract

The present invention relates to the field of data processing technology, and in particular to a data classification processing dynamic optimization method suitable for an OA system and related products, the method comprising: processing input raw data to generate structured data; generating a classification rule base according to user-defined classification requirements and structured data; classifying data using an adapted classification model to generate classification results; optimizing the classification rule base and adjusting the classification model; outputting the classification results, the optimized classification rule base and the updated classification model; the present invention solves the problem of heterogeneous data processing by introducing a specific feature extraction strategy and a format standardization method for multimodal data; dynamically optimizing the classification rule base in combination with real-time feedback, so that the system can quickly respond to changes in business scenarios and requirements, and improves the accuracy and logical consistency of classification; and dynamically adjusting through online learning to enhance the adaptability of the classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data classification processing dynamic optimization method applicable to an OA system and related products. Background Art

[0002] With the popularization of office automation (OA) systems, the management and classification of massive heterogeneous data has become one of the core needs. OA systems often involve multimodal data such as text, pictures, and videos. These data structures are complex and the formats are diverse. Traditional static classification methods are difficult to guarantee classification accuracy and efficiency when processing such data. At the same time, due to the dynamic changes in user needs, classification standards and classification rule bases need to have the ability to adjust in real time to meet the diverse needs of different scenarios.

[0003] At present, most OA systems use fixed classification rules or classification methods based on pre-trained models. However, these methods have significant limitations when facing multimodal data. The fixed rule base cannot effectively adapt to changes in data characteristics, resulting in reduced classification accuracy. The classification method based on pre-trained models cannot meet the classification requirements in complex and changing business scenarios due to the lack of real-time response capabilities to user needs. In addition, existing classification methods usually lack comprehensive management of the logical relationship between rules, which may lead to conflicts or redundancies between rules, further reducing the classification efficiency of the system.

[0004] In terms of dynamic optimization of classification tasks, existing technologies mainly focus on offline model updates and lack a mechanism for online learning based on user feedback. This makes it difficult for the classification system to correct misclassifications or optimize classification model parameters in a timely manner, which in turn limits the system's adaptability and intelligence. In addition, for the preprocessing of multimodal data, existing technologies usually use general standardized processing methods and fail to design specific feature extraction strategies for different data types, which has become one of the bottlenecks in improving classification performance.

[0005] In summary, efficient classification of multimodal data in OA systems and dynamic optimization of the classification rule base is a technical problem that needs to be solved urgently in the current technical field. Summary of the invention

[0006] The technical problem to be solved by the present invention is the low classification accuracy of multimodal data in existing office automation (OA) systems, the lack of dynamic adaptability of the rule base, and the difficulty in real-time optimization of the classification model based on user feedback. The present invention aims to provide a dynamic optimization method for data classification processing suitable for OA systems and related products, which realize the efficient classification of multimodal data, dynamic optimization of the classification rule base, and online learning and real-time adjustment of the classification model.

[0007] The present invention is achieved through the following technical solutions:

[0008] A data classification processing dynamic optimization method applicable to an OA system, comprising:

[0009] Data preprocessing: extract features, clean data, and standardize the format of the input raw data to generate structured data;

[0010] Classification rule generation: Based on user-defined classification requirements and structured data, natural language processing technology is used to parse the requirements, and a classification rule library is generated through a machine learning model;

[0011] Data classification execution: Assign classification tasks to structured data according to the classification rule base, classify the data using the adapted classification model, and generate classification results;

[0012] Dynamic optimization: Based on the classification results and user feedback, the classification rule base is optimized through real-time feedback collection, and the classification model is adjusted through online learning algorithms;

[0013] Output results: Output classification results, optimized classification rule base and updated classification model.

[0014] Specifically, the method for data preprocessing includes:

[0015] Automatically extract features based on the type of raw data; extract keyword features or semantic features from text data, edge features from image data, and key frame features from video data;

[0016] The extracted features are used to remove outliers and fill in missing values ​​to obtain cleaned multi-source heterogeneous data and form structured data.

[0017] Specifically, the method for feature extraction includes:

[0018] The input raw data is processed into a unified format. The raw data is unprocessed multimodal data, including text data, image data and video data; the text data is uniformly converted into UTF-8 encoding format; the image data is standardized to the target resolution by using the resolution adjustment function of the bilinear interpolation algorithm for image scaling; the video data is aligned to the time series through dynamic time warping to generate a frame sequence of uniform length;

[0019] Text data is analyzed through TF-IDF and sentence vector model Extract feature vectors to obtain text data The eigenvector of , ,in, is the feature vector based on word frequency, is the context feature vector generated by the sentence vector model, and is the weight parameter;

[0020] Image data is processed through multi-scale edge detection and convolutional neural network Extract feature vectors and obtain image data The eigenvector of , ,in, For edge features extracted by multi-scale edge detection, is the deep feature extracted by convolutional neural network. and is the weight parameter;

[0021] Video data is processed through key frame extraction and 3D convolutional network Extract feature vectors and obtain video data The eigenvector of , ,in, For the Keyframes The edge features of The deep features extracted by the 3D convolutional network based on time series features, and is the weight parameter;

[0022] Get structured data .

[0023] Specifically, the classification rule generation method includes:

[0024] Use natural language processing technology to convert user needs into classification conditions And the classification results ;

[0025] Build a set of logical rules based on classification requirements: ,in, is the initial rule set, For the Classification rules, , is the input feature;

[0026] Construct a rule logic relationship tree that represents the relevance and conditional consistency between rules through dependency syntax analysis ;

[0027] Defining the state space ,state For the rule set in The current state in iteration ; ; For the The rule set for the iteration; is the classification accuracy calculated by cross-validation, is the rule complexity, The internal consistency of the rule logic relationship tree;

[0028] Defining the action space ,action For the rule set Modifications include adjusting rule condition thresholds, deleting redundant rules, and merging classification results; is the total number of adjustment actions;

[0029] Design a reward function that evaluates the impact of each action on the rule set , ,in, , and is the weight parameter, For the The modification action of each

[0030] From the current state Go, choose an action Rule Set Make adjustments and get a new set of rules , ;

[0031] Through the reward function Determine action , select the action with the highest reward function value as the optimal action to update the rule;

[0032] Iterate and update until convergence conditions are reached to obtain the final rule set And use it as the classification rule base.

[0033] Specifically, ,in, For rule conditions complexity;

[0034] , For the Sub-classification conditions, For the Sub-classification conditions, is the consistency indicator function;

[0035] Convergence conditions include: classification accuracy , The classification accuracy requirements provided to users;

[0036] Internal consistency of the logical relationship tree Above a set threshold;

[0037] Rule complexity minimize.

[0038] Specifically, the method for performing data classification includes:

[0039] Construct different classification models for different data types, and train the classification models after constructing training sets with existing data to obtain different classification models; text data The eigenvector of Corresponding semantic classification model, image data The eigenvector of Corresponding to the deep learning-based image classification model, video data The eigenvector of Corresponding time series classification model;

[0040] Determine the classification rule base , and parse the logical relationship tree in the rule set to match the structured data , each piece of data is matched with the corresponding classification rule Establish connections;

[0041] Use the logical conditions in the classification rules to assign appropriate classification models to different types of data in the structured data;

[0042] For structured data Each piece of data in , the classification results are calculated through the classification model and classification confidence ; among them, among them, For data Belongs to category The probability of The final labels for classifying the data;

[0043] Generate classification results .

[0044] Specifically, the method for dynamic optimization includes:

[0045] Get classification results ,in, For the data in structured data, is the final label for data classification, is the classification confidence;

[0046] Obtaining feedback data from users ,in, is the input feature, Classification results for users amendments;

[0047] Label the misclassified sample set , and generate classification error distribution;

[0048] Dynamically adjust the classification conditions in the classification rule base and update the classification conditions : ,in, To adjust the rule threshold according to feedback data and error distribution; to obtain an optimized classification rule base ;

[0049] For the set of misclassified samples Perform feature extraction and update the training set : ;

[0050] Perform online learning on the classification model based on the training set and use gradient update to optimize model parameters , ,in, is the learning rate, is the model loss function, is the gradient of the loss function;

[0051] Iterative optimization to obtain the final model parameters , and based on the optimized model parameters Update the classification model.

[0052] Specifically, adjust the rule threshold according to the feedback data and error distribution The methods of obtaining include:

[0053] Determine rule error weights , ,in, For misclassified samples, the rule The number of misclassified samples, is the total number of misclassified samples;

[0054] Targeting rules Classification conditions , determine the adjustment rule threshold , ,in, To adjust the learning rate, For rules The current classification threshold of Misclassified samples The direction of deviation from the current threshold, .

[0055] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the data classification processing dynamic optimization method applicable to OA systems as described above.

[0056] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the dynamic optimization method for data classification processing applicable to OA systems as described above.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] The present invention performs feature extraction, data cleaning and format standardization on the input multimodal raw data (text, pictures, videos, etc.) to generate unified structured data; combines natural language processing technology to parse user-defined classification requirements, builds an initial classification rule base, and optimizes the classification rule set through a machine learning model to form a classification rule base that adapts to dynamic scenarios; in the data classification execution process, according to the logical structure and classification conditions of the classification rule base, allocates an adaptive classification model (such as a BERT model, a CNN model, and a 3D-CNN model) to the multimodal data for classification, and generates a classification result including a classification label and confidence level; collects user feedback in real time, generates an error distribution in combination with misclassified samples, dynamically optimizes the classification rule base and the classification model, and adjusts the model parameters using an online learning algorithm.

[0059] The present invention solves the problem of heterogeneous data processing by introducing specific feature extraction strategies and format standardization methods for multimodal data; dynamically optimizes the classification rule base in combination with real-time feedback, so that the system can quickly respond to changes in business scenarios and needs, and improves the accuracy and logical consistency of classification; and dynamically adjusts through online learning, thereby enhancing the adaptability of the classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings illustrate exemplary embodiments of the present invention and, together with the description thereof, are used to explain the principles of the present invention. These drawings are included to provide a further understanding of the present invention, and the accompanying drawings are included in and constitute a part of this specification and do not constitute a limitation of the embodiments of the present invention.

[0061] Figure 1 It is a flow chart of the dynamic optimization method for data classification processing applicable to OA system according to the present invention.

[0062] Figure 2 It is a flowchart of the method for generating classification rules according to the present invention. DETAILED DESCRIPTION

[0063] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and implementation methods. It is understood that the specific implementation methods described herein are only used to explain the relevant content, rather than to limit the present invention.

[0064] It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the drawings.

[0065] In the absence of conflict, the embodiments and features of the embodiments of the present invention may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0066] Embodiment 1, as Figure 1 As shown, a data classification processing dynamic optimization method applicable to an OA system includes:

[0067] Data preprocessing: extract features, cleanse data, and standardize the format of the input raw data to generate structured data.

[0068] Feature extraction refers to extracting key features through algorithms based on data types (text, images, videos, etc.). For example: extracting keywords or semantic features from text data; extracting edge features from image data; and extracting key frame features from video data.

[0069] Data cleaning refers to removing noisy data through anomaly detection and filling in missing values ​​reasonably to ensure data quality.

[0070] Format standardization refers to converting heterogeneous data (different formats or structures) into unified structured data for subsequent processing.

[0071] Classification rule generation: Based on user-defined classification requirements and structured data, natural language processing technology is used to parse the requirements, and a classification rule library is generated through a machine learning model.

[0072] Natural language processing (NLP) technology is used to parse the classification requirements provided by users. NLP refers to the technology that enables computers to understand and generate natural language through algorithms, including semantic analysis, named entity recognition and other functions.

[0073] The machine learning model can generate a classification rule base and dynamically optimize the rules. The initial rule base forms the logical relationship of the rules by analyzing user needs; the machine learning model adjusts the rule conditions according to the data distribution to make the rule base more suitable for actual classification needs.

[0074] Data classification execution: Assign classification tasks to structured data according to the classification rule base, classify the data using the adapted classification model, and generate classification results.

[0075] The role of the classification rule base is that the classification rule base defines the logical conditions that each type of data should meet, such as "image resolution > 1080p and contains specific edge features."

[0076] Selection of classification model: Assign an appropriate model to each type of data based on the data type. For example, a semantic classification model (such as BERT) is used for text data; a model based on a convolutional neural network (CNN) is used for image data; and a time series classification model (such as 3D-CNN) is used for video data. The classification process determines the classification result and confidence level of each piece of data through probability calculation.

[0077] Dynamic optimization: Based on the classification results and user feedback, the classification rule base is optimized through real-time feedback collection, and the classification model is adjusted through an online learning algorithm.

[0078] Collect classification results and user-corrected feedback data, mark misclassified samples, and generate error distribution. Dynamically adjust classification conditions. For example, when a rule leads to a high misclassification rate, optimize the rule by adjusting the classification threshold. Add misclassified samples to the training set, use incremental learning methods to optimize classification model parameters, and continuously improve classification performance.

[0079] Output results: Output classification results, optimized classification rule base and updated classification model.

[0080] The core of this method lies in data streaming and iterative optimization:

[0081] After the raw data is input, the preprocessing module standardizes it into unified structured data; the classification rule generation module builds a classification rule base by combining user needs and machine learning models; the data classification execution module assigns classification models according to the rule base and completes classification; the dynamic optimization module uses real-time feedback to adjust the rule base and classification model to form a closed-loop optimization; the output classification results and the updated content of the optimization module will serve as the starting point for new tasks to achieve continuous evolution of the system.

[0082] Embodiment 2 automatically extracts features according to the type of original data; extracts keyword features or semantic features from text data, extracts edge features from image data, and extracts key frame features from video data.

[0083] The purpose of feature extraction is to simplify data representation while retaining the most important information for the classification task. The following are feature extraction methods for different data types:

[0084] Text data:

[0085] Extract keyword features: Extract keywords from text by calculating term frequency-inverse document frequency (TF-IDF) and other methods to highlight the topic features of the text.

[0086] Extracting semantic features: Using semantic embedding models (such as BERT) to generate context-dependent semantic feature vectors to capture the deep meaning of the text.

[0087] Image data:

[0088] Extract edge features: Capture the contour information of the image through multi-scale edge detection algorithms (such as Canny edge detection). Edge features reflect the structure of salient objects in the image.

[0089] Video data:

[0090] Extract key frame features: Identify key frames in the video through dynamic time warping or motion detection methods, and further extract image features such as edge features or depth features from the key frames.

[0091] The extracted features are used to remove outliers and fill missing values. The extracted features may contain outliers or missing values. These problems need to be processed through data cleaning steps to improve data quality. Data cleaning method:

[0092] Outlier removal: Detect and remove abnormal feature values ​​that deviate from the normal range using statistical methods (such as mean and standard deviation) or machine learning algorithms (such as isolation forest).

[0093] Missing value filling: For missing feature values, the following method is used to fill them:

[0094] Mean or median filling: logarithmic data;

[0095] Predictive filling: predict missing values ​​using regression models;

[0096] Interpolation method: interpolate missing values ​​of time series or image series data.

[0097] Obtain cleaned multi-source heterogeneous data and form structured data. In other words, integrate the cleaned features into structured data in a unified format as input to the classification rule generation module.

[0098] Data preprocessing combines feature extraction, data cleaning, and format standardization to transform multi-source heterogeneous raw data into high-quality, structured data in a unified format. Feature extraction ensures that key information in the data is effectively captured; data cleaning improves the credibility of features; and format standardization provides a unified representation for data of different modalities, simplifying the input processing flow of subsequent classification tasks.

[0099] Methods for feature extraction include:

[0100] The input raw data is processed in a unified format. The raw data is unprocessed multimodal data, including text data, image data and video data; the text data is uniformly converted to UTF-8 encoding format to ensure that all text data has a consistent encoding standard in different operating systems and tools to avoid parsing problems caused by inconsistent encoding. The image data is standardized to the target resolution through the resolution adjustment function of image scaling using the bilinear interpolation algorithm; the video data is aligned to the time series through dynamic time warping (DTW) to generate a frame sequence of uniform length; and time series of different lengths are dynamically matched and aligned.

[0101] The representative keyword features in the text are extracted through TF-IDF (Term Frequency-Inverse Document Frequency), and the context-related semantic features are extracted through the sentence vector model to capture the deep semantic information of the text; realize the text data Extract feature vectors to obtain text data The eigenvector of , ,in, is the feature vector based on word frequency, is the context feature vector generated by the sentence vector model, and is a weight parameter used to balance the contribution of keyword features and semantic features.

[0102] Image data is processed through multi-scale edge detection and convolutional neural network Extract feature vectors and obtain image data The eigenvector of , ,in, In order to extract edge features through multi-scale edge detection, a multi-scale edge detection algorithm (such as Canny edge detection) is used to extract edge features of the image and capture the basic structure and contour information of the image. The deep features extracted by convolutional neural networks are used to extract high-level deep features of images, including texture, shape, and semantic information, through deep learning models (such as ResNet and VGG). and are weight parameters, which control the weights of edge features and depth features respectively.

[0103] Video data is processed through key frame extraction and 3D convolutional network Perform feature vector extraction and use motion detection or entropy-based algorithms to extract key frames that best represent scene changes from video sequences. , based on the deep learning model of temporal and spatial features, extract the time series features of the video frame sequence. Get video data The eigenvector of , ,in, For the Keyframes The edge features of The deep features extracted by the 3D convolutional network based on time series features, and is a weight parameter used to control the weights of edge features and depth features.

[0104] Get structured data .

[0105] In the third embodiment, the classification rule generation method of the present invention realizes the automatic construction and optimization from user needs to the classification rule base through natural language processing, rule optimization and logic modeling. Figure 2 As shown, the specific classification rule generation method includes:

[0106] Use natural language processing technology to convert user requirements (natural language provided by users) into classification conditions And the classification results ; Classification conditions define the attribute characteristics or threshold ranges that the data must meet, such as "text contains the keyword 'contract'", "image resolution is greater than 1080p". The classification results are specific category labels for the corresponding classification, such as "contract documents" and "high-definition images".

[0107] Build a set of logical rules based on classification requirements: ,in, is the initial rule set, For the Classification rules, , is the input feature;

[0108] Construct a rule logic relationship tree that represents the relevance and conditional consistency between rules through dependency syntax analysis ; Through dependency syntax analysis technology, analyze the logical relationship between the rules in the initial rule set, generate a rule logical relationship tree, and generate a logical relationship tree Indicates the association and conditional consistency between rules. For example, some rules may be subsets of each other or have intersections.

[0109] The consistency of the logical relationship tree is used to measure the degree of logical conflict between rule conditions. , For the Sub-classification conditions, For the Sub-classification conditions, is a consistency indicator function used to judge and Is it compatible?

[0110] Defining the state space , represents the different states of the rule set during the optimization process. Each state For the rule set in The current state in iteration ; ; For the The rule set for the iteration; is the classification accuracy calculated by cross-validation, is the internal consistency of the rule logic relationship tree, is the rule complexity, ,in, For rule conditions complexity.

[0111] Define the action space to represent the set of operations that adjust the rule set. ,action For the rule set Modifications include adjusting rule condition thresholds, deleting redundant rules, and merging classification results; is the total number of adjustment actions;

[0112] Adjust rule condition threshold: Modify a rule The classification conditions can be relaxed, such as relaxing a certain numerical range.

[0113] Remove redundant rules: Remove rules that have low contribution to the classification results or have overlapping coverage.

[0114] Merge logically similar classification results: Merge rules with similar conditions and consistent classification results into simpler rules.

[0115] Each time an action is performed Finally, the reward function is used to evaluate the effect of the adjustment. The reward function comprehensively considers the classification accuracy, logical consistency and rule complexity, that is, the reward function is designed to evaluate the impact of each action on the rule set. , ,in, , and is a weight parameter used to control the priority of different indicators; For the The modification action of each

[0116] From the current state Go, choose an action Rule Set Make adjustments and get a new set of rules , ;

[0117] Through the reward function Determine action , select the action with the highest reward function value as the optimal action to update the rule;

[0118] Iterate and update until convergence conditions are reached to obtain the final rule set And use it as the classification rule base.

[0119] Convergence conditions include: classification accuracy , The classification accuracy requirements provided to users;

[0120] Internal consistency of the logical relationship tree Above a set threshold;

[0121] Rule complexity minimize.

[0122] In the fourth embodiment, the data classification execution method of the present invention realizes efficient classification of multimodal data by constructing a classification model adapted to different data types and combining the logical conditions of the classification rule base. The method for executing data classification includes:

[0123] Different classification models are constructed for different data types, and the classification models are trained after the existing data is used to construct a training set to obtain different classification models.

[0124] Text Data The eigenvector of The corresponding semantic classification model adopts the BERT model based on the Transformer architecture of deep learning, which can capture context-related semantic features and is suitable for complex text classification tasks.

[0125] Image Data The eigenvector of Corresponding to the image classification model based on deep learning, it uses the image classification model based on deep learning (such as CNN or ResNet). The CNN model can extract high-level features such as texture and shape of the image and is suitable for multi-category image recognition tasks.

[0126] Video Data The eigenvector of Corresponding time series classification model; it uses a time series classification model (such as 3D-CNN or LSTM). The 3D-CNN model processes the spatial and temporal features of the video at the same time and is suitable for complex dynamic video classification tasks.

[0127] Determine the classification rule base , and parse the logical relationship tree in the rule set to match the structured data , each piece of data is matched with the corresponding classification rule Establish connections;

[0128] According to the logical conditions in the rule base, each piece of data is assigned to the appropriate classification model for classification.

[0129] Use the logical conditions in the classification rules to assign appropriate classification models to different types of data in the structured data; Each piece of data in , the classification results are calculated through the classification model and classification confidence ;in, For data Belongs to category The probability of The final labels for the data classification.

[0130] Generate classification results .

[0131] In the fifth embodiment, the dynamic optimization method of the present invention adjusts the classification rule base and the classification model in real time by combining the classification results, user feedback data and online learning algorithms to improve the classification accuracy and adaptability of the system. The method for dynamic optimization includes:

[0132] Get classification results ,in, For the data in structured data, is the final label for data classification, is the classification confidence; it indicates the degree of confidence of the model in the classification result.

[0133] Obtaining feedback data from users ,in, is the input feature, Classification results for users The correction of (i.e. the true category);

[0134] Compare classification results and user feedback, and mark the misclassified sample set , and generates a classification error distribution; this set contains the input data that was misclassified and is used to guide rule adjustment and model optimization. The error distribution is used to analyze the characteristics of classification errors (such as the misclassification rate under specific conditions) and provide a basis for rule base optimization.

[0135] Dynamically adjust the classification conditions in the classification rule base and update the classification conditions : ,in, To adjust the rule threshold according to feedback data and error distribution; to obtain an optimized classification rule base ;

[0136] For the set of misclassified samples Perform feature extraction and update the training set : ;

[0137] Perform online learning on the classification model based on the training set and use gradient update to optimize model parameters , ,in, is the learning rate, is the model loss function, is the gradient of the loss function;

[0138] Iterative optimization to obtain the final model parameters , and based on the optimized model parameters Update the classification model. The dynamic optimization process is iterated until the following conditions are met: the classification accuracy meets user requirements; the classification rule base reaches an optimized state (logical consistency is enhanced and complexity is minimized); and the loss function of the classification model converges.

[0139] Among them, the rule threshold is adjusted according to the feedback data and error distribution The methods of obtaining include:

[0140] Determine rule error weights , ,in, For misclassified samples, the rule The number of misclassified samples, is the total number of misclassified samples; the error weight measures the influence of the rule on misclassification.

[0141] Targeting rules Classification conditions , determine the adjustment rule threshold , ,in, To adjust the learning rate, control the step size of the threshold adjustment. When the value is larger, the adjustment amplitude is larger; For rules The current classification threshold of Misclassified samples The direction of deviation from the current threshold, When the feature value of the misclassified sample Greater than the current threshold When , the function returns +1, indicating that the threshold needs to be increased; when When it is less than the current threshold, the function returns -1, indicating that the threshold needs to be lowered.

[0142] This method dynamically adjusts the threshold of the classification rule through three steps, using the proportion of misclassified samples Prioritize the adjustment of rules that have a large impact on misclassification; Determine the adjustment direction; the number of misclassified samples Normalization is performed to prevent rules with large sample sizes from being over-adjusted.

[0143] Embodiment 6. A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the dynamic optimization method for data classification processing applicable to an OA system is implemented.

[0144] Without loss of generality, computer readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instruction data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technology, CD-ROM, DVD or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory and mass storage devices can be collectively referred to as memory.

[0145] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the dynamic optimization method for data classification processing applicable to OA systems as described above.

[0146] A computer program product includes a computer program or set of instructions for performing specific tasks or implementing specific functions. These programs or instructions are designed to be executed by a processor to implement a series of predefined steps or operations. The program product may be stored in various forms of computer storage media, such as memory, hard disk, solid-state drive, optical disk or other forms of digital storage devices. It may exist in the form of compiled binary code or in the form of scripts or bytecodes that can be executed by an interpreter. The program product uses carefully designed algorithms and logical instructions to enable the processor to process data in a specific order and manner to complete various functions such as data analysis, user interaction, device control, etc.

[0147] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0148] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0149] It should be understood by those skilled in the art that the above embodiments are only for the purpose of clearly illustrating the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, other changes or modifications may be made based on the above invention, and these changes or modifications are still within the scope of the present invention.

Claims

1. A data classification processing dynamic optimization method applicable to OA system, characterized in that: include: Data preprocessing: extract features, clean data, and standardize the format of the input raw data to generate structured data; Classification rule generation: Based on user-defined classification requirements and structured data, natural language processing technology is used to parse the requirements, and a classification rule library is generated through a machine learning model; Data classification execution: Assign classification tasks to structured data according to the classification rule base, classify the data using the adapted classification model, and generate classification results; Dynamic optimization: Based on the classification results and user feedback, the classification rule base is optimized through real-time feedback collection, and the classification model is adjusted through online learning algorithms; Output classification results: Output optimized classification rule base and updated classification model; The methods for generating classification rules include: Use natural language processing technology to convert user needs into classification conditions And the classification results ; Build a set of logical rules based on classification requirements: ,in, is the initial rule set, For the Classification rules, , is the input feature; Construct a rule logic relationship tree that represents the relevance and conditional consistency between rules through dependency syntax analysis ; Defining the state space ,state For the rule set in The current state in iteration ; ; For the The rule set for the iteration; is the classification accuracy calculated by cross-validation, is the rule complexity, The internal consistency of the rule logic relationship tree; Defining the action space ,action For the rule set Modifications include adjusting rule condition thresholds, deleting redundant rules, and merging classification results; is the total number of adjustment actions; Design a reward function that evaluates the impact of each action on the rule set , ,in, , and is the weight parameter, For the The modification action of each From the current state Start, choose an action Rule Set Make adjustments and get a new set of rules , ; Through the reward function Determine action , select the action with the highest reward function value as the optimal action to update the rule; Iterate and update until convergence conditions are reached to obtain the final rule set And use it as the classification rule base.

2. The data classification processing dynamic optimization method applicable to OA system according to claim 1 is characterized in that: Methods for data preprocessing include: Automatically extract features based on the type of raw data; extract keyword features or semantic features from text data, edge features from image data, and key frame features from video data; The extracted features are used to remove outliers and fill in missing values ​​to obtain cleaned multi-source heterogeneous data and form structured data.

3. The data classification processing dynamic optimization method applicable to OA system according to claim 2 is characterized in that: Methods for feature extraction include: The input raw data is processed into a unified format. The raw data is unprocessed multimodal data, including text data, image data and video data; the text data is uniformly converted into UTF-8 encoding format; the image data is standardized to the target resolution by using the resolution adjustment function of the bilinear interpolation algorithm for image scaling; the video data is aligned to the time series through dynamic time warping to generate a frame sequence of uniform length; Text data is analyzed through TF-IDF and sentence vector model Extract feature vectors to obtain text data The eigenvector of , ,in, is the feature vector based on word frequency, is the context feature vector generated by the sentence vector model, and is the weight parameter; Image data is processed through multi-scale edge detection and convolutional neural network Extract feature vectors and obtain image data The eigenvector of , ,in, For edge features extracted by multi-scale edge detection, is the deep feature extracted by convolutional neural network. and is the weight parameter; Video data is processed through key frame extraction and 3D convolutional network Extract feature vectors and obtain video data The eigenvector of , ,in, For the Keyframes The edge features of The deep features extracted by the 3D convolutional network based on time series features, and is the weight parameter; Get structured data .

4. The data classification processing dynamic optimization method applicable to OA system according to claim 1 is characterized in that: ,in, For rule conditions complexity; , For the Sub-classification conditions, For the Sub-classification conditions, is the consistency indicator function; Convergence conditions include: classification accuracy , The classification accuracy requirements provided to users; Internal consistency of logical relationship tree Above a set threshold; Rule complexity minimize.

5. The data classification processing dynamic optimization method applicable to OA system according to claim 1 is characterized in that: Methods for performing data classification include: Construct different classification models for different data types, and train the classification models after constructing training sets with existing data to obtain different classification models; text data The eigenvector of Corresponding semantic classification model, image data The eigenvector of Corresponding to the deep learning-based image classification model, video data The eigenvector of Corresponding time series classification model; Determine the classification rule base , and parse the logical relationship tree in the rule set to match the structured data , each piece of data is matched with the corresponding classification rule Establish connections; Use the logical conditions in the classification rules to assign appropriate classification models to different types of data in the structured data; For structured data Each piece of data in , calculate the classification results through the classification model and classification confidence ;in, For data Belongs to category The probability of The final labels for classifying the data; Generate classification results .

6. The data classification processing dynamic optimization method applicable to OA system according to claim 1 is characterized in that: Methods for dynamic optimization include: Get classification results ,in, For the data in structured data, is the final label for data classification, is the classification confidence; Obtaining feedback data from users ,in, is the input feature, Classification results for users amendments; Label the misclassified sample set , and generate classification error distribution; Dynamically adjust the classification conditions in the classification rule base and update the classification conditions : ,in, To adjust the rule threshold according to feedback data and error distribution; to obtain an optimized classification rule base ; For the set of misclassified samples Perform feature extraction and update the training set : ; Perform online learning on the classification model based on the training set and use gradient update to optimize model parameters , ,in, is the learning rate, is the model loss function, is the gradient of the loss function; Iterative optimization to obtain the final model parameters , and based on the optimized model parameters Update the classification model.

7. A data classification processing dynamic optimization method applicable to OA system according to claim 6, characterized in that: Adjust rule thresholds based on feedback data and error distribution The methods of obtaining include: Determine rule error weights , ,in, For misclassified samples, the rule The number of misclassified samples, is the total number of misclassified samples; Targeting rules Classification conditions , determine the adjustment rule threshold , ,in, To adjust the learning rate, For rules The current classification threshold of Misclassified samples The direction of deviation from the current threshold, .

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the dynamic optimization method for data classification processing applicable to an OA system as described in any one of claims 1 to 7 is implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by the processor, the dynamic optimization method for data classification processing applicable to the OA system as described in any one of claims 1 to 7 is implemented.

Images