Data classification and grading method and system

Abstract

The invention provides a data classification and grading method and system, and belongs to the technical field of data processing, and the method comprises the steps: obtaining the class information of to-be-processed data, and determining the grading rule information according to the class information; inputting the to-be-processed data and the grading rule information into a target cue word template to generate a target cue word; the target prompt word template is determined according to the format of the to-be-processed data; inputting the target cue word into a target grading model to obtain grading information output by the target grading model; wherein the target grading model is obtained by matching from a pre-constructed model library based on the format of the processing data. According to the method, the accuracy of data classification and grading is effectively improved by combining dynamic category recognition, target cue word generation and grading model dynamic matching based on the model library.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data classification and grading method and system. Background Art

[0002] With the rapid development of information technology, enterprises and organizations generate a large amount of data every day. In order to effectively manage and utilize this data, data classification and grading become particularly important. Data classification and grading is to protect and manage data at different levels according to its importance and sensitivity.

[0003] However, existing technical methods usually do not distinguish between data types when classifying and grading data, and use a unified model or algorithm to process all data. Different types of data have their own unique characteristics and structures. For example, text data contains grammatical and semantic information, which is suitable for analysis based on natural language processing models; while image data contains rich visual features and is more suitable for processing by computer vision models. Using a unified model to process all types of data may not fully utilize the characteristics of each data, resulting in reduced grading accuracy.

[0004] Therefore, how to improve the accuracy of data classification and grading has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a data classification and grading method, system, electronic device and storage medium to solve the defects in the prior art and improve the accuracy of data classification and grading.

[0006] The present invention provides a data classification and grading method, comprising the following steps: Acquiring category information of the data to be processed, so as to determine classification rule information according to the category information; Inputting the data to be processed and the classification rule information into a target prompt word template to generate a target prompt word; the target prompt word template is determined according to the format of the data to be processed; Inputting the target prompt word into the target classification model to obtain classification information output by the target classification model; The target classification model is matched from a pre-built model library based on the format of the processed data.

[0007] According to a data classification and grading method provided by the present invention, the step of obtaining category information of the data to be processed specifically includes: When it is determined that the format of the data to be processed is text data, inputting the data to be processed into a first text classification model to obtain category information output by the first text classification model; The first text classification model is trained based on text data samples from different data sources and the category label corresponding to each text data sample.

[0008] According to a data classification and grading method provided by the present invention, the step of obtaining category information of the data to be processed specifically further includes: When it is determined that the format of the data to be processed is image data, the data to be processed is input into a first cross-modal model to obtain an output result output by the first cross-modal model; the first cross-modal model is a zero-sample learning model; The category information of the data to be processed is determined according to the output result.

[0009] According to a data classification and grading method provided by the present invention, the step of obtaining category information of the data to be processed specifically further includes: When the category information of the data to be processed cannot be determined according to the output result, the data to be processed is input into a second cross-modal model to obtain a text description associated with the data to be processed output by the second cross-modal model; the second cross-modal model is a zero-sample learning model; Inputting the text description into a second text classification model to obtain category information output by the second text classification model; The second text classification model is trained based on text data samples from different data sources and the category label corresponding to each text data sample.

[0010] A data classification and grading method provided by the present invention further includes a training method for the first text classification model: Collecting a preset number of initial text data samples from different data sources, and labeling each of the initial text data samples with a category label according to a preset data classification rule; Adjusting the formats of all the initial text data samples to generate a plurality of actual text data samples; All of the actual text data samples and all of the category labels are input into the initial UTC model for training. During the training process, the initial UTC model is used to predict the category of each of the actual text data samples to obtain the predicted category of each of the actual text data samples, and based on the gap between all of the predicted categories and all of the category labels, the initial UTC model is updated to obtain the first text classification model.

[0011] The present invention also provides a data classification and grading system, comprising the following modules: A first processing module, used for obtaining category information of the data to be processed, so as to determine classification rule information according to the category information; A prompt word generation module, used for inputting the data to be processed and the classification rule information into a target prompt word template to generate a target prompt word; the target prompt word template is determined according to the format of the data to be processed; The second processing module is used to input the target prompt word into the target classification model to obtain the classification information output by the target classification model; wherein the target classification model is matched from a pre-constructed model library based on the format of the processed data.

[0012] A data classification and grading system provided according to the present invention also includes: A model training module is used to collect a preset number of initial text data samples from different data sources, and to label each of the initial text data samples with a category label according to a preset data classification rule; The model training module is further used to adjust the format of all the initial text data samples to generate multiple actual text data samples; The model training module is also used to input all the actual text data samples and all the category labels into the initial UTC model for training. During the training process, the initial UTC model is used to predict the category of each of the actual text data samples to obtain the predicted category of each of the actual text data samples, and based on the gap between all the predicted categories and all the category labels, the initial UTC model is updated to obtain the first text classification model.

[0013] A processing module, used to determine a data classification model and a data classification and grading model corresponding to the data to be processed, wherein the data classification model and the data classification and grading model are both determined based on the type of the data to be processed; A classification module, used for inputting the data to be processed into a corresponding data classification model to obtain a classification result; The processing module is further used to match the classification prompt words associated with the classification result from a preset classification prompt word database based on preset classification rules; The classification module is used to input the data to be processed and the classification prompt words into the corresponding data classification and classification model to obtain classification information.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the data classification and grading methods described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the data classification and grading methods described above.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the data classification and grading methods described above.

[0017] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: By obtaining the category information of the data to be processed, the classification rule information is determined according to the category information, thereby ensuring a high match between the classification rules and the data semantics, enabling the system to dynamically adapt the classification rules according to different categories of data, and improving the accuracy and flexibility of the classification task. This solution solves the problem that traditional static rules are difficult to cover diverse data by accurately identifying the categories of the data to be processed, and enhances the system's adaptability to complex task scenarios. By inputting the data to be processed and the classification rule information into the target prompt word template and generating the target prompt word, the structured expression of the classification task is realized, so that the classification model can clearly understand the characteristics and classification requirements of the data to be processed. By determining the target prompt word template based on the format of the data to be processed, the adapted prompt word is generated for different data formats (such as text or image), ensuring that the prompt word can fully express the data characteristics and classification rule requirements. By inputting the target prompt word into the target classification model and dynamically matching the adapted target classification model using the pre-built model library, the model's understanding of the characteristics of the specific data format is fully utilized to accurately output the classification information, thereby effectively improving the accuracy of data classification and grading. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the data classification and grading method provided by the present invention.

[0020] Figure 2 This is the second flow chart of the data classification and grading method provided by the present invention.

[0021] Figure 3 This is the third flow chart of the data classification and grading method provided by the present invention.

[0022] Figure 4 This is the fourth flow chart of the data classification and grading method provided by the present invention.

[0023] Figure 5This is the fifth flow chart of the data classification and grading method provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the data classification and grading system provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that, in the description of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0028] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" means at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0029] Combine the following Figure 1-Figure 7 The data classification and grading method, system, electronic device and storage medium provided by the present invention are described.

[0030] Figure 1 This is one of the flow charts of the data classification and grading method provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps: Step 101: Obtain category information of the data to be processed to determine classification rule information according to the category information.

[0031] In the process of data classification and grading, accurately obtaining the category information of the data to be processed is the basis for subsequent grading tasks. For data in different formats (such as text data and image data), an adaptive classification model is needed to ensure the accuracy and generalization ability of the category information. Therefore, the implementation of step 101 starts from the format characteristics of text data and image data, combines pre-training models and few-sample learning technology, and realizes efficient acquisition of category information.

[0032] The following embodiments will describe in detail the specific implementation methods for obtaining the category information of the data to be processed when the data to be processed is image data, and obtaining the category information of the data to be processed when the data to be processed includes text data.

[0033] In one possible implementation, refer to Figure 2 , Figure 2 This is the second flow chart of the data classification and grading method provided by the present invention. Figure 2 As shown, step 101 specifically includes step 201: Step 201: When it is determined that the format of the data to be processed is text data, the data to be processed is input into a first text classification model to obtain category information output by the first text classification model; the first text classification model is trained based on text data samples from different data sources and category labels corresponding to each text data sample.

[0034] In the process of data classification and grading, when the data to be processed is text data, accurately obtaining its category information is a prerequisite for subsequent grading tasks. In order to cope with the problems of diversity of text data categories and insufficient samples, a text classification technology based on a pre-trained model is adopted, which can achieve efficient classification under the conditions of few samples or even zero samples. Therefore, the implementation of step 201 aims to achieve high-precision prediction of text data categories through the application of the first text classification model.

[0035] First, the text data to be processed is input into the first text classification model. The model is trained by text data samples collected from different data sources and the category labels corresponding to each text data sample. In this solution, the training process of the first text classification model is preferably based on the framework of the initial UTC (Universal Text Classification) model. Since the initial UTC model has been fully pre-trained on large-scale universal text data and has learned the rich semantic features and contextual information of the text, it has good adaptability to unseen tasks. Therefore, for specific task scenarios, the model is fine-tuned through a small amount of annotated data and can be quickly adjusted to the target classification task.

[0036] After inputting text data, the first text classification model extracts the semantic features and matches them with its internal classification rules, and outputs classification results according to predefined categories. For example, the input text may be described as "this quarter's financial report", and the first text classification model can accurately classify it as a "financial document", while the text described as "technical equipment procurement contract" may be classified as a "contract document".

[0037] The classification ability of the first text classification model comes from its efficient semantic understanding and adaptation capabilities. In practical applications, text data may involve a variety of complex business scenarios or fields, and the first text classification model solves the problem that traditional deep learning models are highly dependent on large-scale labeled data through few-sample learning. At the same time, its accuracy and generalization ability ensure its applicability in diverse text classification scenarios and can maintain a high classification accuracy.

[0038] The following embodiments will explain in detail the specific training method of the first text classification model.

[0039] In one possible implementation, refer to Figure 3 , Figure 3 This is the third flow chart of the data classification and grading method provided by the present invention, such as Figure 3 As shown, step 101 specifically also includes steps 301-302: Step 301: when it is determined that the format of the data to be processed is image data, the data to be processed is input into a first cross-modal model to obtain an output result output by the first cross-modal model; the first cross-modal model is a zero-sample learning model.

[0040] Step 302: Determine the category information of the data to be processed according to the output result.

[0041] In data classification and grading tasks, when the data to be processed is image data, traditional image classification methods usually rely on large-scale annotated data for training, which is not only costly, but also difficult to adapt to the needs of category diversity and dynamic changes in actual scenarios. In order to address this problem, step 301 uses the first cross-modal model. The first cross-modal model of this solution is preferably the CLIP (Contrastive Language-Image Pretraining) model. The CLIP model is a powerful zero-sample classification model that can directly judge the category of image data based on the multimodal characteristics of image and text alignment without special training. The core purpose of this design is to efficiently filter predefined categories in image data, thereby reducing the overall complexity and computational burden of the classification system.

[0042] Specifically, the CLIP model has the ability to understand a variety of common image categories through its pre-training on large-scale image-text pair data. When the image data is input into the model, the CLIP model first matches the image features with its internal predefined categories, and determines whether the image data conforms to these categories by calculating and sorting the matching scores. Since the CLIP model can be used directly under zero-sample conditions and has fast processing capabilities, this process can complete the recognition of predefined categories in a short time, thereby avoiding unnecessary delays in complex image classification tasks.

[0043] The CLIP model will first compare the image data with the predefined categories to determine whether the image data conforms to the predefined categories. When the CLIP model recognizes that the image data conforms to its internal predefined categories, its matching result has a high degree of confidence. At this time, there is no need to further process the image data, and its output result can be directly used as the category information of the data to be processed. This not only reduces the computational burden of subsequent classification steps, but also shortens the processing time of common category image data.

[0044] By directly outputting the category information of the data to be processed, efficient processing of image data is achieved. This design fully considers the large number of repetitive or standardized categories (such as "landscape pictures" and "commodity pictures") that may exist in image data in practical applications. For these common categories, the CLIP model's rapid judgment ability can ensure the efficiency and accuracy of the classification results, avoiding the complex processing of all image data in the same way, thereby optimizing resource allocation.

[0045] This processing path not only improves classification efficiency, but also leaves more resources for subsequent processing of complex image data. The direct output of the category information of the data to be processed reflects the flexibility of the classification process and the rationality of the hierarchical design, enabling the system to balance processing speed and classification accuracy when facing large-scale image data, meeting the needs of diverse data classification scenarios.

[0046] In one possible implementation, refer to Figure 4 , Figure 4 This is the fourth flow chart of the data classification and grading method provided by the present invention, such as Figure 4 As shown, step 101 specifically also includes steps 401-402: Step 401: When the category information of the data to be processed cannot be determined according to the output result, the data to be processed is input into a second cross-modal model to obtain a text description associated with the data to be processed output by the second cross-modal model; the second cross-modal model is a zero-sample learning model.

[0047] Step 402: input the text description into a second text classification model to obtain category information output by the second text classification model; the second text classification model is trained based on text data samples from different data sources and category labels corresponding to each text data sample.

[0048] When the CLIP model cannot determine whether the image data belongs to a predefined category, the image data is input into the multimodal large model to generate a text description associated with the image data. This step is designed to deal with complex or non-standardized image data, which is beyond the predefined category range of the CLIP model, and further classification may lack effective information support. By utilizing the second cross-modal model, in this solution, the second cross-modal model is preferably a multimodal large model, which can convert the visual features of the image into a semantic text description, which can provide stronger adaptability and accuracy for subsequent classification tasks.

[0049] Specifically, the multimodal large model has the ability to convert visual information in image data into textual semantic information. In this step, the image data that has not been classified by the CLIP model is input into the multimodal large model. The model extracts the key features of the image and generates a text description closely related to the image content through its built-in generation capabilities. For example, a picture containing a natural scene may be described as "a landscape picture containing mountains and rivers", while a picture containing a product may be described as "a beverage bottle in red packaging". These text descriptions retain the core information of the image and convert it into a semantic format suitable for use by the second text classification model. Similarly, the training process of the second text classification model is preferably based on the framework of the initial UTC (Universal Text Classification) model. It should be noted that the first text classification model and the second text classification model can be the same text classification model or different text classification models.

[0050] Through this processing method, the multimodal large model makes up for the limitations of the CLIP model in processing non-predefined category images and provides new possibilities for subsequent classification steps. This cross-modal conversion method significantly enhances the flexibility and adaptability of the system, especially when dealing with complex and diverse image data, it can efficiently extract and express the key features of the image.

[0051] Furthermore, in step 402, the text description is input into the second text classification model, and the specific implementation method of obtaining the category information output by the second text classification model is consistent with the specific implementation method of step 201, so it will not be elaborated here.

[0052] In one possible implementation, refer to Figure 5 , Figure 5 This is the fifth flow chart of the data classification and grading method provided by the present invention, such as Figure 5 As shown, the training method of the first text classification model includes steps 501-503: Step 501: Collect a preset number of initial text data samples from different data sources, and annotate a category label for each initial text data sample according to a preset data classification rule.

[0053] In the text data classification task, building a high-performance classification model requires abundant training data, and the quality and diversity of the training data directly affect the generalization ability and classification effect of the model. The implementation of step 501 collects initial text data samples from different data sources and labels each sample with a category label according to the preset data classification rules, aiming to provide a high-quality training data set for subsequent model training, thereby meeting the model's requirements for diversity and accuracy.

[0054] First, according to the preset data classification rules, the classification categories that need to be supported and the feature definitions of each category are clarified. Data classification rules are usually set according to actual business needs, covering the semantic features and structural characteristics of text data in specific scenarios to ensure that the labeled sample data can fully reflect the actual needs of the target task. After the rules are clarified, a preset number of initial text data samples are collected from different sources (such as internal corporate databases, public text datasets, industry documents, etc.). The selection of these data sources is intended to cover a wide range of text types and semantic features, ensure the diversity of training data, and thus improve the generalization ability of the model.

[0055] Next, each initial text data sample is labeled according to the classification rules. The labeling process usually uses professional labeling tools or platforms, combined with manual review to ensure the accuracy and consistency of data labeling. For complex scenes or samples with vague category definitions, expert opinions can be introduced to assist in labeling to further improve the quality of labeling. In this way, the initial text data samples are given clear category labels to form a structured training data set.

[0056] This method of collecting and labeling initial text data samples from different sources solves the model performance bottleneck problem caused by single training data and insufficient samples in traditional methods. Guided by preset classification rules, the data labeling process can cover a wide range of category features while ensuring high quality and consistency of data. Ultimately, this process provides a solid data foundation for subsequent model training, significantly improving the adaptability and accuracy of the model in complex text classification tasks.

[0057] Step 502: Adjust the formats of all initial text data samples to generate multiple actual text data samples.

[0058] In the training of text classification models, ensuring that the input data format is unified and adapted to the model requirements is a key step to improve model training efficiency and classification accuracy. Step 502 generates multiple actual text data samples by formatting the annotated initial text data samples, aiming to optimize the quality and consistency of the data and provide adapted input data for model training.

[0059] First, after completing the collection and annotation of the initial text data samples, the data needs to be formatted to meet the input requirements of the text classification model, especially the UTC (Universal Text Classification) model. The initial text data samples come from various sources and may contain structured data (such as tabular text), unstructured data (such as natural language paragraphs), and text in different encodings or language formats. In order to ensure that these data can be efficiently processed by the model, they need to be uniformly converted into a standardized text format, such as removing redundant symbols, standardizing line breaks, and adjusting the encoding format. This process can effectively clean up noisy data and improve the overall quality of training data.

[0060] Then, based on the pre-training characteristics of the UTC model, the text data is further segmented. For longer texts, they are divided into multiple paragraphs based on semantic completeness; for short texts, their contextual information is supplemented by merging or expanding to make their length fit the input requirements of the model. Through such adjustments, the data can better match the semantic parsing capabilities of the UTC model, while avoiding model performance fluctuations caused by inconsistent input data length.

[0061] In addition, to enhance the generalization and anti-interference ability of the model, data augmentation operations can be introduced in the format adjustment. For example, through synonym replacement, random masking or translation back translation, training samples with different expressions can be generated. These augmented data retain the original semantics while introducing diversified expressions, so that the model can adapt to more complex language scenarios.

[0062] Step 503: All actual text data samples and all category labels are input into the initial UTC model for training. During the training process, the initial UTC model is used to predict the category of each actual text data sample to obtain the predicted category of each actual text data sample, and based on the gap between all predicted categories and all category labels, the initial UTC model is updated to obtain the first text classification model.

[0063] In the text classification task, the key to model training is to enable the model to accurately identify the category labels of text samples through iterative learning of input data. Step 503 inputs the generated actual text data samples and corresponding category labels into the initial UTC (Universal Text Classification) model for training, and continuously optimizes the model parameters, and finally obtains the first text classification model suitable for the target classification task. This step is designed to utilize the powerful pre-training ability and few-sample learning advantages of the UTC model to achieve efficient model training.

[0064] First, the actual text data samples generated in step 502 and their corresponding category labels are imported into the initial UTC model. The UTC model has been pre-trained with common semantic features based on large-scale text data and has good semantic understanding and classification capabilities. At the beginning of training, the initial parameters of the model are derived from the knowledge in its pre-training stage, which provides a basis for rapid adaptation of the target task. By inputting the actual text data samples, the model can fine-tune its classification logic based on the existing semantic features in combination with the category labels of the target task.

[0065] During the training process, the UTC model predicts the category of each actual text data sample one by one and generates the predicted category of the sample. In order to optimize the classification ability of the model, the gap between the predicted category and the actual category label is used to calculate the loss function, and the model parameters are updated through the back propagation algorithm. This process continuously narrows the deviation between the predicted category and the actual category through iterative training, and gradually enhances the model's classification ability for the target task. The hyperparameters of model training (such as learning rate, training batch, weight initialization, etc.) are set according to task requirements to ensure that the model can achieve a balance between convergence speed and classification accuracy.

[0066] In addition, in order to improve the generalization ability of the model, a verification mechanism is added to the training process. After each round of training, some actual text data samples that did not participate in the training are input into the model, and the model performance is evaluated through the prediction accuracy of the verification set, and the training parameters are adjusted. This method can effectively prevent overfitting and ensure that the model not only performs well on the training data, but also can adapt to unseen data in actual application scenarios.

[0067] Through this training method, the first text classification model finally obtained can not only make full use of the semantic features of the pre-training stage, but also quickly adapt to the target classification task under the condition of a small amount of annotated data. The accuracy and generalization ability of this model in text classification tasks are significantly improved, which solves the problem of traditional classification methods' dependence on large-scale annotated data, and at the same time has efficient training performance and flexible applicability. The construction of the first text classification model provides reliable support for subsequent data classification tasks and lays a technical foundation for complex text data processing needs.

[0068] Step 102: input the data to be processed and the classification rule information into the target prompt word template to generate the target prompt word; the target prompt word template is determined according to the format of the data to be processed.

[0069] In the data classification and grading task, in order to ensure that the classification rules and the semantics of the data to be processed can be accurately understood and executed by the classification model, step 102 combines the data to be processed with the classification rule information by generating target prompt words, and inputs them into the classification model in an adaptive format. The design and generation of target prompt words is not only a key link for the classification model to understand the semantics of the task, but also plays a decisive role in the accuracy and flexibility of the classification process.

[0070] First, select an appropriate target prompt word template according to the format of the data to be processed (such as text or image). The target prompt word template is pre-designed and has a common grading task description structure built in for different data formats. For example, for text data, the target prompt word template may contain a task description (such as "Grade the text according to the following rules"), a placeholder for the content of the data to be processed (such as "Text content: [data to be processed]"), and a placeholder for the grading rule description (such as "Grading rule: [rule description]"). For image data, the template will adapt to the image processing requirements, including describing the semantic features of the image and the grading requirements.

[0071] Next, the data to be processed and the classification rule information are embedded into the target prompt word template to generate the final target prompt word. Specifically, when the data to be processed is text, its content is directly filled into the "text content" placeholder in the template; when the data to be processed is an image, the semantic description is embedded into the "image description" placeholder in the template in combination with the image semantic description generated in the early stage. At the same time, the user-defined classification rules are embedded in the "classification rule" placeholder. Through this structured template generation process, the target prompt word can accurately express the semantic characteristics of the data to be processed and its classification requirements.

[0072] After the target prompt is generated, it contains all the key information required for the task and clearly describes the task requirements through structured language. Its design allows the target prompt to not only serve as a direct input to the grading model, but also adapt to the needs of different grading task scenarios, ensuring the dynamic adaptation of grading rules and data content.

[0073] By generating target cue words, the grading model can accurately understand the task requirements and avoid grading information deviations caused by unclear rule descriptions or ambiguous data content expressions. This method effectively improves the flexibility and interpretability of grading tasks, ensuring that the system can demonstrate high accuracy and stability when processing different data formats and diverse grading rules. Ultimately, the generation of target cue words provides strong semantic support for data classification and grading, making the entire grading process more efficient and accurate.

[0074] Step 103: Input the target prompt word into the target classification model to obtain the classification information output by the target classification model; wherein the target classification model is matched from a pre-built model library based on the format of the processed data.

[0075] In step 103, the data to be processed and the classification prompt words are input into the corresponding data classification and grading model to obtain the classification information. The purpose of this design is to guide the classification model to perform accurate and flexible classification operations on the data by combining the actual content of the data to be processed and the dynamically generated classification prompt words. Due to the diverse types of data to be processed and the complex classification requirements, it is difficult to cover the diverse requirements in actual scenarios by relying solely on static rules. By embedding dynamic prompt words, the classification model can adapt to different data and rules more efficiently, thereby ensuring the accuracy of the classification information.

[0076] In the classification task, we further select the appropriate classification model based on the different characteristics of image data and text data. For image data, we use the multimodal large model as the classification model, which uses its comprehensive understanding of the image content and the prompt word template to complete the image classification task. For text data, we choose the language large model as the classification model, which uses its deep understanding of natural language and powerful reasoning ability to accurately classify text data.

[0077] For image data, the multimodal large model combines the visual features of the image content with the rules in the prompt words, and determines the classification information of the image through feature extraction and semantic matching. For example, if a company logo is detected in a product image, the model will classify it as a first-level classification based on the rules in the prompt words. For text data, the language large model uses deep language understanding to semantically align the text content with the classification prompt words, and outputs the classification information according to the classification rules. For example, if a text marked as "contract document" contains sensitive terms, the model can classify it as a first-level classification based on the prompt words.

[0078] The effect of this design is that, through the guidance of the classification prompt words, the classification model can fully understand the relationship between the data content and the classification rules, avoiding the single and static problems in the traditional classification method. The dynamically input prompt words provide the classification model with flexible rule adaptation capabilities, while the characteristics of the data to be processed itself provide the core basis for the classification judgment of the model. Through the implementation of step 103, the classification model can achieve efficient and accurate classification operations based on the combination of the data to be processed and the classification prompt words.

[0079] Reference Figure 6 , Figure 6 : is a schematic diagram of the structure of the data classification and grading system provided by the present invention, the system comprises: A first processing module is used to obtain category information of the data to be processed, so as to determine classification rule information according to the category information; A prompt word generation module is used to input the data to be processed and the classification rule information into the target prompt word template to generate the target prompt word; the target prompt word template is determined according to the format of the data to be processed; The second processing module is used to input the target prompt word into the target classification model to obtain the classification information output by the target classification model; wherein the target classification model is matched from a pre-built model library based on the format of the processed data.

[0080] In a possible implementation, the first processing module is also used to input the data to be processed into a first text classification model when it is determined that the format of the data to be processed is text data, so as to obtain category information output by the first text classification model; the first text classification model is trained based on text data samples from different data sources and category labels corresponding to each text data sample.

[0081] In a possible implementation manner, the first processing module is further configured to: When it is determined that the format of the data to be processed is image data, the data to be processed is input into a first cross-modal model to obtain an output result output by the first cross-modal model; the first cross-modal model is a zero-sample learning model; Determine the category information of the data to be processed based on the output results.

[0082] In a possible implementation manner, the first processing module is further configured to: When the category information of the data to be processed cannot be determined according to the output result, the data to be processed is input into the second cross-modal model to obtain a text description associated with the data to be processed output by the second cross-modal model; the second cross-modal model is a zero-shot learning model; Inputting the text description into a second text classification model to obtain category information output by the second text classification model; The second text classification model is trained based on text data samples from different data sources and the category label corresponding to each text data sample.

[0083] In a possible implementation, the system further includes: A model training module is used to collect a preset number of initial text data samples from different data sources and annotate a category label for each initial text data sample according to a preset data classification rule; The model training module is also used to adjust the format of all initial text data samples to generate multiple actual text data samples; The model training module is also used to input all actual text data samples and all category labels into the initial UTC model for training. During the training process, the initial UTC model is used to predict the category of each actual text data sample to obtain the predicted category of each actual text data sample, and based on the gap between all predicted categories and all category labels, the initial UTC model is updated to obtain the first text classification model.

[0084] It should be noted that the data classification and grading system provided by the present invention can execute the data classification and grading method of any of the above-mentioned embodiments during specific operation, which will not be elaborated in this embodiment.

[0085] Figure 7 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 7 As shown, the electronic device may include: a processor 710 (processor), a communication interface 720 (CommunicationsInterface), a memory 730 (memory) and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the data classification and grading method, which includes: obtaining the category information of the data to be processed to determine the grading rule information according to the category information; inputting the data to be processed and the grading rule information into the target prompt word template to generate the target prompt word; the target prompt word template is determined according to the format of the data to be processed; inputting the target prompt word into the target grading model to obtain the grading information output by the target grading model; wherein the target grading model is matched from a pre-built model library based on the format of the processed data.

[0086] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0087] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the data classification and grading methods provided in the above-mentioned embodiments, and the method includes: obtaining category information of the data to be processed to determine the grading rule information according to the category information; inputting the data to be processed and the grading rule information into a target prompt word template to generate a target prompt word; the target prompt word template is determined according to the format of the data to be processed; inputting the target prompt word into a target grading model to obtain the grading information output by the target grading model; wherein the target grading model is matched from a pre-built model library based on the format of the processed data.

[0088] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by the processor 710, is implemented to execute the data classification and grading methods provided in the above-mentioned embodiments, the method comprising: obtaining category information of the data to be processed to determine the grading rule information according to the category information; inputting the data to be processed and the grading rule information into a target prompt word template to generate a target prompt word; the target prompt word template is determined according to the format of the data to be processed; inputting the target prompt word into a target grading model to obtain the grading information output by the target grading model; wherein the target grading model is matched from a pre-built model library based on the format of the processed data.

[0089] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0090] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data classification and grading method, characterized in that: include: Acquiring category information of the data to be processed, so as to determine classification rule information according to the category information; Inputting the data to be processed and the classification rule information into a target prompt word template to generate a target prompt word; The target prompt word template is determined according to the format of the data to be processed; Inputting the target prompt word into the target classification model to obtain classification information output by the target classification model; The target classification model is matched from a pre-built model library based on the format of the processed data.

2. The data classification and grading method according to claim 1, characterized in that: The obtaining of category information of the data to be processed specifically includes: When it is determined that the format of the data to be processed is text data, inputting the data to be processed into a first text classification model to obtain category information output by the first text classification model; The first text classification model is trained based on text data samples from different data sources and the category label corresponding to each text data sample.

3. The data classification and grading method according to claim 1, characterized in that: The obtaining of category information of the data to be processed specifically includes: When it is determined that the format of the data to be processed is image data, the data to be processed is input into a first cross-modal model to obtain an output result output by the first cross-modal model; the first cross-modal model is a zero-sample learning model; The category information of the data to be processed is determined according to the output result.

4. The data classification and grading method according to claim 3, characterized in that: The obtaining of category information of the data to be processed specifically includes: When the category information of the data to be processed cannot be determined according to the output result, the data to be processed is input into a second cross-modal model to obtain a text description associated with the data to be processed output by the second cross-modal model; the second cross-modal model is a zero-sample learning model; Inputting the text description into a second text classification model to obtain category information output by the second text classification model; The second text classification model is trained based on text data samples from different data sources and the category label corresponding to each text data sample.

5. The data classification and grading method according to claim 2, characterized in that: Also included is a training method for the first text classification model: Collecting a preset number of initial text data samples from different data sources, and labeling each of the initial text data samples with a category label according to a preset data classification rule; Adjusting the formats of all the initial text data samples to generate a plurality of actual text data samples; All of the actual text data samples and all of the category labels are input into the initial UTC model for training. During the training process, the initial UTC model is used to predict the category of each of the actual text data samples to obtain the predicted category of each of the actual text data samples, and based on the gap between all of the predicted categories and all of the category labels, the initial UTC model is updated to obtain the first text classification model.

6. A data classification and grading system, characterized in that: include: A first processing module, used for obtaining category information of the data to be processed, so as to determine classification rule information according to the category information; A prompt word generation module, used for inputting the data to be processed and the classification rule information into a target prompt word template to generate a target prompt word; The target prompt word template is determined according to the format of the data to be processed; The second processing module is used to input the target prompt word into the target classification model to obtain the classification information output by the target classification model; wherein the target classification model is matched from a pre-constructed model library based on the format of the processed data.

7. The data classification and grading system according to claim 6, characterized in that: The system further comprises: A model training module is used to collect a preset number of initial text data samples from different data sources, and to label each of the initial text data samples with a category label according to a preset data classification rule; The model training module is further used to adjust the format of all the initial text data samples to generate multiple actual text data samples; The model training module is also used to input all the actual text data samples and all the category labels into the initial UTC model for training. During the training process, the initial UTC model is used to predict the category of each of the actual text data samples to obtain the predicted category of each of the actual text data samples, and based on the gap between all the predicted categories and all the category labels, the initial UTC model is updated to obtain the first text classification model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data classification and grading method as described in any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data classification and grading method as described in any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data classification and grading method as described in any one of claims 1 to 5 is implemented.

Images