Automatic classification and grading method for unstructured data based on large model

Abstract

The invention relates to an unstructured data automatic classification and grading method based on a large model, and belongs to the field of data automatic classification and grading. Comprising the steps of performing fine adjustment on a large model based on a classification and classification vector knowledge base, and generating a lightweight field adaptive classification and classification large model through knowledge distillation; when an unstructured image to be classified and graded is processed, text image blocks are extracted through text detection and image segmentation, then a text is generated by adopting a Transform OCR technology and vectorized, and matching information is matched and acquired from a knowledge base by utilizing a retrieval enhancement generation mechanism; and inputting the data integrated with the matching information into an adaptive classification and grading large model in the lightweight field for automatic classification and grading, and continuously updating the knowledge base and iteratively optimizing the model through expert auditing feedback. According to the method, the problems of inaccurate complex image text extraction and limited general large model application are solved, and the accuracy, economical efficiency and adaptability of classification and grading are improved.

Description

Technical Field

[0001] This invention relates to an automated classification and grading method for unstructured data based on a large model, belonging to the field of automated data classification and grading technology. Background Technology

[0002] In today's information society, the automated classification and grading of massive amounts of unstructured data, especially images containing key textual information (such as invoices, contracts, and engineering drawings), is crucial for improving operational efficiency and compliance across industries. However, existing technologies generally face insurmountable bottlenecks when processing such unstructured image data efficiently and accurately.

[0003] First, existing technologies face significant challenges in accurately extracting text from complex images. Traditional optical character recognition (OCR) technology performs reasonably well when processing clearly scanned, well-laid-out documents, but its accuracy drops drastically when faced with unstructured images commonly found in the real world, such as low-resolution images, images with uneven lighting, complex backgrounds, or varied fonts. Although deep learning-based text detection methods have improved robustness to some extent, their multi-stage processing is prone to error accumulation and still struggles to perfectly handle all complex scenarios.

[0004] Secondly, even if text extraction is successful, enabling machines to deeply understand the text content for professional classification and grading presents a greater challenge for current technologies. One direct approach is to input the extracted text into a general, pre-trained large-scale language model (LLM) for analysis. However, this method has fundamental flaws: First, general-scale models lack in-depth understanding and expertise in specific business domains (such as financial regulations and public safety standards), resulting in insufficient accuracy and reliability when handling highly specialized tasks; second, general-scale models are bulky and computationally expensive, making them unsuitable for large-scale, cost-effective deployment; third, general-scale models are static knowledge entities, lacking an effective mechanism to dynamically absorb new domain rules or expert feedback, thus failing to adapt to continuously changing business needs.

[0005] In summary, existing technological approaches suffer from significant gaps: front-end text extraction techniques struggle to efficiently and accurately process complex unstructured images, while back-end analysis models, lacking domain expertise, are costly and unable to be continuously optimized, failing to meet the high standards required for practical applications. Therefore, there is an urgent need in this field for a new technological solution that can efficiently address and process these image processing challenges. This solution utilizes a dedicated model deeply adapted to domain knowledge for intelligent analysis and establishes a continuously optimizing closed-loop system, thereby achieving truly efficient, accurate, and economical automated classification and grading of unstructured data. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides an automated classification and grading method for unstructured data. This invention can achieve truly efficient, accurate, and economical automated classification and grading of unstructured data.

[0007] This invention first constructs a quantitative classification and grading knowledge base by collecting domain-specific standards and specifications. Then, it fine-tunes a basic large-scale model using this knowledge base and generates a cost-effective, lightweight, domain-adaptive model specifically for classification and grading tasks through knowledge distillation. When processing unstructured image data to be classified and graded, the system first performs text detection and image segmentation to extract clean text image blocks, and then uses Transformer OCR technology to generate text. Subsequently, the generated text is vectorized, and relevant contextual information is matched and obtained from the knowledge base based on a retrieval-enhanced generation mechanism. Finally, the data integrating the matching information is input into the lightweight model for automated classification and grading. The knowledge base is continuously updated and the model iteratively optimized through expert review and feedback, thus forming a complete, accurate, and self-evolving closed-loop processing system.

[0008] The technical solution of this invention is: an automated classification and grading method for unstructured data based on a large model, the method comprising:

[0009] S1. Collect data classification standards, specifications, and rules to form a corpus. Based on the embedding model, vectorize the classification rules and hierarchical rules in the corpus and construct a classification vector knowledge base and a hierarchical vector knowledge base respectively.

[0010] S2. Based on the two constructed knowledge bases, fine-tune the large model to obtain a domain-adaptive classification large model and a hierarchical large model. Based on knowledge distillation, obtain a lightweight domain-adaptive classification large model and a hierarchical large model. Dynamically update all the large models obtained in this step according to the two constructed knowledge bases.

[0011] S3. Perform text detection and image segmentation on unstructured data to be classified and graded, and separate text image blocks containing text from complex backgrounds and interference elements;

[0012] S4. Use Transformer OCR technology for image understanding and character-level text generation, vectorize the data based on the embedding model to obtain data feature vectors; then perform similarity matching with the classification vector knowledge base and the hierarchical vector knowledge base based on the retrieval enhancement generation mechanism to obtain matching information.

[0013] S5. After integrating the unstructured data to be classified and graded with the matching information, input the data into the lightweight domain adaptive classification model and the grading model for automated grading and classification. The two classification vector knowledge bases and grading vector knowledge bases are continuously updated and expanded through expert review and feedback.

[0014] Further, S2 includes:

[0015] S21. Domain-Adaptive Fine-Tuning: Adapting a general, pre-trained, security-domain-based large language model to classification and hierarchical domains to obtain a teacher model with specialized domain knowledge; specifically including:

[0016] For input sample x i The output of the security domain-based large language model is a probability distribution over all C categories. The result is obtained through the Softmax function: ;

[0017] in, is the output logarithm of the fundamental large language model in the security field, where C is the total number of categories; Model for "samples" The raw score for "belonging to category C" is the output without probability normalization, also known as "logits"; Model for "samples" The original score for "belonging to the j-th category" is a single element when iterating through all C categories;

[0018] The fine-tuning process optimizes the parameters θ of the underlying large language model for the security domain by minimizing a loss function, which is the cross-entropy loss L. CE :

[0019] ;

[0020] Where N is the total number of training samples, y i,c It is an indicator function, if sample x i If the true label is category c, its value is 1; otherwise, it is 0.

[0021] The gradient-based stochastic gradient descent algorithm is used to optimize the parameters θ of the basic large language model in the security domain. The parameter update rule is expressed as follows:

[0022] ;

[0023] Where, θ t Here, η is the parameter for the t-th iteration, and η is the learning rate. It is the gradient of the loss function with respect to the parameters;

[0024] Through this fine-tuning process, we obtain the domain-adaptive classification model and the domain-adaptive hierarchical model, denoted as teacher model M, respectively. Tclass Teacher Model M Tgrade Their parameters are θ Tclass θ Tgrade ;

[0025] S22. Knowledge Distillation and Model Lightweighting:

[0026] Using the domain adaptive classification model and hierarchical model obtained in S21 as teacher models, the knowledge learned by them is transferred to a student model with a small number of parameters and a simple structure, resulting in a lightweight domain adaptive classification model and hierarchical model.

[0027] S23. Incremental learning or continuous fine-tuning through dynamic updates:

[0028] Let the model parameters at time point t be θ. t When new or updated data forms dataset D in the classification vector knowledge base and the hierarchical vector knowledge base... new At that time, the system will use D new Alternatively, it can be combined with some old data to further fine-tune the existing model; the fine-tuning process is the same as that in S21, which is to minimize the loss function L. CE To update the model parameters and obtain the new parameters θ t+1 ;

[0029] This dynamic update process is first applied to the teacher model. After the teacher model is updated, the knowledge distillation process in S22 is re-executed to generate a new generation of lightweight domain adaptive classification large models and hierarchical large models that contain the latest knowledge.

[0030] Further, S3 includes:

[0031] S31. The YOLOv8 object detection model is used to automatically locate the position and extent of all text instances in the input unstructured image;

[0032] The training process of the object detection model YOLOv8 aims to minimize a composite loss function L. YOLOv8 The composite loss function includes classification loss L cls and bounding box regression loss L box :

[0033] ;

[0034] Where, λ cls and λ box These are the weight hyperparameters used to balance the various loss terms;

[0035] Classification loss L cls Used to measure the accuracy of the object detection model in distinguishing text from background; for each prediction, its classification loss L cls The calculation is as follows:

[0036] ;

[0037] Where p is the probability that the object detection model predicts the region to be text, and y is the true label, where 1 represents text and 0 represents background;

[0038] By employing the gradient descent optimization algorithm, the total loss function L is minimized. YOLOv8 The network parameters of the YOLOv8 object detection model are iteratively optimized to enable it to accurately output the bounding boxes of all text regions in unstructured images.

[0039] S32. After determining the bounding box containing the text region through text detection, the sub-image within each bounding box is segmented to separate the text characters from their local background and noise, generating a binarized text image patch.

[0040] Further, S4 includes:

[0041] S41. The text image patch generated in S3 is received as input via an image encoder to generate visual context features; specifically including:

[0042] First, scale the input text image blocks to a preset size;

[0043] Subsequently, the scaled text image patch is segmented into a series of fixed-size, non-overlapping image patches; these image patches are linearly flattened and combined with positional encoding to form a visual feature sequence, which serves as the input to the image encoder;

[0044] The image encoder processes the visual feature sequence through multiple stacked Transformer modules to extract deep visual context features;

[0045] S42. The text decoder generates characters or words in the text sequence one by one based on the visual context features output by the image encoder; at each time step t, the text decoder receives the output of the image encoder and the character sequence generated in all previous time steps as input to predict the probability distribution of the next character;

[0046] The image encoder and text decoder use a multi-head self-attention mechanism to enable the large model to capture long-distance dependencies within the image and the text sequence, ultimately generating a text string S that is highly consistent with the image content. ocr .

[0047] S43. Text Vectorization: Convert the text string S generated in S42 into vectorized text.ocr It is converted into a fixed-dimensional real-number vector that can represent its semantic information; a pre-trained embedding model using the Sentence-BERT architecture takes a text string as input and outputs a d-dimensional data feature vector. ;

[0048] S44. Employ a retrieval enhancement generation mechanism to process the original data feature vector generated in S43. As query vectors, respectively in the classification vector knowledge base K class ={k1,k2,...,k M} and hierarchical vector knowledge base K grade ={g1,g2,...,g N In the similarity search, the vectors in the classification vector knowledge base and the hierarchical vector knowledge base are collectively referred to as document vectors. , where k i and g j All are d-dimensional vectors;

[0049] Query vector Each document vector in the classification vector knowledge base and the hierarchical vector knowledge base Similarity between The formula is: ; Expanded to: ;

[0050] Among them, || || is the query vector The Euclidean norm, || || is a document vector The Euclidean norm, yes , The i-th element, Document vectors The i-th element, where d is the dimension of the vector;

[0051] calculate The similarity scores are calculated against all vectors in the classification vector knowledge base and the hierarchical vector knowledge base, and the text or vectors corresponding to the top-N rules or specifications with the highest scores are retrieved. This set of retrieved highly similar text or vectors constitutes the matching information output in this step. This matching information is then compared with the original data feature vectors. Together, they serve as inputs to the domain adaptive classification large model and hierarchical large model in S5.

[0052] Further, S5 includes: S51. Input data integration: Integrating the data feature vector generated in S4 with the matching information obtained from retrieval using vector concatenation to form the final enhanced feature vector of the input large model. ;

[0053] S52. Enhance the feature vector The data are respectively input into the lightweight domain adaptive classification model and the hierarchical model trained in S2 for hierarchical classification.

[0054] S53. Expert Review: Submit the automated grading and classification results generated in S52 to the human expert review interface; domain experts review and confirm the model's judgment results and provide correction suggestions;

[0055] S54. Closed-loop feedback and continuous optimization: By utilizing feedback information reviewed by human experts, a closed-loop optimization mechanism is constructed to continuously improve system performance; this closed-loop optimization mechanism includes two levels: knowledge base updates and large model iterations.

[0056] Further, S52 includes:

[0057] For the lightweight domain, adaptive classification large model M Sclass For input The output is a probability distribution over all C categories. The result is obtained through the softmax function: ;

[0058] in, =( , ,..., )and ;

[0059] Final automated classification results The category identified as having the highest probability: ;

[0060] Similarly, the results of automated grading Similarly, a lightweight adaptive hierarchical large model M is also used. Sgrade Calculated;

[0061] Finally, the automated classification results and automated grading results are integrated and output as the final judgment on the original unstructured data.

[0062] The present invention also provides an automated classification and grading system for unstructured data based on a large model, the system comprising: a module for executing the aforementioned automated classification and grading method for unstructured data based on a large model.

[0063] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method for automated classification and grading of unstructured data based on a large model.

[0064] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned method for automated classification and grading of unstructured data based on a large model.

[0065] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for automated classification and grading of unstructured data based on a large model.

[0066] The beneficial effects of this invention are:

[0067] This invention is based on a technical framework that deeply integrates advanced visual recognition and customized large-scale models, making automated classification and grading of unstructured data possible. This invention achieves truly efficient, accurate, and economical automated classification and grading of unstructured data. On one hand, by employing a combination of text detection, image segmentation, and Transformer OCR, it solves the problems of low text extraction accuracy and poor robustness in existing technologies when processing unstructured images with complex backgrounds and varied layouts. On the other hand, by constructing a domain knowledge base for model fine-tuning and combining knowledge distillation and retrieval enhancement generation mechanisms, it solves the problems of insufficient professional accuracy and high deployment costs when general-purpose large-scale models are applied to professional fields. Furthermore, this invention has the ability to perform closed-loop optimization through expert review and feedback, continuously updating the knowledge base and iterating the model to ensure the system can self-evolve and adapt to new rules over the long term. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0069] Figure 1 This is a flowchart illustrating an automated classification and grading method for unstructured data based on a large model, provided by the present invention.

[0070] Figure 2 This is a flowchart illustrating the architecture of an automated classification and grading method for unstructured data based on a large model, provided by this invention.

[0071] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0072] The labels in the diagram are as follows:

[0073] 310 - Processor; 320 - Communication interface; 330 - Memory; 340 - Communication bus. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0075] It should be noted that the executing entity of the embodiments of the present invention can be various types of terminals, such as computers, smartphones, tablets, wearable devices, personal digital assistants (PDAs), mobile internet devices (MIDs), and other terminals capable of text processing. The present invention does not impose any restrictions on this.

[0076] Example 1: As Figures 1-3 As shown in the figure, this embodiment of the invention provides an automated classification and grading method for unstructured data based on a large model. The method includes:

[0077] S1. Collect data classification standards, specifications, and rules to form a corpus. Based on the embedding model, vectorize the classification rules and hierarchical rules in the corpus and construct a classification vector knowledge base and a hierarchical vector knowledge base respectively.

[0078] When performing data classification and grading, the operation must adhere to a clear compliance level. The primary basis is laws, regulations, and general standards and specifications. Based on this, the classification and grading systems or standards of various vertical industries should be carefully referenced. If a specific industry lacks written standards, then the public data classification and grading guidelines or related standards issued by local governments should be used as supplementary references. Therefore, building a corpus requires a systematic collection of a series of authoritative documents, such as national standards like GB / T43697-2024 "Data Security Technology - Data Classification and Grading Rules," GB / T 35273-2020 "Information Security Technology - Personal Information Security Specification," and TC260-PG-20212A "Network Security Standard Practice Guide - Network Data Classification and Grading Guidelines," etc. Simultaneously, the scope of collection must be expanded to industry-specific regulations in key areas such as finance, transportation, market supervision, medical insurance, natural resources, and education, as well as other relevant standards for public data classification and grading.

[0079] To create a classification vector knowledge base, a series of classification criteria are processed. For example, methodological principles such as "data should first be categorized by industry sector, and then further subdivided according to business attributes" are vectorized. Specific industry classification standards, such as categorizing data into industrial, telecommunications, financial, energy, transportation, natural resources, health, education, and scientific data, are also processed. Furthermore, procedural regulations regarding "industry regulators or supervisory bodies should further refine the data classification based on their business attributes" are also incorporated into the vectorization process.

[0080] Similarly, in order to construct a hierarchical vector knowledge base, the hierarchical criteria are vectorized. For example, the core hierarchical basis, namely "based on the importance of data to economic and social development, and the level of harm that data may cause to national security, economic operation, social order, public interests, and even the rights and interests of organizations and individuals once it suffers a security incident (such as leakage, tampering, damage, or illegal acquisition, use, or sharing), the data is classified into three levels from high to low: core data, important data, and general data," is converted into vector form and stored in the knowledge base.

[0081] S2. Based on the two constructed knowledge bases, fine-tune the large model to obtain a domain-adaptive classification large model and a hierarchical large model. Based on knowledge distillation, obtain a lightweight domain-adaptive classification large model and a hierarchical large model. Dynamically update all the large models obtained in this step according to the two constructed knowledge bases.

[0082] In some feasible implementations, S2 includes:

[0083] S21. Domain Adaptive Fine-tuning: Adapting a general, pre-trained security domain-based large language model to classification and hierarchical domains to obtain a teacher model with specialized domain knowledge;

[0084] Specifically, let the fundamental large language model in the security domain be a model consisting of parameters θ. base Defined function This step is performed separately for classification tasks and hierarchical tasks; taking the classification task as an example, the "classification vector knowledge base" constructed in step S1 is used as the training dataset. , where x i The input sample is a d-dimensional feature vector of the rule text extracted from the knowledge base, y i Set its corresponding preset category label; Specifically, it includes: For input sample x i The output of the security domain-based large language model is a probability distribution over all C categories. The result is obtained through the Softmax function: ; in, is the output logarithm of the fundamental large language model in the security field, where C is the total number of categories; Model for "samples" The raw score for "belonging to category C" is the output without probability normalization, also known as "logits"; Model for "samples" The original score for "belonging to the j-th category" is a single element when iterating through all C categories; The fine-tuning process optimizes the parameters θ of the underlying large language model for the security domain by minimizing a loss function, namely the Cross-Entropy Loss. CE : ; Where N is the total number of training samples, y i,c It is an indicator function, if sample x i If the true label is category c, its value is 1; otherwise, it is 0. The gradient-based stochastic gradient descent algorithm is used to optimize the parameters θ of the basic large language model in the security domain. The parameter update rule is expressed as follows: ;

[0085] Where, θ t Here, η is the parameter for the t-th iteration, and η is the learning rate. It is the gradient of the loss function with respect to the parameters; Through this fine-tuning process, we obtain the domain-adaptive classification model and the domain-adaptive hierarchical model, denoted as teacher model M, respectively. Tclass Teacher Model M TgradeTheir parameters are θ Tclass θ Tgrade ; S22. Knowledge Distillation and Model Lightweighting: The purpose of this sub-step is to significantly reduce model size and computational requirements while maintaining model performance, thereby reducing deployment and inference costs; The domain-adaptive classification model and hierarchical model obtained in S21 are used as teacher models. The knowledge learned by these models is then transferred to student models with fewer parameters and simpler structures, resulting in lightweight domain-adaptive classification models and hierarchical models. Specifically, this includes: Assume the teacher model is M T Its parameter is θ T The student model is M. S Its parameter is θ S ,and The core of knowledge distillation lies in enabling the student model to learn not only the true labels (hard target), but also the softened probability distribution output by the teacher model (soft target).

[0086] To generate soft targets, a temperature hyperparameter T is introduced to scale the logits of both the teacher and student models, and the softened probability distribution p is calculated using the Softmax function. T : ;

[0087] Among them, z i It is the logit value of category i. This represents the probability distribution of category i. When T>1, the output probability distribution is smoother and contains richer information such as the similarity between categories.

[0088] The training process of knowledge distillation involves minimizing a combined loss function L. KD To complete: ;

[0089] Among them, L hard This is the loss of the student model on the true labels, typically the standard cross-entropy loss, calculated using the standard Softmax output of the student model (i.e., T=1); L soft It is the matching loss between the softened output of the student model and the softened output of the teacher model, usually using the Kullback-Leibler (KL) divergence D. KL To measure the difference between the two distributions, multiply by a scaling factor T.2 To ensure that the magnitude of the gradient is comparable to that of the hard target loss: ;

[0090] in, and These are the softening probability outputs of the teacher model and the student model at temperature T, respectively.

[0091] Minimize L using gradient descent KD To optimize the parameters θ of the student model S Ultimately, a lightweight domain adaptive classification large model (M) was obtained. Sclass ) and lightweight domain adaptive hierarchical large model (M Sgrade ).

[0092] S23. Incremental learning or continuous fine-tuning through dynamic updates: Let the model parameters at time point t be θ. t When new or updated data forms dataset D in the classification vector knowledge base and the hierarchical vector knowledge base... new At that time, the system will use D new Alternatively, it can be combined with some old data to further fine-tune the existing model; the fine-tuning process is the same as that in S21, which is to minimize the loss function L. CE To update the model parameters and obtain the new parameters θ t+1 ; This dynamic update process is first applied to the teacher model (domain adaptive classification large model and hierarchical large model). After the teacher model is updated, the knowledge distillation process in S22 is re-executed to generate a new generation of lightweight domain adaptive classification large model and hierarchical large model that contains the latest knowledge. This mechanism ensures that the model deployed in the production environment can continuously and dynamically reflect the latest domain knowledge. S3. Perform text detection and image segmentation on unstructured data to be classified and graded, and separate text image blocks containing text from complex backgrounds and interference elements; In some feasible implementations, S3 includes: S31. Utilize the object detection model YOLOv8 (You Only Look Once version 8) to automatically locate the position and extent of all text instances in the input unstructured image; The training process of the object detection model YOLOv8 aims to minimize a composite loss function L. YOLOv8 The composite loss function includes classification loss L cls and bounding box regression loss L box : ; Where, λ cls and λ box These are the weight hyperparameters used to balance the various loss terms; Classification loss L cls This is used to measure the accuracy of the object detection model in distinguishing between text and background; since text detection is essentially a binary classification problem (text / non-text), a binary cross-entropy (BCE) loss is employed, where for each prediction, the classification loss L... cls The calculation is as follows: ; Where p is the probability that the object detection model predicts the region to be text, and y is the true label, where 1 represents text and 0 represents background; Bounding box regression loss L box Used to measure the predicted bounding box B p With the true bounding box B gt The differences between them; this embodiment uses Complete IoU (CIoU), which, based on Intersection over Union (IoU), additionally considers the consistency of center point distance and aspect ratio; the formula for calculating IoU is: ;

[0093] CIoU loss L CIoU The definition of is: ;

[0094] in, It is the center point of the two boxes and The square of the Euclidean distance, It is the diagonal length of the smallest closure box that can contain two boxes simultaneously. It is a parameter that measures the consistency of aspect ratio. It is a balance parameter; It is the width of the actual bounding box (such as the horizontal length of the bounding box of manually annotated text, objects, etc.). This refers to the width of the bounding box predicted by the model (the horizontal length of the bounding box predicted by the model for the target region). Both are used to measure the difference in width between the predicted box and the ground truth box, helping to determine whether their aspect ratios are consistent, and helping the model predict the shape of the bounding box more accurately.

[0095] By employing the gradient descent optimization algorithm, the total loss function L is minimized. YOLOv8The network parameters of the YOLOv8 object detection model are iteratively optimized to enable it to accurately output the bounding boxes of all text regions in unstructured images. S32. After determining the bounding box containing the text region through text detection, the sub-image within each bounding box is segmented to separate the text characters from their local background and noise, generating a binarized text image patch.

[0096] Adaptive thresholding is used to segment the sub-image within each bounding box because it effectively handles the uneven illumination problem common in unstructured images. Unlike global thresholding methods that use a single threshold, adaptive thresholding calculates a local threshold T(x,y) for each pixel (x,y) in the image. This local threshold T(x,y) is calculated based on the pixel intensity within the neighborhood of pixel (x,y) (a region of size blockSize × blockSize). A Gaussian weighted sum method can be used for the specific calculation. ;

[0097] Where I(i,j) is the intensity value of the pixel in the neighborhood, w(i,j) is a Gaussian kernel weight, which makes the pixel closer to the center pixel (x,y) have a larger weight, and D is a constant subtracted from the calculated weighted sum, used to fine-tune the threshold. After obtaining the local threshold T(x,y), the sub-image is binarized to generate the output pixel Iout(x,y): ; Among them, maxVal is usually set to 255 (white).

[0098] To further refine the segmented binary image, an opening operation is employed: first, an erosion operation is performed, followed by a dilation operation. This combined operation effectively removes small background noise points and smooths object contours. Its mathematical definition is: ;

[0099] Where A is the input binary image, and B is the structuring element. ⊕ indicates corrosion, and ⊕ indicates expansion.

[0100] By performing the text detection and image segmentation steps in S3, the method of the present invention can robustly extract high-quality text image patches that are separated from the background and noise from any unstructured image, laying a solid foundation for high-precision text recognition in the subsequent S4.

[0101] S4. Use Transformer OCR technology for image understanding and character-level text generation, vectorize the data based on the embedding model to obtain data feature vectors; then perform similarity matching with the classification vector knowledge base and the hierarchical vector knowledge base based on the retrieval enhancement generation mechanism to obtain matching information. In some feasible implementations, S4 includes: S41. Transformer OCR technology adopts a standard encoder-decoder architecture, and its core component is the Transformer module; The image encoder receives text image patches generated in S3 as input to generate visual context features; specifically, this includes: First, scale the input text image blocks to a preset size (e.g., 384×384 pixels). Subsequently, the scaled text image block is divided into a series of fixed-size, non-overlapping image patches, such as 16×16 24×24 pixel image patches; these image patches are linearly flattened and combined with positional encoding to form a visual feature sequence, which serves as the input to the image encoder. The image encoder processes the visual feature sequence through multiple stacked Transformer modules to extract deep visual context features; S42. Text Decoder: An autoregressive Transformer model whose task is to generate characters or words in a text sequence one by one based on the visual context features output by the image encoder. At each time step t, the text decoder receives the output of the image encoder and the character sequence generated in all previous time steps (1 to t-1) as input to predict the probability distribution of the next character. The core of the encoder and decoder is the multi-head self-attention mechanism; this mechanism allows the model to weigh the importance of all other elements in the sequence when processing one element; its core computational unit is scaled dot-product attention, and its mathematical expression is: ;

[0102] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k The dimension of the key vector; The image encoder and text decoder use a multi-head self-attention mechanism to enable the large model to capture long-distance dependencies within the image and the text sequence, ultimately generating a text string S that is highly consistent with the image content. ocr .

[0103] S43. Text Vectorization: Convert the text string S generated in S42 into vectorized text. ocr This is converted into a fixed-dimensional real-number vector that represents its semantic information, i.e., text embedding. A pre-trained embedding model using the Sentence-BERT architecture takes a text string as input and outputs a d-dimensional data feature vector. ; ;

[0104] in, This vector S was captured ocr The deep semantic information allows texts with similar semantics to have similar vector representations in vector space.

[0105] S44. Employ the Retrieval-Augmented Generation (RAG) mechanism to process the original data feature vector generated in S43. As query vectors, respectively in the classification vector knowledge base K class ={k1,k2,...,k M} and hierarchical vector knowledge base K grade ={g1,g2,...,g N In the similarity search, the vectors in the classification vector knowledge base and the hierarchical vector knowledge base are collectively referred to as document vectors. , where k i and g j All are d-dimensional vectors;

[0106] Query vector Each document vector in the classification vector knowledge base and the hierarchical vector knowledge base Similarity between The formula is: ; Expanded to: ;

[0107] Among them, || || is the query vector The Euclidean norm, || || is a document vector The Euclidean norm, yes , The i-th element, Document vectors The i-th element, where d is the dimension of the vector;

[0108] calculate The similarity scores are calculated against all vectors in the classification vector knowledge base and the hierarchical vector knowledge base, and the text or vectors corresponding to the top-N rules or specifications with the highest scores are retrieved. This set of retrieved highly similar text or vectors constitutes the matching information output in this step. This matching information is then compared with the original data feature vectors. Together, they serve as inputs to the domain adaptive classification model and hierarchical model in S5, providing rich and directly relevant domain knowledge references for their final classification and hierarchical decisions.

[0109] S5. After integrating the unstructured data to be classified and graded with the matching information, input the data into the lightweight domain adaptive classification model and the grading model for automated grading and classification. The two classification vector knowledge bases and grading vector knowledge bases are continuously updated and expanded through expert review and feedback.

[0110] In some feasible implementations, S5 includes: S51. Input data integration: integrating the data feature vector generated in S4. The matching information obtained from the retrieval is integrated with the data using vector concatenation to form the final enhanced feature vector input to the large model. The matching information is a set of feature vectors of the Top-N most similar rules or norms retrieved from the classification and hierarchical knowledge base. ;

[0111] First, calculate the aggregate vector of the matching information, such as the mean vector:

[0112] ;in, Subsequently, the data feature vectors Aggregation vector with matching information The vectors are then concatenated to form the final feature vector for the input model. : ;in, The semicolon ";" indicates a vector concatenation operation. This enhanced feature vector simultaneously contains semantic information from the original text and contextual information from relevant domain rules.

[0113] S52. Enhance the feature vector The data are respectively input into the lightweight domain adaptive classification model and the hierarchical model trained in S2 for hierarchical classification.

[0114] In some feasible implementations, S52 includes:

[0115] For the lightweight domain, adaptive classification large model M Sclass For input The output is a probability distribution over all C categories. The result is obtained through the softmax function: ;

[0116] in, =( , ,..., )and ;

[0117] Final automated classification results The category identified as having the highest probability: ;

[0118] Similarly, the results of automated grading Similarly, a lightweight adaptive hierarchical large model M is also used. Sgrade Calculated;

[0119] Finally, the automated classification results and automated grading results are integrated and output as the final judgment on the original unstructured data.

[0120] S53. Expert Review: To ensure the accuracy and authority of the system output, the automated grading and classification results generated in S52 are submitted to the human expert review interface; domain experts review and confirm the model's judgment results and provide feedback, such as correct classification labels. and rating labels ;

[0121] S54. Closed-loop feedback and continuous optimization: By utilizing feedback information reviewed by human experts, a closed-loop optimization mechanism is constructed to continuously improve system performance; this closed-loop optimization mechanism includes two levels: knowledge base updates and large model iterations.

[0122] Continuous updates and expansion of the knowledge base, along with expert review feedback, are used to dynamically maintain and expand the vector knowledge base built in S1. When experts provide corrective opinions, it means that an incorrect judgment in a model has been discovered or a new rule scenario has been introduced;

[0123] New knowledge vectorization: This involves using the original text string S corresponding to the case. ocr (Source: S4) and the correct tags Treat it as new knowledge; use the embedding model in S4 to analyze S ocr Vectorization yields new knowledge vectors. ;

[0124] Knowledge base injection: injecting newly generated knowledge vectors Add the corresponding correct labels to the classification vector knowledge base and the hierarchical vector knowledge base;

[0125] Dynamic updates to the knowledge base will act as triggers to initiate the model's retraining or fine-tuning process, ensuring that the model learns the latest knowledge; if the original knowledge base is... The updated knowledge base is This process enables the continuous learning and expansion of the knowledge base;

[0126] Triggering conditions: This embodiment uses a triggering mechanism based on update volume; the system sets an update threshold. ;

[0127] When the number of newly added or updated entries in the knowledge base, i.e. Reaching this threshold The model update process is automatically triggered at certain times;

[0128] Model iteration: Once triggered, the system will use the updated knowledge base. Incremental fine-tuning is performed on the teacher models (domain-adaptive classification large model and hierarchical large model) obtained in S2. This process is mathematically consistent with the fine-tuning process in S21, aiming to allow the teacher models to learn new data and rules;

[0129] Lightweight Model Regeneration: After the teacher model is updated, the system will re-execute the knowledge distillation process in S22, using the updated teacher model to train a new generation of more powerful lightweight models. Subsequently, the newly generated lightweight model will be deployed to the production environment to replace the old model.

[0130] Through the complete process of automated processing, expert review, and closed-loop feedback optimization described above, the method of this invention can not only provide real-time automated classification and grading services, but also continuously improve its accuracy and adaptability through continuous learning, forming an intelligent and self-evolving system.

[0131] The present invention also provides an automated classification and grading system for unstructured data based on a large model, the system comprising: a module for executing the aforementioned automated classification and grading method for unstructured data based on a large model.

[0132] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method for automated classification and grading of unstructured data based on a large model.

[0133] This invention also provides an electronic device. Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the aforementioned automated classification and grading method for unstructured data based on a large model.

[0134] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned automated classification and grading method for unstructured data based on a large model.

[0136] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the automated classification and grading method for unstructured data based on a large model as described in the first aspect.

[0137] This invention innovatively proposes an automated classification and grading method for unstructured data that combines deep vision processing, multi-stage model optimization, and retrieval enhancement generation techniques. Based on a dual optimization framework: on the one hand, text detection, image segmentation, and Transformer OCR techniques ensure high-quality extraction of text information from unstructured images with complex backgrounds and varied layouts; on the other hand, a cascaded process of "domain fine-tuning-knowledge distillation" constructs a cost-effective and lightweight large model specifically for classification and grading tasks. This design solves the problem of inaccurate text extraction by traditional OCR and general vision models when processing low-quality, varied images; and on the other hand, knowledge distillation addresses the high computational cost and low inference efficiency associated with directly deploying a domain-fine-tuned large model, resulting in a solution that combines high accuracy and economy. Furthermore, by introducing a retrieval enhancement generation mechanism based on an external knowledge base during the inference stage, this invention injects real-time, dynamic domain knowledge into the lightweight model, effectively addressing the lack of specific domain expertise in large models and providing rich contextual references for each decision, thereby significantly improving the performance of automated classification and grading.

[0138] Table 1 compares the advantages and disadvantages of different methods. method Automation level advantage shortcoming Manually created classification and grading templates Low Highly flexible and controllable: The rules are manually defined, fully controllable, and suitable for handling small amounts of highly sensitive data. Inefficient and costly: It relies entirely on manual labor, is time-consuming and labor-intensive, involves a huge workload, and is difficult to handle massive amounts of data. Based on machine learning model middle Once the model is trained, it can automatically classify new data; it can learn complex patterns and relationships from the data that are difficult to express with manual rules. Reliance on massive amounts of high-quality labeled data: Model performance is heavily dependent on the quantity and quality of training data, and data preparation is the main bottleneck; "black box" problem, poor interpretability: The decision-making process is not transparent, and it is difficult to explain why a certain classification is made. Based on the general large model method higher It can handle multiple tasks without specific training and has excellent natural language understanding capabilities. Classification can be quickly achieved through prompting engineering or few-shot learning, facilitating prototyping. It can flexibly handle various open-ended questions and diverse text inputs. Insufficient in-depth knowledge of specific industries leads to limited accuracy in professional tasks and a tendency to develop "illusions"; high cost makes scaling difficult; and knowledge updates are challenging. Method of the present invention high This method achieves high-precision, domain-adaptive classification. By combining advanced visual processing with a knowledge base and retrieval enhancement mechanisms, it addresses the issue of insufficient professional knowledge in large models. It utilizes knowledge distillation to generate lightweight models, balancing high performance and cost-effectiveness. Its external knowledge base can be dynamically updated based on expert feedback, exhibiting strong adaptability and eliminating the need for expensive retraining. This end-to-end closed-loop optimization process ensures that the system can continuously learn and optimize itself. The initial construction is complex, requiring the integration of multiple advanced technologies such as visual processing, knowledge base, model fine-tuning and distillation; the final performance is highly dependent on the quality of the constructed knowledge base and an effective expert feedback loop.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0140] The above-described contents can be implemented individually or in various combinations, and these variations are all within the protection scope of this invention.

[0141] It should be noted that in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that an article or device comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the article or device comprising said element.

[0142] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. The present invention has been described in detail with reference to preferred embodiments. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications and substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A large model-based unstructured data automatic classification grading method, characterized in that: The method comprises: S1. Collect data classification standard specifications, rules, form a corpus, vectorize the classification rules and hierarchical rules in the corpus based on an embedding model, and respectively construct a classification vector knowledge base and a hierarchical vector knowledge base; S2. Fine-tune the large model based on the two constructed knowledge bases to obtain a domain adaptive classification large model and a hierarchical large model, and obtain a lightweight domain adaptive classification large model and a hierarchical large model based on knowledge distillation, and dynamically update all the large models obtained in the step based on the two constructed knowledge bases; S3. Perform text detection and image segmentation on the unstructured data to be classified and graded, and separate text image blocks containing text from complex backgrounds and interference elements; S4. Perform image understanding and character-level text generation using a Transformer OCR technology, vectorize based on an embedding model to obtain a data feature vector, and then perform similarity matching with the classification vector knowledge base and the hierarchical vector knowledge base based on a retrieval enhancement generation mechanism to obtain matching information; S5. Integrate the unstructured data to be classified and graded with the matching information, input the lightweight domain adaptive classification large model and the hierarchical large model for automatic classification and grading, and continuously update and expand the two classification vector knowledge bases and the hierarchical vector knowledge bases through expert review feedback.

2. The method of claim 1, wherein the method is based on a large model for automatic classification and grading of unstructured data. The S2 comprises: S21. Domain adaptive fine-tuning: adapt a general, pre-trained safety domain basic large language model to the classification and grading field to obtain a teacher model with professional field knowledge; specifically comprising: For an input sample x i The safety domain base large language model output is a probability distribution over all C categories , calculated by the Softmax function: ； wherein, is the output logit of a base large language model in the security domain, C is the total number of classes, : the raw score of the model for "sample belongs to class C", is the output without probability normalization, also known as "logits"; : the raw score of the model for "sample belongs to the jth class", is the single element when traversing all C classes; The fine-tuning process optimizes the safety field base large language model parameters θ by minimizing a loss function, which is a cross-entropy loss L CE : ； where N is the total number of training samples, y i,c is an indicator function that takes the value 1 if the true label of the sample x i is class c, and 0 otherwise; A gradient-based stochastic gradient descent algorithm is used to optimize the safety domain basic large language model parameters θ, and the parameter update rule is represented as: ； where θ t is the parameter at the tth iteration, η is the learning rate, is the gradient of the loss function with respect to the parameter; Through the fine-tuning process, a domain adaptive classification large model and a domain adaptive hierarchical large model are obtained respectively, and are denoted as teacher model M Tclass and teacher model M Tgrade ; the parameters thereof are θ Tclass and θ Tgrade respectively. S22. Knowledge distillation and model lightweight: The domain adaptive classification large model and the hierarchical large model obtained in S21 are used as teacher models, and the knowledge learned by the teacher models is migrated to a student model with small parameter quantity and simple structure to obtain a lightweight domain adaptive classification large model and a hierarchical large model; S23. Incremental learning or continuous fine-tuning through dynamic updating: Let the model parameters at time point t be θ. t When new or updated data forms dataset D in the classification vector knowledge base and the hierarchical vector knowledge base... new At that time, the system will use D new Alternatively, it can be combined with some old data to further fine-tune the existing model; the fine-tuning process is the same as that in S21, which is to minimize the loss function L. CE To update the model parameters and obtain the new parameters θ t+1 ; This dynamic updating process is preferentially applied to the teacher model, and after the teacher model is updated, the knowledge distillation process in S22 is re-executed to generate a new generation of lightweight domain adaptive classification large model and hierarchical large model containing the latest knowledge.

3. The method of claim 1, wherein the method is based on a large model for automatic classification and grading of unstructured data. The S3 comprises: S31. Automatically locate the position and range of all text instances in the input unstructured image using a target detection model YOLOv8; The training process of the object detection model YOLOv8 aims to minimize a composite loss function L YOLOv8 , which includes a classification loss L cls and a bounding box regression loss L box : ； where λ cls and λ box are weight hyperparameters for balancing the respective loss terms; classification loss L cls for measuring how well the target detection model distinguishes text from background; for each prediction, its classification loss L cls is computed as follows: ； Wherein, p is the probability of the target detection model predicting that the region is text, y is the real label, 1 is text, and 0 is background; By using gradient descent optimization algorithm, the network parameters of the target detection model YOLOv8 are iteratively optimized to minimize the total loss function L YOLOv8 , so that the target detection model YOLOv8 can accurately output the bounding boxes of all text regions in the unstructured image. S32. After the text detection determines the boundary box containing the text region, the sub-image in each boundary box is segmented to separate the text characters from the local background and noise, and a binary text image block is generated.

4. The method of claim 1, wherein the method is based on a large model for automatic classification and grading of unstructured data. The S4 comprises: S41. Receive the text image block generated in S3 as input through an image encoder to generate visual context features; specifically comprising: First, uniformly scale the input text image block to a preset size; Subsequently, the scaled text image patch is divided into a series of fixed-size, non-overlapping image patches; these image patches are linearly flattened and combined with positional encodings to form a sequence of visual features as input to the image encoder; The image encoder processes the sequence of visual features through a multi-layer stacked Transformer module to extract deep visual contextual features; S42. The text decoder generates characters or tokens in the text sequence one by one according to the visual contextual features output by the image encoder; at each time step t, the text decoder receives the output of the image encoder and the sequence of characters generated at all previous time steps as input, and predicts the probability distribution of the next character; The image encoder and the text decoder make the large model capture long-distance dependencies inside the image and inside the text sequence through the multi-head self-attention mechanism, and finally generate a text string S highly consistent with the image content ocr； S43. Text Vectorization: Convert the text string S generated in S42 into a fixed-dimension real number vector that can represent its semantic information; use a pre-trained embedding model with a Sentence-BERT architecture to receive a text string as input and output a d-dimensional data feature vector ocr ;​ S44. Adopting a retrieval enhancement generation mechanism to generate the original data feature vector to be processed in S43 As query vectors, similarity search is performed in the classification vector knowledge base K class ={k1,k2,...,k M} and the hierarchical vector knowledge base K grade ={g1,g2,...,g N}, where the vectors in the classification vector knowledge base and the hierarchical vector knowledge base are collectively referred to as document vectors , where k i and g j are d-dimensional vectors; Query vector Similarity between each document vector in the classification vector knowledge base and the hierarchical vector knowledge base Similarity between each document vector in the classification vector knowledge base and the hierarchical vector knowledge base The formula is: ； Unfolded as: ； wherein || is the Euclidean norm of the query vector || is the Euclidean norm of the document vector is the i-th element of is the i-th element of the document vector d is the dimension of the vector;​​​ Computing Similarity scores are computed for all vectors in the classification vector knowledge base and the hierarchical vector knowledge base, and the top-N rules or specifications corresponding to the highest scored text or vectors are retrieved; these retrieved high similarity text or vector sets form the matching information output of this step; this matching information is combined with the original data feature vector together as the input of the domain adaptive classification large model and the hierarchical large model in S5.

5. The method of claim 1, wherein: The S5 comprises: S51. Input data integration: the data feature vector generated in S4 is integrated with the matching information obtained by retrieval in a vector splicing manner to form the final enhanced feature vector of the input large model ; S52. The enhanced feature vector are input into the lightweight domain adaptive classification large model and the hierarchical large model trained in S2, respectively, for hierarchical classification. S53. Expert review: submit the automatic classification and grading results generated in S52 to an artificial expert review interface; domain experts review and confirm the model's determination results and provide correction opinions; S54. Closed-loop feedback and continuous optimization: by using the feedback information of artificial expert review, a closed-loop optimization mechanism is constructed to continuously improve the system performance; the closed-loop optimization mechanism includes two aspects of knowledge base updating and large model iteration.

6. The method of claim 5, wherein the method further comprises: The S52 comprises: For lightweight field-adaptive classification large model M Sclass , the output of which for an input is a probability distribution over all C classes, computed by a softmax function: ; wherein ( , ,..., ) and ; Final automated classification result Class determined to have the highest probability: ; By analogy, automated grading results Also by similar means, the lightweight domain-adaptive grading large model M Sgrade Calculated; Finally, the automatic classification results and the automatic grading results are integrated and output as the final determination of the original unstructured data.

7. A large model-based unstructured data automatic classification grading system, characterized in that, The system comprises a module for executing a large model-based unstructured data automatic classification and grading method according to any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize a large model-based unstructured data automatic classification and grading method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize a large model-based unstructured data automatic classification and grading method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize a large model-based unstructured data automatic classification and grading method according to any one of claims 1 to 6.

Images