Multi-OCR model rearrangement method for AI corpus arrangement

Abstract

The invention provides a multi-OCR model rearrangement method for AI corpus arrangement, and relates to the technical field of artificial intelligence, the method comprises the following steps: constructing a plurality of lightweight OCR sub-models, the lightweight OCR sub-models are obtained by decomposing a large OCR model through knowledge distillation and structured pruning technologies, and model pedigree with different calculation complexity and precision are obtained; based on the model pedigree, evaluating the processing capability of the edge equipment, and establishing an equipment performance prediction model for predicting the reasoning time of different OCR sub-models on the equipment by collecting equipment hardware parameters and executing a benchmark test; dynamically selecting an OCR model combination meeting a preset reasoning time threshold based on the equipment performance prediction model and the real-time resource state of the equipment; and performing OCR processing on the input document image by using the OCR model combination to obtain text contents and position information corresponding to the plurality of text blocks. According to the method, the structured document meeting the AI corpus format requirement can be efficiently and precisely generated at the edge end.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a multi-OCR model rearrangement method for AI corpus processing. BACKGROUND

[0002] An enterprise's field staff needs to quickly process paper procurement contracts on the customer site using a smart phone, the goal being to convert the contract content into structured corpus for training a contract analysis AI model. The existing technical solution has limitations: if a lightweight OCR model pre-installed on the phone is used, the model is simplified in the feature extraction layer to adapt to device resources, resulting in low accuracy in identifying details such as the specification parameter table and small numeric amounts in the contract, and only outputting text blocks arranged in visual order, unable to distinguish logical structures such as contract title clauses, main text, attachments, and instructions. Single multi-page contracts need to be manually corrected and reorganized, which is time-consuming. If a high-precision OCR model is selected for uploading to the cloud, the single request response delay is obvious due to unstable network signals on site, and there is a risk of leakage of sensitive data such as customer pricing strategies and contact information uploaded to third-party clouds. At the same time, the cloud output results still do not integrate semantic logic, and still need to be manually processed into structured documents that meet the AI corpus format, which cannot meet the real-time and data security requirements of the field scene, and increases the time and labor costs of AI corpus production. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a multi-OCR model rearrangement method for AI corpus processing, which realizes efficient and high-precision generation of structured documents that meet the requirements of AI corpus format on the edge.

[0004] To solve the above technical problems, the technical solution of the present application is as follows: In a first aspect, a multi-OCR model rearrangement method for AI corpus processing, the method comprising: Constructing a plurality of lightweight OCR sub-models, the lightweight OCR sub-models being obtained by knowledge distillation and structured pruning technology from a large OCR model, resulting in a model spectrum with different computational complexity and accuracy; Based on the model spectrum, evaluating the processing capacity of the edge device, establishing a device performance prediction model by collecting device hardware parameters and performing benchmark testing, for predicting the inference time of different OCR sub-models on the device; Based on the device performance prediction model and the real-time resource state of the device, dynamically selecting an OCR model combination that meets the preset inference time threshold; using the OCR model combination to perform OCR processing on the input document image, obtaining the text content and position information corresponding to the multiple text blocks; Based on the position information, the text blocks are spatially sorted to obtain an ordered text block sequence; for each text block in the ordered text block sequence, text content features and spatial position features are extracted respectively, and the text content features and the spatial position features are spliced to obtain a comprehensive feature vector of each text block; The comprehensive feature vectors of each text block are arranged in the order of the ordered text block sequence and input into a Transformer encoder for encoding to obtain an encoded feature sequence; Based on the feature sequence, the logical role classification of each text block is performed through a classification head to obtain a logical role classification result of each text block; based on the logical role classification result and the ordered text block sequence, text blocks with the same logical role are merged and reorganized to obtain a structured document.

[0005] Further, a plurality of lightweight OCR sub-models are constructed, and the lightweight OCR sub-models are obtained by knowledge distillation and structured pruning technology from a large OCR model to obtain a model spectrum with different computational complexity and accuracy, including: Select a high-precision teacher OCR model including a text detection module and a text recognition module; Based on the teacher OCR model, a plurality of student model initial architectures with different architecture complexity are constructed by performing channel reduction and network depth reduction operations; The student model initial architecture is taken as a training object, and the output of the teacher OCR model is taken as a soft supervision signal through knowledge distillation technology, and combined with real annotation data for joint optimization training to obtain a group of preliminary lightweight student models which retain part of the knowledge features of the teacher model; The preliminary lightweight student model is taken as a processing object, and a structured pruning operation is applied, and the pruning proportion of each network layer is determined according to the resource constraints of the target edge device, and the channels or weights with lower importance are removed to obtain a pruned model with more sparse structure; The pruned model is supervised fine-tuned to compensate for and restore the recognition accuracy lost in the pruning process, and finally a series of lightweight OCR sub-models with different balances between computational complexity, memory occupation and recognition accuracy are obtained, and a model spectrum with different computational complexity and accuracy is obtained.

[0006] Further, based on the model spectrum, the processing capability of the edge device is evaluated, the device performance prediction model is established by collecting device hardware parameters and performing benchmark tests, which is used to predict the inference time of different OCR sub-models on the device, including: Collect hardware configuration parameters of the target edge device to obtain key indicators including the number of CPU cores, memory capacity and GPU performance; based on the hardware configuration parameters, select at least three benchmark test models with different complexities from the lightweight OCR sub-model family; Run the benchmark model sequentially on the target edge device and record the inference time of the benchmark model when processing standard test images; Based on the inference time and the computational complexity of the corresponding benchmark model, a linear regression model is established to describe the relationship between the device's computing power and the inference time of the benchmark model. The prediction accuracy of the linear regression model was verified. When the prediction error was determined to be within a preset threshold, it was used as the equipment performance prediction model for inference time estimation of the OCR sub-model combination.

[0007] Furthermore, based on the device performance prediction model and the device's real-time resource status, a combination of OCR models that meets a preset inference time threshold is dynamically selected; the OCR model combination is used to perform OCR processing on the input document image to obtain the text content and location information corresponding to multiple text blocks, including: Obtain the real-time available memory of the target edge device, set the maximum allowed inference time threshold, and construct a combination of OCR models to be selected; Based on the OCR model combinations to be selected, the first round of screening is performed on the pre-built lightweight OCR sub-model lineage, retaining all OCR model combinations whose total memory requirement does not exceed the real-time available memory, thus obtaining the first candidate model set; For each OCR model combination in the first candidate model set, the established device performance prediction model is invoked to predict the theoretical inference time, and a prediction time label is attached to each candidate model combination. Step 3.4: Based on the maximum allowed inference time threshold, perform a second round of screening on the first candidate model set with attached prediction time labels, and remove OCR model combinations whose prediction time exceeds the threshold to obtain the second candidate model set; From the second set of candidate models, an OCR model combination is selected as the final OCR model combination for the current document image processing task, based on the predetermined optimization objective. The selected final OCR model combination is deployed to the operating environment of the target edge device to drive optical character recognition processing on the input document image, thereby obtaining the coordinate position information of the content information of all text blocks in the document image.

[0008] Furthermore, based on location information, the text blocks are spatially sorted to obtain an ordered sequence of text blocks. For each text block in the ordered sequence, text content features and spatial location features are extracted, and the text content features and spatial location features are concatenated to obtain a comprehensive feature vector for each text block, including: Based on the positional information of each text block output by OCR processing, spatial sorting is performed on all text blocks under preset rules to obtain an ordered sequence of text blocks that reflects their spatial distribution order. For each text block in the ordered text block sequence, two parallel feature extraction operations are performed: the semantic embedding of the text content is extracted as the text content feature through a pre-trained language model, and the spatial location feature representing its spatial attributes is calculated based on the location coordinates. Text content features and spatial location features are concatenated and fused to obtain a comprehensive feature vector for each text block.

[0009] Furthermore, the comprehensive feature vectors of each text block are arranged in the order of the ordered text block sequence and input into the Transformer encoder for encoding, resulting in the encoded feature sequence, including: The vectors in the comprehensive feature vector sequence are arranged according to their original order in the sequence to form the input sequence of the Transformer encoder. Positional encoding is applied to the input sequence to inject spatial order information of text blocks into the vector representation, resulting in an enhanced input sequence that the encoder is aware of positional relationships; the enhanced input sequence is then fed into a pre-trained Transformer encoder. By using a multi-layer self-attention mechanism and a feedforward network of the Transformer encoder, global context modeling and feature transformation are performed on the enhanced input sequence, resulting in an encoded feature sequence that integrates global semantic and structural information of the document.

[0010] Furthermore, based on the feature sequence, each text block is logically classified using a classification head to obtain the logical role classification result for each text block; based on the logical role classification result and the ordered text block sequence, text blocks with the same logical role are merged and recombined to obtain a structured document, including: The encoded feature sequence is used as input and fed into a pre-trained classification head model. The classification head model processes each feature vector in the encoded feature sequence to predict and obtain the logical role category corresponding to each text block, thus obtaining the logical role classification result for each text block. Based on the logical role classification results and combined with the actual content of the text blocks in the ordered text block sequence, adjacent text blocks with the same logical role category are merged and semantically reorganized to obtain logically coherent text fragments. Text fragments are organized, packaged, and formatted according to their corresponding logical roles, ultimately resulting in an electronic document with a clear hierarchical structure that meets the format requirements of artificial intelligence corpora.

[0011] Secondly, the multi-OCR model reordering system for AI corpus processing includes: The acquisition module is used to construct multiple lightweight OCR sub-models. The lightweight OCR sub-models are obtained by decomposing the large OCR model through knowledge distillation and structured pruning techniques, resulting in a model family with different computational complexity and accuracy. The evaluation module is used to evaluate the processing capabilities of edge devices based on the model family. By collecting device hardware parameters and performing benchmark tests, it establishes a device performance prediction model to predict the inference time of different OCR sub-models on the device. The execution module is used to dynamically select an OCR model combination that meets a preset inference time threshold based on the device performance prediction model and the real-time resource status of the device; and to use the OCR model combination to perform OCR processing on the input document image to obtain the text content and location information corresponding to multiple text blocks. The extraction module is used to spatially sort text blocks based on location information to obtain an ordered sequence of text blocks; for each text block in the ordered sequence, text content features and spatial location features are extracted respectively, and the text content features and spatial location features are concatenated to obtain a comprehensive feature vector for each text block; The arrangement module is used to arrange the comprehensive feature vector of each text block in the order of the ordered text block sequence, and input it into the Transformer encoder for encoding to obtain the encoded feature sequence; The merging module is used to classify the logical roles of each text block based on the feature sequence and the classification head, and obtain the logical role classification result of each text block. Based on the logical role classification result and the ordered text block sequence, the text blocks with the same logical role are merged and recombined to obtain a structured document.

[0012] Thirdly, a computing device includes: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0013] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0014] The above-described solution of the present invention has at least the following beneficial effects: This method constructs a teacher and student model system, uses knowledge distillation and structured pruning techniques to generate a lightweight OCR sub-model family with varying complexity and precision, and combines edge device hardware parameter acquisition, benchmark testing, and linear regression to establish a device performance prediction model. Then, it dynamically selects the optimal OCR model combination through initial screening with memory constraints and secondary screening with time thresholds. Simultaneously, it integrates semantic and spatial features into the text blocks output by OCR, and achieves structured recombination through Transformer encoder encoding and logical role classification by the classification head. By employing these techniques, it effectively overcomes the technical problems of balancing OCR model accuracy and real-time performance under resource-constrained edge devices, high latency in cloud computing networks and the risk of sensitive data leakage, and the fragmented and logically unstructured output of traditional OCR requiring extensive manual proofreading. This results in a significant reduction in edge device memory usage and computing resource consumption, OCR processing time controllable within preset thresholds, and output text directly forming structured documents conforming to AI corpus formats, eliminating the need for post-processing manual editing, reducing labor costs, and improving the efficiency of AI corpus production. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the multi-OCR model rearrangement method for AI corpus processing provided in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of a multi-OCR model rearrangement system for AI corpus processing provided in an embodiment of the present invention. Detailed Implementation

[0017] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0018] like Figure 1 As shown, embodiments of the present invention propose a multi-OCR model rearrangement method for AI corpus processing, the method comprising the following steps: Step 1: Construct multiple lightweight OCR sub-models. The lightweight OCR sub-models are obtained by decomposing the large OCR model through knowledge distillation and structured pruning techniques, resulting in a model family with different computational complexity and accuracy. Step 2: Based on the model spectrum, evaluate the processing capabilities of edge devices. By collecting device hardware parameters and performing benchmark tests, establish a device performance prediction model to predict the inference time of different OCR sub-models on the device. Step 3: Based on the device performance prediction model and the real-time resource status of the device, dynamically select the OCR model combination that meets the preset inference time threshold; use the OCR model combination to perform OCR processing on the input document image to obtain the text content and location information corresponding to multiple text blocks; Step 4: Based on the location information, spatially sort the text blocks to obtain an ordered text block sequence; for each text block in the ordered text block sequence, extract the text content features and spatial location features respectively, and concatenate the text content features and spatial location features to obtain the comprehensive feature vector of each text block; Step 5: Arrange the comprehensive feature vector of each text block in the order of the ordered text block sequence, and input it into the Transformer encoder for encoding to obtain the encoded feature sequence; Step 6: Based on the feature sequence, classify each text block into logical roles using the classification head to obtain the logical role classification result for each text block; based on the logical role classification result and the ordered text block sequence, merge and reorganize text blocks with the same logical role to obtain a structured document.

[0019] In this embodiment of the invention, the method effectively overcomes the technical problems of traditional OCR large models being difficult to run in real time under limited edge device resources, lightweight models having insufficient accuracy and relying on cloud computing, being susceptible to network latency and data privacy risks, and outputting fragmented text blocks without logical structure that require a lot of manual proofreading. This is achieved by using knowledge distillation and structured pruning to construct a lightweight OCR sub-model family with multiple complexities / accuracies, collecting edge device hardware parameters and establishing a performance prediction model in combination with benchmark tests to dynamically select the optimal model combination that meets the time threshold, and spatially sorting and fusing content and position features of OCR output text blocks, followed by Transformer encoding and logical role classification of classification heads to achieve structured reorganization.

[0020] In a preferred embodiment of the present invention, step 1 above may include: Step 1.1: Select a high-precision teacher OCR model that includes a text detection module and a text recognition module. Specifically, this includes selecting a high-precision teacher OCR model that includes both text detection and text recognition modules as the knowledge source for model compression. The teacher model selection criteria are that it should possess high-precision OCR capabilities. Taking the DBNet+CRNN combination as an example, the structure of the DBNet text detection model can be represented as follows: DBNet represents a text detection model. The backbone network is the fundamental feature extraction module of the model. That is, a 50-layer residual network, which is a commonly used backbone feature extraction network. The neck network is used for feature fusion. That is, a feature pyramid network, used to fuse features from different levels. The head network is used to output detection results. v is the output module for text detection. The structure of the CRNN text recognition model can be represented as: CRNN = CNN (VGG) → RNN (Bidirectional LSTM) → CTC decoding. CRNN, or Convolutional Recurrent Neural Network, is a classic text recognition model. CNN, or Convolutional Neural Network, is used to extract visual features of text. VGG is a classic CNN network structure. RNN, or Recurrent Neural Network, is used to process sequence features. Bidirectional LSTM is a Long Short-Term Memory network with bidirectional sequence processing capabilities. CTC, or Connection Temporal Classification, is a decoding method used to handle sequences of variable length alignment. The teacher model is pre-trained on ImageNet and uses a large... The large-scale document image dataset is fine-tuned to ensure that its output contains rich text location and content information. The text detection module of the teacher model needs to be able to accurately locate various text regions in the procurement contract and clearly identify the location of different types of text blocks such as titles, clause texts, specification parameter tables, and amount annotations. The text recognition module needs to have high accuracy recognition capabilities and be able to accurately identify details such as lowercase amount numbers, specification parameter characters, and special clause expressions in the contract. This ensures that it can provide comprehensive and high-precision knowledge references for the student model and lay the knowledge foundation for solving the problem of low accuracy in recognizing contract details by lightweight models on mobile devices.

[0021] The loss function of the text detection student model combines the bounding box regression loss from the ground truth annotations with the soft label loss from the teacher model output: ,in, This represents the total loss of the student model for the text detection task. α and β are hyperparameters for balancing the weights, typically set to α=0.1 and β=0.9 to emphasize the knowledge of the teacher model. Lreg is the bounding box regression loss. That is, the actual text bounding box. That is, the text bounding boxes predicted by the student model, and Lce, which is the cross-entropy loss used to match the region probability map of the teacher model. That is, the probability distribution output by the teacher model on the text detection task. This refers to the probability distribution output by the student model on the text detection task, where Lreg is the bounding box regression loss, using the SmoothL1 loss function: ,in, It is a robust regression loss. The parameters of the true bounding box, The bounding box parameters predicted by the student model, and Lce, the cross-entropy loss used to match the region probability map of the teacher model. , The region probability map output by the teacher model. The student model outputs a region probability map, where N is the total number of samples, i=1 to N represents traversing all input samples, and i is the index of the sample, identifying the i-th sample.

[0022] Step 1.2: Based on the teacher OCR model, multiple initial architectures of student models with different architectural complexities are constructed by performing channel reduction and network depth reduction operations. Specifically, this includes: based on the selected teacher OCR model, and combined with the resource characteristics of smartphones such as memory capacity and CPU computing power, channel reduction and network depth reduction operations are performed to construct multiple initial architectures of student models with different architectural complexities. Student model group construction method: Channel reduction: replacing standard convolution with depthwise separable convolution to reduce the number of channels. For example, reducing the 512 channels in the teacher model to 128 channels can reduce the computational complexity by about 75%. Network depth reduction: removing some intermediate layers or reducing the number of residual blocks. For example, reducing the 12 residual blocks in the teacher model to 4 can reduce the computational load by about 66%. Backbone replacement: replacing ResNet50 in the teacher model with MobileNetV3, reducing the number of parameters from 25.5M to 5.5M. Knowledge distillation: softening the teacher model output to enable the student model to learn hidden knowledge. The logits of the teacher model are softened by the temperature parameter T using Softmax. ,in The teacher model outputs the class probability distribution for input sample x. It is a Softmax function with a temperature coefficient, used to convert the model's raw output into a probability distribution. The teacher model's original output for the i-th category, This is a temperature coefficient used to adjust the smoothness of the probability distribution. When T=1, the output distribution is close to the standard Softmax, retaining more hard label information; when T>1, the distribution is softened, allowing the student model to learn the teacher model's implicit knowledge. The optimal temperature value is determined experimentally, typically between 3 and 10. The natural exponential function and channel reduction operations reduce the number of channels in each network layer of the model, thereby reducing the model's memory requirements on the mobile phone. The network depth reduction operation reduces the number of intermediate network layers or residual blocks in the model, thereby reducing the computational pressure on the mobile phone. Based on the resource differences of smartphones with different configurations, multiple channel reduction ratios and network depth reduction schemes are designed. For example, for smartphones with higher configurations, an initial architecture with a lower channel reduction ratio and a higher network depth retention is designed, while for smartphones with lower configurations, an initial architecture with a higher channel reduction ratio and a lower network depth retention is designed, resulting in multiple initial architectures of student models adapted to different mobile phone resources.

[0023] Step 1.3: Using the initial architecture of the student model as the training object, knowledge distillation technology is employed, with the output of the teacher's OCR model serving as the soft supervision signal. Joint optimization training is then performed using real-world labeled data to obtain a preliminary lightweight student model that retains some knowledge features of the teacher model. Specifically, this involves using the constructed initial architecture of the student model as the training object and employing knowledge distillation technology for training. During training, the processing results of the teacher's OCR model on the procurement contract image are used as the soft supervision signal. This soft supervision signal includes information such as the teacher model's predicted probability distribution of the contract text region and the confidence level of text content recognition. Simultaneously, a large number of accurately labeled procurement contract images are introduced as real-world labeled data. The labels include the accurate characters of the contract text and the logical structure categories of text blocks. Through joint optimization training, the student model learns the contract text recognition rules from the real-world labeled data and absorbs the knowledge and experience of the teacher model in processing contract details. Ultimately, a preliminary lightweight student model that retains some knowledge features of the teacher model and can initially recognize contracts is obtained. The loss function of the student model for text recognition also combines the real labels and the teacher model output: Lrec = γ × Lce(labelgt, ys) + δ × Lkl(Oteacherrec / T, Ostudentrec / T), where Lrec is the total loss of the recognition task, γ and δ are hyperparameters for balancing weights, usually set to γ ​​= 0.7 and δ = 0.3 to take into account both real labels and teacher knowledge, Lce is the cross-entropy loss based on real character labels, used to measure the difference between the student model output and the real labels, labelgt is the real label of the task, ys is the recognition output of the student model, Lkl is the KL divergence, used to bring the probability distribution of the student model output and the teacher model output after scaling by the temperature parameter T, Oteacherrec is the output of the teacher model on the recognition task, T is the temperature coefficient, used to smooth the probability distribution, Ostudentrec is the output of the student model on the recognition task, and Lce is the cross-entropy loss based on real character labels; ,in, That is, real character tags, This represents the output probability of the student model. N is the total number of samples, where i = 1 to N represents traversing all input samples, i is the i-th sample, C is the total number of character categories, and c = 1 to C represents traversing all categories, where c is the index of the character category. The natural logarithm of the student model output probability, and Lkl, the KL divergence, are used to narrow down the probability distributions of the student model output and the teacher model output after scaling by the temperature parameter T. ,in, That is, the probability distribution output by the teacher model. That is, the probability distribution output by the student model.

[0024] Step 1.4: Using the initial lightweight student model as the processing object, apply structured pruning operations. Determine the pruning ratio of each network layer based on the resource constraints of the target edge device, and remove channels or weights deemed to be of low importance to obtain a more sparse pruned model. Specifically, this includes: using the initial lightweight student model as the processing object, applying structured pruning operations, and analyzing the real-time resource constraints of the target smartphone before pruning, including currently available memory and real-time CPU computing power. Based on these constraints, determine the pruning ratio of each network layer of the model. For example, when the available memory of the phone is small, appropriately increase the pruning ratio of non-critical network layers of the model; when the CPU computing power of the phone is limited, focus on reducing the number of channels in computationally intensive network layers in the model.

[0025] Channel pruning based on BN layer scaling factor: Channel importance assessment: Using the scaling factor γ in the BN layer as a channel importance index: ,in Let be the scaling factor for the c-th channel. This involves the importance score of the c-th channel, channel ranking, and pruning. To reduce the computational complexity and parameter count of the neural network model, a structured pruning method based on channel importance is adopted. For each output channel of a convolutional layer, an importance score is calculated based on its mean gradient magnitude, L1 norm, or feature map response sparsity on the validation set. Then, the channels are ranked from lowest to highest score. The pruning rate is determined by the ratio of the number of pruned channels to the total number of original channels in that layer. Npruned is the number of channels pruned in this layer, and Ntotal is the original total number of channels in this layer. During the pruning process, the contribution of each channel and weight of the model to the recognition of the procurement contract text is evaluated, and channels or weights that are judged to be less important are removed, such as those that contribute very little to the recognition of contract amount figures and specifications. In the end, a pruned model with a sparser structure and further reduced mobile phone resource consumption is obtained.

[0026] Step 1.5 involves supervised fine-tuning of the pruned model to compensate for and restore the recognition accuracy lost during the pruning process. This results in a series of lightweight OCR sub-models that achieve different balances between computational complexity, memory usage, and recognition accuracy, creating a model family with varying computational complexity and accuracy. Specifically, this involves supervised fine-tuning of the pruned model to compensate for and restore any recognition accuracy lost during pruning. The data used for fine-tuning primarily consists of procurement contract images, with a focus on selecting images of easily misidentified areas within the contracts, such as images containing lowercase monetary amounts or complex specification tables. Enhanced training was conducted for text recognition tasks in these areas. By fine-tuning and optimizing model parameters, the pruned model was re-adapted to the text recognition requirements of procurement contracts, especially improving the accuracy of recognizing contract details. Finally, through multiple rounds of fine-tuning, a series of lightweight OCR sub-models were obtained, achieving different balances between computational complexity, memory usage, and recognition accuracy. For example, some models have low computational cost while ensuring accuracy in recognizing contract details, making them suitable for low-configuration mobile phones; other models have higher recognition accuracy and are suitable for handling contracts containing complex clauses. These sub-models together constitute a model family that can be dynamically selected based on the real-time status of the mobile phone.

[0027] Post-pruning fine-tuning recovery process: To compensate for the performance degradation caused by pruning, supervised fine-tuning is performed on the pruned sparse model. Specific parameter configurations are as follows: number of fine-tuning rounds: 50 rounds, optimizer: AdamW, initial learning rate set to... Cosine annealing is used for attenuation, batch size is 32, and the loss function weight adjustment strategy is to dynamically adjust the loss weights of each subtask for the multi-task model. The total loss is set as follows: ,in, The total loss of the model is a weighted sum of the detection loss and the classification loss, used to measure the overall prediction bias of the model. This is the weighting coefficient for the detection loss, used to balance the proportion of detection loss in the total loss. To detect the loss, To classify the loss, during the fine-tuning process, the first 10 rounds were set... =0.7, adjusted to 0.7 in the last 40 rounds. =0.5, through fine-tuning strategy, the pruning model can recover to a performance level close to the original model in fewer training rounds. Mask matrix generation, generate binary mask matrix M, and achieve parameter sparsity by element-wise multiplication; ,in, This is the new weight matrix obtained after the pruning operation, where W represents the original parameters, M is the mask matrix, and ⊙ denotes element-wise multiplication. Fine-tuning to restore accuracy: Fine-tuning the pruned model to recover some of the accuracy loss: fine-tuning the loss function. ,in, For the detection loss, Lstudent is the original loss of the student model, and φ is the adjustment weight, which is usually set to 0.5.

[0028] In this embodiment of the invention, a high-precision teacher OCR model containing text detection and recognition modules is selected as the knowledge source. Then, a student model with multiple architectural complexities is constructed through channel reduction and network depth reduction. A preliminary lightweight student model is obtained through joint training with knowledge distillation and real labeled data. Then, structured pruning is applied according to the resource constraints of edge devices, and supervised fine-tuning is performed on the pruned model. Therefore, this invention effectively overcomes the technical problems of traditional OCR technology, such as the high computation and memory consumption of large models making them difficult to adapt to edge devices, the lack of knowledge transfer and accuracy compensation in simple lightweight models leading to insufficient recognition accuracy, and the lack of multiple precision models to adapt to different edge device resource requirements. This results in the construction of a series of lightweight OCR sub-models that can achieve different balances between computational complexity, memory consumption, and recognition accuracy. This provides a basic support for dynamically selecting the optimal model combination according to the actual situation of edge devices, ensuring that edge devices can still maintain high OCR recognition performance when using lightweight models.

[0029] In a preferred embodiment of the present invention, step 2 above may include: Step 2.1: Collect the hardware configuration parameters of the target edge device to obtain key indicators including the number of CPU cores, memory capacity, and GPU performance. Based on the hardware configuration parameters, select at least three benchmark test models of different complexities from the lightweight OCR sub-model family. Specifically, this includes: collecting key hardware configuration indicators of the target smartphone, including the number of CPU cores, memory capacity, and GPU performance, to fully understand the basic computing and storage resources of the device; combining these hardware configuration parameters, selecting at least three benchmark test models of different complexities from the lightweight OCR sub-model family. The selected models need to cover different levels of computational intensity and accuracy, including lightweight models adapted to low-configuration phones and medium-to-high complexity models that can handle complex contract texts. The benchmark tests can comprehensively reflect the device's ability to process OCR tasks of different difficulty levels, including parameter memory computation. ,in For parameter memory, ParamCount is the number of model parameters, DataTypeSize is the data type size, and peak activation memory is used for computation. Where Hl, Wl, and Cl are the height, width, and number of channels of the feature map of layer l, and DataTypeSize is the size of the data type. , representing active memory, which is the total amount of memory occupied by the feature maps output by each layer during the neural network inference process.

[0030] Step 2.2: Run the benchmark test models sequentially on the target edge device and record the inference time of the benchmark test models when processing standard test images. Specifically, this includes: running the selected benchmark test models sequentially on the target smartphone. The standard test images used in the tests simulate the actual characteristics of procurement contracts, including typical text areas such as contract titles, clauses, body text, specification parameter tables, and amount annotations, and covering scenarios with different font density and clarity. During the operation of each benchmark test model, accurately record its inference time for processing the complete standard test image to ensure that the recorded results can truly reflect the actual running efficiency of the model in processing contract documents on the mobile phone.

[0031] Step 2.3: Based on the inference time and the computational complexity of the corresponding benchmark model, establish a linear regression model describing the relationship between the device's computing power and the inference time of the benchmark model. Specifically, this includes: compiling inference time data for each benchmark model, and simultaneously calculating the computational complexity of each benchmark model for contract text processing, clarifying the computational resource consumption required by different models to process various types of text content in contracts. Based on this data, establish a linear regression model describing the relationship between the smartphone's computing power and the inference time of the benchmark model. This allows the model to quickly map the approximate inference time for processing procurement contracts on the phone based on the computational complexity of any subsequent OCR sub-model. (Linear regression model, model formula). ,in, It is the model inference time. This represents the total computational complexity of the model. a and b are the regression coefficients and intercept, respectively. The parameters are calculated using the least squares method, with a and b calculated using three benchmark test points: (C1, T1), (C2, T2), and (C3, T3). , Where n=3 is the number of test points. It is the value of the independent variable corresponding to the i-th sample. It represents the value of the dependent variable corresponding to the i-th sample. Model validation is performed by evaluating the fit of the regression model through the R² value. R² ≥ 0.9 indicates that the model is effective.

[0032] Step 2.4 verifies the prediction accuracy of the linear regression model. When the prediction error is determined to be within a preset threshold, it is used as the device performance prediction model for estimating the inference time of the OCR sub-model combination. Specifically, this includes selecting additional test images related to the procurement contract and lightweight OCR sub-models not involved in the modeling, using the established linear regression model to estimate the inference time of these sub-models, and comparing the estimated results with the actual running time of the sub-models on the smartphone to calculate the error between the two and verify the prediction accuracy of the linear regression model. When the prediction error is determined to be within a preset reasonable threshold, the linear regression model can be used as the device performance prediction model for estimating the inference time of different OCR sub-model combinations when processing procurement contract tasks, providing a reliable basis for model selection.

[0033] In this embodiment of the invention, by collecting key hardware indicators such as the number of CPU cores, memory capacity, and GPU performance of the target edge device, selecting at least three benchmark test models of different complexities from the lightweight OCR sub-model family, running them on the device, and recording their inference time for processing standard test images, and combining the inference time with the model's computational complexity to establish a linear regression model, which is then verified for accuracy and used as a device performance prediction model, this technical means effectively overcomes the problem in traditional technologies that cannot accurately quantify the matching relationship between the edge device's computing power and the OCR model's inference efficiency. This leads to the inability to predict the performance of different OCR sub-models on the device in advance, making it difficult to achieve accurate matching between the model and device resources. As a result, this invention achieves the ability to quickly and accurately predict the inference time of various OCR sub-model combinations on the target edge device, providing reliable data support for the subsequent dynamic selection of the optimal model combination that both adapts to device resource constraints and meets real-time requirements, thus avoiding the technical effect of blindly selecting models.

[0034] In a preferred embodiment of the present invention, step 3 above may include: Step 3.1: Obtain the real-time available memory of the target edge device and set a maximum allowable inference time threshold to form a candidate OCR model combination. This includes: using the smartphone's system functions to obtain the device's current available memory in real time; comprehensively checking the memory resources occupied by various background applications to ensure that the obtained available memory data accurately reflects the device's actual available storage capacity when processing procurement contract tasks; considering the actual work needs of field personnel to quickly complete contract processing at the customer's site; and referring to reasonable processing time standards in similar scenarios, setting a scientific maximum allowable inference time threshold. This threshold must sufficiently ensure that field personnel can obtain processing results promptly on-site, without affecting work progress due to excessive waiting time. Based on the obtained real-time available memory and the set maximum allowable inference time threshold, clarify the selection boundary conditions for the OCR model combination to ensure that all candidate OCR model combinations can operate effectively within device resource and time constraints, laying the foundation for subsequent accurate model selection.

[0035] Step 3.2: Based on the OCR model combinations to be selected, perform the first round of screening on the pre-built lightweight OCR sub-model lineage, retaining all OCR model combinations whose total memory requirement does not exceed the real-time available memory, to obtain the first candidate model set. Specifically, this includes: using a model initial screening algorithm to fully traverse all possible OCR model combinations in the pre-built lightweight OCR sub-model lineage. For each model combination, the total memory consumption required during its operation is calculated one by one. This total memory consumption strictly covers the memory usage of the model's own parameters and the peak activation memory usage generated when processing the procurement contract image, ensuring that the assessment of the model's memory requirements is comprehensive and accurate, fully in line with the memory dimension judgment criteria of the model screening algorithm. According to the core logic of the model screening algorithm, the total memory requirement of each OCR model combination is precisely compared with the previously obtained real-time available memory of the smartphone, and the first round of screening is performed. The algorithm will automatically retain all OCR model combinations whose total memory requirement does not exceed the real-time available memory of the smartphone, forming the first candidate model set. Through this round of screening by the model screening algorithm, model combinations that may cause device lag, interruption of processing tasks, or inability to start due to excessive memory usage can be completely eliminated. From the algorithm level, it is guaranteed that all models in the selected first candidate model set can run stably and smoothly under the current storage resource conditions of the smartphone.

[0036] The purpose of the initial model screening algorithm is to filter all model combinations in the model library L based on the available memory of the device, removing model combinations whose memory requirements exceed the limit. For each model (d, r) ∈ L, its total memory requirement is: The filtered model set F is defined as follows: , where L is the model library, which contains multiple model pairs, each model pair is (d, r), where d represents the detector model, r represents the recognizer model, Mparam represents the memory usage of the model parameters, Mactivation represents the memory usage of the model activation, and Mavailable represents the available memory of the device.

[0037] Step 3.3: For each OCR model combination in the first candidate model set, call the established device performance prediction model to predict the theoretical inference time and attach a prediction time label to each candidate model combination. Specifically, for each OCR model combination in the first candidate model set, call the previously established device performance prediction model one by one. The device performance prediction model combines the specific features of the procurement contract image, including factors affecting computational complexity such as text layout density, font type, specification parameter table structure, and presentation format of monetary numbers in the contract, to accurately predict the theoretical inference time for each OCR model combination to process this type of procurement contract image. After completing the theoretical inference time prediction for each model combination, add a clear prediction time label to the corresponding candidate model combination, clearly and intuitively presenting the real-time performance of each model combination in processing procurement contract tasks on the current smartphone, providing clear and referable data basis for subsequent time-constrained screening.

[0038] Step 3.4: Based on the maximum allowed inference time threshold, a second round of screening is performed on the first candidate model set with attached prediction time labels. OCR model combinations whose prediction times exceed the threshold are removed, resulting in a second candidate model set. Specifically, this involves using the set maximum allowed inference time threshold as the core screening criterion to perform a second round of screening on the first candidate model set with attached prediction time labels. The prediction time label of each OCR model combination in the first candidate model set is checked one by one, and all OCR model combinations whose prediction inference times exceed the maximum allowed inference time threshold are removed. This round of screening yields the second candidate model set. The screening process strictly ensures that all model combinations in the selected second candidate model set can meet the core real-time requirements of field operations, avoiding situations where field personnel have to wait too long at the customer's site due to excessive processing time. Furthermore, since all model combinations run locally, there is no need to upload them to the cloud.

[0039] Step 3.5: From the second candidate model set, select an OCR model combination as the final OCR model combination for the current document image processing task based on the predetermined optimization objective. Specifically, this includes: defining the predetermined optimization objective for this document image processing task, which prioritizes meeting memory and time constraints, ensuring high accuracy in recognizing text content in the procurement contract, especially for details such as specifications and parameters, and amounts in lowercase numerals, which are prone to errors, while also considering model stability and efficiency. Based on this optimization objective, conduct a comprehensive performance evaluation of each OCR model combination in the second candidate model set, focusing on comparing the recognition performance of different model combinations when processing typical text areas of the procurement contract, including the accuracy of text blocks corresponding to different logical structures such as contract titles, clauses, body text, and appendices. Based on the comprehensive evaluation results, select an OCR model combination from the second candidate model set that achieves the optimal balance between recognition accuracy, stability, and operational efficiency as the final OCR model combination for the current procurement contract image processing task, effectively solving the problem of insufficient recognition accuracy of traditional mobile-based lightweight OCR models.

[0040] Step 3.6: Deploy the selected final OCR model combination to the operating environment of the target edge device, driving optical character recognition processing on the input document image to obtain the coordinate position information of the content information of all text blocks in the document image. Specifically, this includes: deploying the selected final OCR model combination to the local operating environment of the target smartphone. The entire deployment process does not rely on external network connections or third-party cloud servers. All data processing operations are completed locally, ensuring that sensitive information such as customer pricing strategies and contact information contained in the procurement contract will not be transmitted to the external environment, maximizing data security. After deployment, drive the OCR model combination to perform comprehensive optical character recognition processing on the procurement contract image input by field personnel through smartphones. During the recognition process, accurately extract the complete content information of all text blocks in the procurement contract image, covering various key text contents such as contract titles, clauses, body text, specifications, parameter tables, amounts, numbers, attachments, and descriptions. At the same time, accurately record the specific coordinate position information of each text block in the original image, providing complete and accurate data support for the structured reorganization of text blocks, solving the problem that traditional OCR output can only be arranged in visual order and lacks logical structure.

[0041] Model deployment optimization: To reduce the storage and computational overhead of the neural network model on terminal devices, integer quantization technology is used to convert the original floating-point model into an 8-bit integer model. Specifically, the model weights W and activation values ​​A are both converted from the floating-point domain to the integer domain through a linear mapping. ,in, These are 8-bit integer weights obtained after quantization. The rounding operation converts the floating-point result after quantization to an integer. αW and βW represent the minimum and maximum values ​​of the weight tensor, respectively, or the quantization range boundary dynamically determined based on calibration data. To reduce the accuracy loss introduced by quantization, this invention uses a calibration method based on KL divergence to determine the optimal quantization parameters. The specific steps are as follows: Select a small number of unlabeled samples from the training or validation set as calibration data; input the calibration data into the original FP32 model and record the floating-point distribution of the activation output of each layer; perform quantization-dequantization operations on the activation values ​​under different quantization ranges to generate corresponding approximate floating-point outputs; for each candidate quantization range, calculate the KL divergence between its simulated output distribution and the original FP32 output distribution: ,in, It is the KL divergence, used to measure the difference between two probability distributions. P(i) is the probability distribution output by the original FP32 model, and Q(i) is the approximate probability distribution output by the model after INT8 quantization. P is the FP32 activation distribution, and Q is the reconstructed distribution after quantization.

[0042] In this embodiment of the invention, by employing the technical means of obtaining the real-time available memory of the target edge device, setting the maximum allowable inference time threshold, screening the lightweight OCR sub-model family through initial screening of memory constraints and secondary screening of time thresholds, and then selecting the final model combination according to the predetermined optimization goal and deploying it to the device to perform optical character recognition processing, the invention effectively overcomes the technical problems of traditional technologies, such as the lack of specificity in the selection of OCR model combinations, inability to adapt to the real-time resource status of edge devices, easy occurrence of insufficient memory or inference timeouts, and reliance on cloud processing with latency and data leakage risks, making it difficult to meet the real-time and data security requirements of field work. This achieves precise adaptation between the OCR model combination and the real-time resources of the edge device, ensuring that the model does not occupy too much memory during operation and that the inference time is controlled within the allowable threshold, avoiding the security risks and latency problems caused by cloud reliance, and efficiently outputting the content and location information of text blocks, providing reliable data support for subsequent corpus structured reorganization.

[0043] In a preferred embodiment of the present invention, step 4 above may include: Step 4.1: Based on the positional information of each text block output by OCR processing, perform spatial sorting on all text blocks under preset rules to obtain an ordered sequence of text blocks that reflects their spatial distribution order. Specifically, this includes: relying on the final OCR model combination selected by the optimal model selection algorithm, obtaining the positional information of all text blocks output by the model combination after processing the procurement contract image. Since the optimal model selection algorithm has selected the model combination that is suitable for smartphone resources and has the best recognition accuracy, its output positional information contains the precise coordinate data of each text block in the procurement contract image, which can restore the actual placement position of the text blocks in the original contract to the greatest extent and avoid the distortion of positional information caused by model recognition deviation. Based on the typical layout of procurement contracts, including centered or left-aligned titles, segmented clauses, and column-based specification tables, as well as people's natural reading habits, a clear and context-appropriate spatial sorting rule was established. This rule prioritizes the vertical position of text blocks, followed by their horizontal position. Specifically, text blocks are first sorted from top to bottom according to their vertical coordinates in the image. When multiple text blocks are within the same vertical height range, they are then sorted from left to right according to their horizontal coordinates. Following this pre-defined rule, a systematic spatial sorting operation is performed on all text blocks carrying precise location information, ultimately resulting in an ordered sequence of text blocks that accurately reflects their spatial distribution within the procurement contract image. This sequence benefits from the precise location data provided by the optimal model combination and aligns with the normal logic of reading procurement contracts, completely avoiding the chaotic text block output of traditional OCR. It provides a precise and ordered data foundation for feature extraction and structured reconstruction.

[0044] The model selection algorithm selects the model combination with the highest total computational complexity from the filtered model set F, while satisfying the maximum inference time constraint Tmax. For each model pair (d, r) ∈ F, its total computational complexity and prediction inference time are: , The optimal model It is a model that satisfies the following conditions: ,in, Let represent the total computational complexity of the model pair, d represent the detector model, r represent the recognizer model, and C represent the computational complexity of model m. The linear regression model has parameters a and b, representing the prediction inference time of the model, where a is the slope and b is the intercept. For the optimal model pair, This is the operator for finding the parameter corresponding to the maximum value. It is used to find the model pair with the largest total computational complexity that meets the conditions. This represents the maximum allowed inference time threshold. If multiple model pairs have the same maximum complexity, then any one of them is selected.

[0045] Step 4.2: For each text block in the ordered text block sequence, perform two parallel feature extraction operations: extract the semantic embedding of the text content as text content features using a pre-trained language model, and calculate the spatial location features representing its spatial attributes based on the position coordinates. Specifically, for each text block in the ordered text block sequence, start two feature extraction operations simultaneously to ensure that the two operations are executed in parallel to improve processing efficiency and meet the real-time requirements of field scenarios. The first operation is to extract text content features. A pre-trained language model specifically trained for legal contract texts is used. The specific content of each text block is input into the pre-trained language model. The model extracts semantic embeddings that can represent the core meaning of the text through in-depth analysis of professional terms, clauses, specifications, parameters, amounts, and other content in the contract text. These are used as the text content features of the text block. This feature can effectively distinguish the semantic attributes of different text blocks, such as distinguishing the text content corresponding to the title clause body and the specification parameter table. The second operation is to extract spatial location features. Based on the coordinate data of each text block, the center point coordinates, width, height, and aspect ratio of the text block are calculated. These parameters can comprehensively characterize the spatial attributes of the text block in the procurement contract image. For example, the center point coordinates can determine whether the text block is at the top, middle, or bottom of the contract, and the aspect ratio can distinguish whether it is a regular paragraph text block or a table-type text block corresponding to the specification parameter table. These parameters are integrated as the spatial location features of the text block.

[0046] Step 4.3 involves vector concatenation and fusion of text content features and spatial location features to obtain a comprehensive feature vector for each text block. Specifically, for each text block, the extracted text content features and spatial location features are concatenated in vector form to ensure seamless fusion of the two features in the vector dimension, forming a comprehensive feature vector containing dual information. The comprehensive feature vector retains the semantic core information of the text block, reflecting its content attributes, such as whether it is a contract title or specific clause text, and also includes the spatial distribution information of the text block, reflecting its positional relationship and morphological features within the contract. Through this feature fusion method, the comprehensive feature vector of each text block possesses comprehensive and rich information dimensions, providing solid data support for capturing the semantic and spatial relationships between text blocks through the Transformer encoder. This effectively solves the problem that traditional OCR cannot accurately identify the logical structure of text by relying on a single feature, laying a key foundation for the final structured reorganization of procurement contract text.

[0047] Text content features are extracted using pre-trained models such as BERT or LayoutLM to extract text content embeddings. ,in Text content embedding vector, For the i-th text block, BERT is a pre-trained language model with spatial location features, calculating the coordinates of the text block's center point and its size features; , ,in, The x-coordinate of the center point of the i-th text block Let be the x-coordinate of the top-left corner of the i-th text block, w be the width of the i-th text block, and cyi be the y-coordinate of the center point of the i-th text block. is the y-coordinate of the top-left corner of the i-th text block, and h is the height of the i-th text block. Spatial location feature vector, combined feature vector, concatenates text content features with spatial location features; ,in, It is the comprehensive feature vector of the i-th text block. Feature splicing operation It is the text content embedding vector corresponding to the i-th text block. It is the spatial location feature vector corresponding to the i-th text block.

[0048] In this embodiment of the invention, because the text block position information based on OCR output is spatially sorted according to preset rules, and the ordered text blocks are extracted in parallel through a pre-trained language model as text content features, and spatial position features are calculated based on position coordinates, and then the two types of features are concatenated and fused to obtain a comprehensive feature vector, the technical means effectively overcome the technical problems of traditional OCR technology, which can only output fragmented text blocks in visual order, has single feature extraction and cannot simultaneously take into account the semantic and spatial position associations of the text, thus making it difficult to support subsequent logical structure recognition and structured reorganization. As a result, the text blocks form an ordered sequence that conforms to the spatial distribution rules, and the comprehensive feature vector of each text block contains both semantic information and spatial attributes. This provides comprehensive and reliable data support for capturing the global association between text blocks through Transformer encoding and realizing logical role classification, thus laying a solid foundation for the generation of structured corpora.

[0049] In a preferred embodiment of the present invention, step 5 above may include: Step 5.1 involves arranging the vectors in the comprehensive feature vector sequence according to their original order within the sequence to form the input sequence for the Transformer encoder. Specifically, this includes collecting the comprehensive feature vectors corresponding to all procurement contract text blocks generated in Step 4. These vectors each carry semantic content and spatial location information for their respective text blocks, forming the core data foundation for subsequent encoding processing. The arrangement logic determined by the ordered text block sequence is strictly followed. This logic perfectly aligns with the standard layout of procurement contracts and the natural order in which people read contracts. For example, the vector corresponding to the contract title is presented first, followed by the vectors for clause titles, clause body, specification parameter tables, appendices, and other text blocks in sequence. The order of any comprehensive feature vector is not arbitrarily adjusted. All comprehensive feature vectors are sequentially connected according to this fixed order to construct an input sequence that the Transformer encoder can directly receive. This ensures that the sequence structurally preserves the inherent relational order of the procurement contract text blocks, providing ordered data support for the encoder to accurately capture the logical relationships between texts. The Transformer encoder input serialization uses the following spatial sorting rule: sorted by cy coordinate as the primary order and cx coordinate as the secondary order. ,in, This represents a set of text blocks sorted according to spatial rules. It's a sorting operation. It is the original, unsorted collection of text blocks. This is the basis for sorting, with cy as the primary order and cx as the secondary order. It is the y-coordinate of the center point of the text block. It is the x-coordinate of the center point of the text block, and the sequence is constructed; the feature vectors of the sorted text blocks are concatenated into a sequence. ,in, It is the input sequence of the Transformer encoder. The comprehensive feature vector of the sorted text blocks from 1 to N, positional encoding: using sine / cosine positional encoding to preserve order information: Here, PE positional encoding is used to pass sequence order information to the Transformer, pos is the position of the ei text block, and i is the feature dimension. The dimension of the feature vector.

[0050] Step 5.2 involves applying positional encoding to the input sequence to inject spatial order information of text blocks into the vector representation, resulting in an enhanced input sequence that the encoder can perceive positional relationships. This enhanced input sequence is then fed into a pre-trained Transformer encoder. Specifically, this includes performing positional encoding operations tailored to the procurement contract text scenario on the constructed Transformer encoder input sequence. Positional encoding, through specific encoding rules, precisely injects the spatial order information of the text blocks corresponding to each comprehensive feature vector in the procurement contract image into the vector's own representation. For example, it explicitly marks the text block corresponding to a certain vector as being located in the title area at the top of the contract's first page, the clause paragraph area in the middle of the contract, or the appendix description area at the end of the contract. Through this operation, vectors that originally only contained semantic and basic spatial information are endowed with clearer positional association attributes, forming an enhanced input sequence that the Transformer encoder can directly perceive the positional relationships between text blocks. Subsequently, the enhanced input sequence is completely fed into the pre-trained Transformer encoder, which has undergone specialized training on a large number of legal procurement contract text samples. It has fully learned and mastered the semantic logic rules, clause association patterns, and special expression habits of the contract text, and can efficiently adapt to the procurement contract processing needs in the current field scenario.

[0051] Step 5.3: The enhanced input sequence is modeled globally and transformed using the multi-layer self-attention mechanism and feedforward network of the Transformer encoder. The result is an encoded feature sequence that integrates the global semantic and structural information of the document. Specifically, after receiving the enhanced input sequence, the Transformer encoder activates its internal multi-layer self-attention mechanism and feedforward network to perform deep global context modeling and feature transformation on the sequence. The multi-layer self-attention mechanism analyzes each comprehensive feature vector in the sequence one by one, focusing on capturing the correlation between the vector and all other vectors in the sequence. For example, it identifies the subordinate relationship between the vector corresponding to the title of a clause and the vectors of multiple subsequent clauses, the corresponding relationship between the vector of the title of a specification parameter table and the vectors of each parameter entry in the table, and the logical connection relationship between different clauses. This breaks the limitation of isolated and unrelated text blocks in traditional OCR technology. Under the action of the feedforward network, the encoder performs further nonlinear transformation and dimensional optimization on the features processed by the self-attention mechanism, strengthening the key information related to the structure of the contract and weakening irrelevant interference information. For example, it highlights the logical role attribute features of the text block and suppresses the interference caused by irrelevant format symbols in the text block. The encoder outputs an encoded feature sequence that integrates the global semantic information and overall structural information of the procurement contract. Each feature vector in this sequence contains its own core information and carries the correlation information with other text blocks, providing comprehensive and in-depth feature support for subsequent accurate identification of the logical role of text blocks.

[0052] In this embodiment of the invention, this step effectively overcomes the technical problem in traditional techniques that make it difficult to capture the global semantic and spatial structural relationships between text blocks, leading to the inability to accurately identify the logical hierarchy of text. This is achieved by using the multi-layer self-attention mechanism of the Transformer encoder and the feedforward network to construct the input sequence of the Transformer encoder in its original order, applying positional encoding to the input sequence to inject spatial order information to obtain an enhanced input sequence, and then using the multi-layer self-attention mechanism of the Transformer encoder and the feedforward network to perform global context modeling and feature transformation.

[0053] In a preferred embodiment of the present invention, step 6 above may include: Step 6.1 involves feeding the encoded feature sequence as input into a pre-trained classification head model. Specifically, this includes: inputting the complete encoded feature sequence into the pre-trained classification head model. This model has undergone specialized training on a large number of procurement contract text samples. The training data covers various logical role texts from different industries and types of procurement contracts, enabling it to accurately adapt to the feature recognition needs in procurement contract scenarios. During input, the original order of the encoded feature sequence is maintained to ensure that the text block information corresponding to each feature vector is not misaligned. This allows the classification head model to perform accurate logical role classification based on the feature sequence that integrates global semantic and structural information, laying the foundation for recognizing the logical attributes of text blocks. The classification head design uses a two-layer fully connected network for text role classification. , This is the probability distribution of the predicted role for the i-th text block. It is an activation function that transforms the output into probabilities for each character. It is a linear rectified activation function, which introduces a nonlinear characteristic transformation. The comprehensive feature vector of the i-th text block, It is a positional encoding, W1 and b1 are the parameters of the first layer fully connected layer, and W2 and b2 are the parameters of the second layer fully connected layer.

[0054] Step 6.2 involves processing each feature vector in the encoded feature sequence using a classification head model to predict and obtain the logical role category corresponding to each text block, thus obtaining the logical role classification result for each text block. Specifically, after receiving the encoded feature sequence, the classification head model performs in-depth processing on each feature vector in the sequence. The model combines the semantic rules and structural characteristics of the procurement contract text to analyze the semantic information, spatial association information, and association information with other text blocks carried by each feature vector, thereby accurately predicting the logical role category of the text block corresponding to each feature vector. The predicted logical role categories include contract title, clause title, clause body, specification parameter table, amount description, attachment description, header, footer, etc., comprehensively covering common text logic types in procurement contracts. Finally, the logical role classification result corresponding to each text block is output, clearly defining the functional attributes of each text block in the contract, and completely solving the problem that traditional OCR technology cannot distinguish the logical structure of contract text.

[0055] Step 6.3: Based on the logical role classification results and the actual content of the text blocks in the ordered text block sequence, adjacent text blocks with the same logical role category are merged and semantically reorganized to obtain logically coherent text fragments. Specifically, this includes: based on the logical role classification results and the actual content of the text blocks in the ordered text block sequence, text block merging and semantic reorganization operations are performed. For adjacent text blocks with the same logical role category in the ordered text block sequence, the correlation of their actual content is checked first to ensure that text blocks belonging to the same logical unit are merged. For example, consecutive clause text blocks are merged, and text blocks in the same table area of ​​the specification parameter table are merged to avoid erroneously merging text blocks with the same role in different logical units. During the merging process, the content is spliced ​​according to the original order of the text blocks, and semantic calibration is performed to eliminate redundant information and expression gaps between text blocks, ensuring that the merged text fragments are semantically coherent and complete, resulting in independent logical unit text fragments, thus solving the problem of fragmented and unlogically related text blocks in traditional OCR output.

[0056] Step 6.4 involves organizing, encapsulating, and formatting the text fragments according to their corresponding logical roles, ultimately resulting in an electronic document with a clear hierarchical structure that meets the format requirements of AI corpora. Specifically, this includes: systematically organizing the text fragments of each logical unit according to the conventional logical hierarchy of a procurement contract. The organization order follows the habits of people reading and using procurement contracts, first arranging the contract title text fragment, then sequentially arranging the clause titles, clause texts, specification parameter tables, amount descriptions, attachment descriptions, and other text fragments. Auxiliary information such as headers and footers are arranged reasonably according to their actual positions in the contract. The organized text fragments are then encapsulated using a structured format that meets the format requirements of AI corpora, ensuring that the logical role attributes of the text fragments are clear and the hierarchical relationships are explicit. Finally, formatting is performed to standardize the text layout, ensuring that the output electronic document has a clear hierarchical structure and good readability. No manual proofreading or reorganization is required; the entire process is completed locally on a smartphone without any data uploads. This meets the real-time requirements of field scenarios while ensuring the security of sensitive data such as customer pricing strategies and contact information, reducing the time and labor costs of AI corpus production.

[0057] In this embodiment of the invention, by employing a technique of inputting the encoded feature sequence into a pre-trained classification head model, predicting the logical role category of each text block through the classification head model, merging adjacent text blocks with the same logical role based on the classification results and the actual content of the text block for semantic recombination, and then organizing, encapsulating, and formatting the text fragments according to logical roles, the invention effectively overcomes the technical problems of traditional OCR technology, which cannot distinguish the logical role categories of contract text blocks and lacks a clear hierarchical structure in the output results. This results in the need for a large amount of manual proofreading and recombination to form a structured document that conforms to the AI ​​corpus format, thereby increasing time and labor costs. As a result, the invention achieves the technical effect of automatically and accurately identifying the logical roles of text blocks in procurement contracts, quickly generating logically coherent, hierarchically clear electronic documents that meet the requirements of the AI ​​corpus format, completely eliminating the manual sorting process, improving the efficiency of AI corpus production, and meeting the real-time requirements of field scenarios.

[0058] like Figure 2 As shown, embodiments of the present invention also provide a multi-OCR model reordering system for AI corpus processing, including: The acquisition module is used to construct multiple lightweight OCR sub-models. The lightweight OCR sub-models are obtained by decomposing the large OCR model through knowledge distillation and structured pruning techniques, resulting in a model family with different computational complexity and accuracy. The evaluation module is used to evaluate the processing capabilities of edge devices based on the model family. By collecting device hardware parameters and performing benchmark tests, it establishes a device performance prediction model to predict the inference time of different OCR sub-models on the device. The execution module is used to dynamically select an OCR model combination that meets a preset inference time threshold based on the device performance prediction model and the real-time resource status of the device; and to use the OCR model combination to perform OCR processing on the input document image to obtain the text content and location information corresponding to multiple text blocks. The extraction module is used to spatially sort text blocks based on location information to obtain an ordered sequence of text blocks; for each text block in the ordered sequence, text content features and spatial location features are extracted respectively, and the text content features and spatial location features are concatenated to obtain a comprehensive feature vector for each text block; The arrangement module is used to arrange the comprehensive feature vector of each text block in the order of the ordered text block sequence, and input it into the Transformer encoder for encoding to obtain the encoded feature sequence; The merging module is used to classify the logical roles of each text block based on the feature sequence and the classification head, and obtain the logical role classification result of each text block. Based on the logical role classification result and the ordered text block sequence, the text blocks with the same logical role are merged and recombined to obtain a structured document.

[0059] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-OCR model reordering method for AI corpus processing, characterized in that, The method includes: Multiple lightweight OCR sub-models are constructed. These lightweight OCR sub-models are decomposed from the large OCR model through knowledge distillation and structured pruning techniques, resulting in a model family with different computational complexity and accuracy. Based on the model spectrum, the processing capabilities of edge devices are evaluated. By collecting device hardware parameters and performing benchmark tests, a device performance prediction model is established to predict the inference time of different OCR sub-models on the device. Based on the device performance prediction model and the real-time resource status of the device, the combination of OCR models that meets the preset inference time threshold is dynamically selected; the combination of OCR models is used to perform OCR processing on the input document image to obtain the text content and location information corresponding to multiple text blocks. Based on location information, text blocks are spatially sorted to obtain an ordered sequence of text blocks. For each text block in the ordered sequence, text content features and spatial location features are extracted, and the text content features and spatial location features are concatenated to obtain a comprehensive feature vector for each text block. The comprehensive feature vector of each text block is arranged in the order of the ordered text block sequence and input into the Transformer encoder for encoding to obtain the encoded feature sequence. Based on the feature sequence, each text block is logically classified by the classification head to obtain the logical role classification result of each text block; based on the logical role classification result and the ordered text block sequence, text blocks with the same logical role are merged and recombined to obtain a structured document.

2. The multi-OCR model rearrangement method for AI corpus processing according to claim 1, characterized in that, Multiple lightweight OCR sub-models were constructed. These sub-models were decomposed from the large OCR model using knowledge distillation and structured pruning techniques, resulting in a model family with different computational complexities and accuracies, including: Select a high-precision teacher OCR model that includes a text detection module and a text recognition module; Based on the teacher OCR model, multiple initial architectures of student models with different architectural complexities were constructed by performing channel reduction and network depth reduction operations. The initial architecture of the student model is used as the training object. Through knowledge distillation technology, the output of the teacher's OCR model is used as the soft supervision signal. Combined with real labeled data, joint optimization training is carried out to obtain a set of preliminary lightweight student models that retain some knowledge features of the teacher model. Using the initial lightweight student model as the processing object, structured pruning operation is applied. The pruning ratio of each network layer is determined according to the resource constraints of the target edge device. Channels or weights that are judged to be less important are removed to obtain a sparser pruned model. Supervised fine-tuning training is performed on the pruned model to compensate for and restore the recognition accuracy lost during the pruning process. This results in a series of lightweight OCR sub-models that achieve different balances between computational complexity, memory usage, and recognition accuracy, thus creating a model family with different computational complexity and accuracy.

3. The multi-OCR model rearrangement method for AI corpus processing according to claim 2, characterized in that, Based on the model hierarchy, the processing capabilities of edge devices are evaluated. By collecting device hardware parameters and performing benchmark tests, a device performance prediction model is established to predict the inference time of different OCR sub-models on the device, including: Collect hardware configuration parameters of the target edge device to obtain key indicators including the number of CPU cores, memory capacity and GPU performance; based on the hardware configuration parameters, select at least three benchmark test models with different complexities from the lightweight OCR sub-model family; Run the benchmark model sequentially on the target edge device and record the inference time of the benchmark model when processing standard test images; Based on the inference time and the computational complexity of the corresponding benchmark model, a linear regression model is established to describe the relationship between the device's computing power and the inference time of the benchmark model. The prediction accuracy of the linear regression model was verified. When the prediction error was determined to be within a preset threshold, it was used as the equipment performance prediction model for inference time estimation of the OCR sub-model combination.

4. The multi-OCR model rearrangement method for AI corpus processing according to claim 3, characterized in that, Based on the equipment performance prediction model and the real-time resource status of the equipment, dynamically select the combination of OCR models that meet the preset inference time threshold; Using a combination of OCR models to perform OCR processing on the input document image, the text content and location information corresponding to multiple text blocks are obtained, including: Obtain the real-time available memory of the target edge device, set the maximum allowed inference time threshold, and construct a combination of OCR models to be selected; Based on the OCR model combinations to be selected, the first round of screening is performed on the pre-built lightweight OCR sub-model lineage, retaining all OCR model combinations whose total memory requirement does not exceed the real-time available memory, thus obtaining the first candidate model set; For each OCR model combination in the first candidate model set, the established device performance prediction model is invoked to predict the theoretical inference time, and a prediction time label is attached to each candidate model combination. Based on the maximum allowed inference time threshold, a second round of screening is performed on the first candidate model set with attached prediction time labels to remove OCR model combinations whose prediction time exceeds the threshold, thus obtaining the second candidate model set. From the second set of candidate models, an OCR model combination is selected as the final OCR model combination for the current document image processing task, based on the predetermined optimization objective. The selected final OCR model combination is deployed to the operating environment of the target edge device to drive optical character recognition processing on the input document image, thereby obtaining the coordinate position information of the content information of all text blocks in the document image.

5. The multi-OCR model rearrangement method for AI corpus processing according to claim 4, characterized in that, Based on location information, text blocks are spatially sorted to obtain an ordered sequence of text blocks. For each text block in the ordered sequence, text content features and spatial location features are extracted, and the text content features and spatial location features are concatenated to obtain a comprehensive feature vector for each text block, including: Based on the positional information of each text block output by OCR processing, spatial sorting is performed on all text blocks under preset rules to obtain an ordered sequence of text blocks that reflects their spatial distribution order. For each text block in the ordered text block sequence, two parallel feature extraction operations are performed: the semantic embedding of the text content is extracted as the text content feature through a pre-trained language model, and the spatial location feature representing its spatial attributes is calculated based on the location coordinates. Text content features and spatial location features are concatenated and fused to obtain a comprehensive feature vector for each text block.

6. The multi-OCR model rearrangement method for AI corpus processing according to claim 5, characterized in that, The comprehensive feature vector of each text block is arranged in the order of the ordered text block sequence and input into the Transformer encoder for encoding, resulting in the encoded feature sequence, including: The vectors in the comprehensive feature vector sequence are arranged according to their original order in the sequence to form the input sequence of the Transformer encoder. Positional encoding is applied to the input sequence to inject spatial order information of text blocks into the vector representation, resulting in an enhanced input sequence that the encoder is aware of positional relationships; the enhanced input sequence is then fed into a pre-trained Transformer encoder. By using a multi-layer self-attention mechanism and a feedforward network of the Transformer encoder, global context modeling and feature transformation are performed on the enhanced input sequence, resulting in an encoded feature sequence that integrates global semantic and structural information of the document.

7. The multi-OCR model rearrangement method for AI corpus processing according to claim 6, characterized in that, Based on the feature sequence, each text block is logically classified by the classification head to obtain the logical role classification result of each text block; Based on the logical role classification results and the ordered text block sequence, text blocks with the same logical role are merged and recombined to obtain a structured document, including: The encoded feature sequence is used as input and fed into a pre-trained classification head model. The classification head model processes each feature vector in the encoded feature sequence to predict and obtain the logical role category corresponding to each text block, thus obtaining the logical role classification result for each text block. Based on the logical role classification results and combined with the actual content of the text blocks in the ordered text block sequence, adjacent text blocks with the same logical role category are merged and semantically reorganized to obtain logically coherent text fragments. Text fragments are organized, packaged, and formatted according to their corresponding logical roles, ultimately resulting in an electronic document with a clear hierarchical structure that meets the format requirements of artificial intelligence corpora.

8. A multi-OCR model reordering system for AI corpus processing, the system implementing the method as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to construct multiple lightweight OCR sub-models. The lightweight OCR sub-models are obtained by decomposing the large OCR model through knowledge distillation and structured pruning techniques, resulting in a model family with different computational complexity and accuracy. The evaluation module is used to evaluate the processing capabilities of edge devices based on the model family. By collecting device hardware parameters and performing benchmark tests, it establishes a device performance prediction model to predict the inference time of different OCR sub-models on the device. The execution module is used to dynamically select an OCR model combination that meets a preset inference time threshold based on the device performance prediction model and the real-time resource status of the device; and to use the OCR model combination to perform OCR processing on the input document image to obtain the text content and location information corresponding to multiple text blocks. The extraction module is used to spatially sort text blocks based on location information to obtain an ordered sequence of text blocks; for each text block in the ordered sequence, text content features and spatial location features are extracted respectively, and the text content features and spatial location features are concatenated to obtain a comprehensive feature vector for each text block; The arrangement module is used to arrange the comprehensive feature vector of each text block in the order of the ordered text block sequence, and input it into the Transformer encoder for encoding to obtain the encoded feature sequence; The merging module is used to classify the logical roles of each text block based on the feature sequence and the classification head, so as to obtain the logical role classification result of each text block; Based on the logical role classification results and the ordered text block sequence, text blocks with the same logical role are merged and recombined to obtain a structured document.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images