An incremental optimization method of a historical pest archive automatic identification model, a storage medium, a device, and equipment

Abstract

This invention provides an incremental optimization method, apparatus, equipment, and storage medium for an automated identification model of historical pest and disease records. A closed-loop incremental learning mechanism is constructed to automatically collect corrected samples reviewed by experts to build a training set. The identification model is fine-tuned online using daily newly added high-value samples, achieving lightweight and rapid updates. The model is then retrained offline using monthly aggregated new data, achieving comprehensive retraining. Specifically, daily high-value samples are filtered for long-tail and conflicting samples through a multi-dimensional weighted scoring model and obtained through domain-adaptive enhancement. Online fine-tuning is achieved by injecting a low-rank adapter and elastic weights to reinforce regularization. Offline retraining includes data deduplication, stratified sampling, hard example mining, and optimization of the multi-task joint loss function. This invention addresses the resource waste caused by the need for full retraining when adding new data in existing models for historical pest and disease records, and also provides a dedicated offline and online training method for this field.

Description

Technical Field

[0001] This invention relates to the technical field of image recognition and machine learning, specifically to an incremental optimization method, storage medium, device, and equipment for an automated identification model of historical pest and disease archives. Background Technology

[0002] In the field of automated archival recognition, existing technologies mainly rely on static deep learning models. Static models are typically trained on a fixed training set and then deployed online, lacking the ability to adapt to dynamic changes in data distribution. As the writing styles of historical handwritten archives evolve, with styles changing from traditional to simplified characters, differences in handwriting across different periods, and image degradation issues such as yellowing of paper and fading of ink due to the passage of time, the recognition accuracy of the model will significantly decrease when encountering new batches of data.

[0003] To address this issue, the traditional approach is to periodically collect new samples manually and retrain the model. However, this process has significant shortcomings: First, sample collection relies on manual annotation, which is costly, time-consuming, and cannot respond promptly to changes in online data. Second, retraining often involves training from scratch with the entire dataset, which consumes a lot of computational resources and is prone to forgetting historical knowledge, leading to performance degradation of the model on older data. Furthermore, existing systems lack an automatic detection mechanism for model performance degradation and usually rely on manual monitoring or periodic evaluation, which cannot intervene in time when performance declines, resulting in long-term inefficiency of online services.

[0004] Some solutions attempt to use online learning, but simply adding new samples to the training set easily introduces noise, and the lack of screening for high-value samples leads to poor fine-tuning results or even catastrophic forgetting. Therefore, existing technologies cannot achieve low-cost, high-frequency, safe, and controllable continuous model evolution, making it difficult to meet the needs of automated document recognition. Summary of the Invention

[0005] Based on this, in order to solve the technical problem that existing historical pest and disease archive recognition models need to be fully retrained when new data is added, which leads to a waste of computing resources and makes it impossible to efficiently utilize incremental knowledge for continuous model optimization, an incremental optimization method, storage medium, device and equipment for an automated historical pest and disease archive recognition model is proposed.

[0006] This invention protects an incremental optimization method for an automated identification model of historical pest and disease records, comprising: constructing a closed-loop incremental learning mechanism, automatically collecting corrected samples after expert review, and constructing a training set based on the corrected samples; using the daily newly added high-value samples in the training set to fine-tune the identification model online, achieving lightweight and rapid model updates; and using the monthly summary of all new data in the training set to retrain the identification model offline, achieving comprehensive model retraining.

[0007] Furthermore, the acquisition of newly added high-value samples daily includes: periodically triggering logs to obtain the full OCR recognition pipeline within the first time period from the distributed log storage; filtering log samples using a multi-dimensional weighted scoring model to retain long-tail and conflicting samples; cropping text regions from the original images based on text region coordinates and aligning them with the labeled sequence; and performing domain-adaptive enhancements on the training set, including adding Gaussian noise, contrast stretching, micro-rotation, and simulating paper yellowing.

[0008] Furthermore, the recognition model is fine-tuned online, including: injecting low-rank adapters into the self-attention layer, cross-attention layer and feedforward network layer of the OCR recognition head respectively; retaining historical high-confidence samples through an empirical replay buffer; and using elastic weights to consolidate the offset of the new weights by regularization constraints.

[0009] Furthermore, online fine-tuning of the recognition model includes: reserving 20% ​​of the daily samples and a fixed monthly baseline set as a validation set; evaluating character error rate, domain entity recall rate, and inference latency stability rate, and setting acceptable thresholds; if all indicators meet the standards, the deployment process begins, and if any indicator exceeds the standard, the pipeline is terminated, and the sample is marked as awaiting manual review.

[0010] Furthermore, using the monthly aggregated new data in the training set, the recognition model is retrained offline to achieve comprehensive retraining. This includes: triggering monthly by the distributed scheduling center to obtain the full business flow within the second time period, including daily online fine-tuning samples, manual review and correction records, and historical original training sets; converting all data into a unified first format and generating a full dataset fingerprint; performing deduplication and cleaning of images and text editing distance verification, and stratifying sampling by crop type, disease category, document age, and writing type to control the deviation of the sample ratio of each layer from the online real distribution to be less than the first ratio; calculating the cross-labeling coefficient, and triggering a two-person arbitration process if it is less than the second ratio; mining difficult cases from samples that have continuously failed to correct errors in the semantic correction module, samples with low recognition confidence but correct manual review, and archive fragments with low entity recall rate in the domain knowledge graph; and classifying and expanding common typical scenarios such as similar-looking characters confusion, continuous writing of professional terms, and variations in historical handwriting.

[0011] Furthermore, by utilizing the monthly aggregated new data in the training set, the recognition model is retrained offline to achieve comprehensive retraining. This also includes: using a lightweight multimodal feature extractor as a shared backbone network, designing three task heads: a detection head, a recognition head, and a semantic alignment branch; constructing a joint loss function, which includes detection loss, recognition loss, contrast loss, and spatial consistency regularization loss, and retraining based on the joint loss function.

[0012] Furthermore, the recognition model is retrained offline using the monthly aggregated new data in the training set to achieve comprehensive retraining. This also includes: evaluation using a fixed monthly benchmark set and a historical version comparison set, with core indicators including a character error rate less than the third proportion, a domain entity recall rate greater than the fourth proportion, and an average detection precision greater than the fifth proportion; dynamic routing based on file risk level and clarity, implementing a phased gray-scale switching strategy, and setting circuit breaker and rollback conditions; and extracting newly added high-confidence entities from the retrained model logs, which are then injected into the domain dictionary and error correction path of the semantic correction module after rule verification.

[0013] This invention protects an incremental optimization device for an automated identification model of historical pest and disease records, comprising: a closed-loop incremental learning module for automatically collecting corrected samples after expert review and constructing a training set based on the corrected samples; an online fine-tuning module for fine-tuning the identification model online using daily newly added high-value samples in the training set; and an offline retraining module for retraining the identification model offline using monthly aggregated new data in the training set.

[0014] This invention protects an electronic device, comprising: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. The machine-readable instructions are executed by the processor to perform the incremental optimization method of an automated identification model for historical pest and disease records.

[0015] This invention protects a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of an incremental optimization method for an automated identification model of historical pest and disease records.

[0016] This invention protects an incremental optimization method for an automated identification model of historical pest and disease archives. It constructs a closed-loop incremental learning mechanism, automatically collecting corrected samples reviewed by experts and building a training set. This enables continuous adaptive optimization of the model based on historical pest and disease archives, improving the timeliness of the identification model in this vertical field over long-term use. Furthermore, it utilizes daily new high-value samples from the training set to fine-tune the identification model online, achieving lightweight and rapid updates. This allows for timely responses to newly emerging pest and disease features with minimal computational overhead, ensuring the model's timeliness and accuracy in real-time applications while reducing the resource consumption of frequent full-scale training. Further, it uses monthly aggregated new data from the training set to perform offline retraining of the identification model, achieving comprehensive model retraining. Through periodic large-scale data integration, it corrects potential biases or overfitting issues accumulated during long-term incremental updates, thereby maintaining the model's stability and generalization ability globally. Ultimately, this forms an automated identification system for historical pest and disease archives that simultaneously satisfies both efficiency and accuracy. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This application provides a flowchart of a model update method based on incremental learning.

[0019] Figure 2 This application provides a flowchart of an online fine-tuning method for a recognition model.

[0020] Figure 3 This application provides a flowchart of an offline retraining method for a recognition model.

[0021] Figure 4 : A diagram of the model update device provided in the embodiments of this application.

[0022] Figure 5 : A diagram of the model update electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0024] Research has revealed that existing automated identification models for historical pest and disease archives face challenges in adapting to the realities of historical archive recognition. Due to the vast span of time in these archives—some covering decades—and the evolving writing styles (from early handwriting to later printed text) and variations in handwriting such as cursive and fading, the model's recognition performance gradually declines after initial training as new archives are continuously added. This is particularly evident when handling newly emerging handwriting, abbreviations of technical terms, or misaligned tables—long-tailed samples where the error rate increases significantly. Traditional one-time training or periodic full retraining methods cannot adapt to this continuous change in data distribution. This results in the model failing to effectively utilize corrective samples generated after expert review, leading to knowledge waste. Furthermore, the long update cycles and high costs make it difficult to meet the business requirements for real-time model evolution.

[0025] Based on this, please refer to Figure 1 This application provides an incremental optimization method for an automated identification model of historical pest and disease records, comprising:

[0026] S101 Constructs a closed-loop incremental learning mechanism to automatically collect corrected samples after expert review and construct a training set based on the corrected samples;

[0027] S102 uses the daily newly added high-value samples in the training set to fine-tune the recognition model online, achieving rapid updates of the lightweight model.

[0028] S103 Using the monthly aggregated new data in the training set, the recognition model is retrained offline to achieve comprehensive model retraining.

[0029] This application provides an incremental optimization method for an automated identification model of historical pest and disease archives. By constructing a closed-loop incremental learning mechanism, it can automatically collect and utilize corrected samples reviewed by experts. Through daily online fine-tuning, high-value samples are used to achieve lightweight and rapid updates, enabling the model to quickly adapt to new writing styles, professional terminology, and other long-tail distributions, achieving daily model evolution and improving the identification accuracy of new archives. Simultaneously, monthly offline retraining, using all new data, comprehensively optimizes the model, ensuring its stability in long-term operation. This mechanism realizes an automated closed loop from sample collection and model updates to performance improvement, significantly reducing the cost of manual intervention and thus ensuring the timeliness and accuracy of model updates.

[0030] In step S101, a closed-loop incremental learning mechanism is constructed to automatically collect corrected samples after expert review and to construct a training set based on the corrected samples.

[0031] In one optional implementation, to address the issue of decreased model recognition accuracy caused by the large time span and continuous evolution of writing styles in historical pest and disease archives, the system constructs a closed-loop incremental learning mechanism. Through a sample feedback pipeline, expert-reviewed corrected samples are automatically exported in a standard annotation format. In another optional implementation, the samples are exported in ICDAR format with attached error type labels, which can include terminology misidentification, broken strokes, or table misalignment, and then stored in the incremental training library. These corrected samples form the basis for subsequent model optimization; the system constructs a training set based on these corrected samples for subsequent online fine-tuning and offline retraining.

[0032] In a specific example, when an expert review finds that rust is misidentified as a causative disease in a handwritten survey form from the 1990s, the corrected sample is automatically collected and marked as a terminology misidentification, thus providing targeted training data for the model. Through this closed-loop mechanism, the system can continuously learn from expert feedback and automatically accumulate high-quality training data.

[0033] In step S102, the recognition model is fine-tuned online using the daily newly added high-value samples in the training set, achieving lightweight and rapid updates. Specifically, after constructing a training set containing expert-reviewed and corrected samples, the system automatically selects newly added high-value samples daily using online fine-tuning technology.

[0034] In one alternative implementation, newly emerging drug names or rare handwriting styles trigger lightweight fine-tuning of the PaddleOCR recognition head, updating model weights to achieve daily model evolution. This process ensures the model can quickly adapt to the continuous evolution of handwriting styles, maintaining a stable improvement in recognition performance at low cost.

[0035] For the online fine-tuning process, please refer to the appendix. Figure 2 .

[0036] In step S1021, the acquisition of newly added high-value samples daily includes multiple steps such as timed log triggering, automatic screening of high-value samples, data preprocessing, and lightweight enhancement. Specifically, the system uses a complete process to automatically select the most valuable samples for model improvement from the daily OCR recognition pipeline, providing high-quality training data for subsequent online fine-tuning.

[0037] In step S10211, the log is triggered periodically to retrieve the full OCR recognition flow data for the first time period from the distributed log storage.

[0038] In one optional implementation, the system triggers a fine-tuning channel at 02:00:00 daily by a scheduled task, retrieving the full OCR recognition pipeline for the previous 24 hours from a distributed log storage. This pipeline data includes information such as image ID, text region coordinates, initial recognition sequence, CTC / Attention confidence distribution, semantic correction module output sequence, error correction confidence, and manual review markings.

[0039] Next, in step S10212, a multidimensional weighted scoring model is used to filter the log samples, retaining long-tail samples and conflict samples.

[0040] In one optional implementation, a multidimensional weighted scoring model is used to filter log samples, retaining the optimal long-tail and conflicting samples. The scoring formula comprehensively considers identification confidence, semantic conflict indicators, and domain novelty indicators. Weighting is configured to prioritize semantic conflicts and long-tail novel samples. Filtering rules include excluding samples with incompleteness rates above a threshold, confidence levels below a threshold, and samples from non-text areas.

[0041] In one optional approach, the number of samples retained per day is controlled between 500 and 2000, and stratified sampling is performed according to a 7:3 ratio of printed to handwritten text, ultimately outputting a structured training set.

[0042] Next, in step S10213, the text region is cropped from the original image according to the text region coordinates and aligned with the annotation sequence.

[0043] In one alternative implementation, text regions are cropped from the original archival image and dynamically time-warped and aligned with the annotation sequence to ensure an accurate correspondence between the image regions and the text annotations, providing accurate input data for subsequent model training.

[0044] In step S10214, the training set is subjected to domain-adaptive enhancement, including adding Gaussian noise, contrast stretching, micro-rotation, and simulating paper yellowing.

[0045] Optionally, domain-adaptive augmentation can be performed on the training set, affecting only the training set without altering the text content. Augmentation operations include adding Gaussian noise (with a mean of 0 and variance of 0.01 in some scenarios), adjusting contrast stretching to 0.8 to 1.2 times in some scenarios, micro-rotation within the range of -5 to 5 degrees in some scenarios, and simulating paper yellowing using HSV hue shift. The method does not use strong geometric deformations, such as perspective transformations, as these could disrupt the topological structure of handwritten strokes in agricultural archives.

[0046] After acquiring high-value samples, online fine-tuning is performed. In step S1022, the recognition model is fine-tuned online, including injecting low-rank adapters into the self-attention layer, cross-attention layer, and feedforward network layer of the OCR recognition head; retaining historical high-confidence samples through an empirical replay buffer; and using elastic weights to consolidate the offset of the new weights by regularization constraints.

[0047] In one optional implementation, after acquiring and preprocessing high-value samples, the system performs fine-tuning injection and lightweight training to prevent forgetting. The injection sites are targeted at the PaddleOCR recognition head, injecting low-rank adapters into the self-attention layer, cross-attention layer, and feedforward network layer, respectively. The fine-tuning hyperparameters are set to rank r=8, scaling factor α=16, and Dropout=0.05, training only the adapter parameters that account for less than 1% of the total parameters.

[0048] Regarding the anti-forgetting mechanism, an experience replay buffer is used to retain 10% of historical high-confidence samples, and stratified sampling is performed according to disease category. Simultaneously, elastic weighting is used to reinforce regularization, and the Fisher information matrix F is calculated for the base model parameters θ to constrain the deviation of the new weights. The specific formula is as follows: The optimizer is configured as AdamW, with a learning rate of lr=2e-4, weight_decay=0.01, gradient clipping max_norm=1.0, and gradient accumulation steps=2.

[0049] In step S1023, the online fine-tuning of the recognition model also includes reserving the sample ratio of the current day's samples and a fixed monthly benchmark set as a validation set; evaluating the character error rate, domain entity recall rate, and inference latency stability rate, and setting a qualified threshold; if all indicators meet the standards, the deployment process begins; if any indicator exceeds the standard, the pipeline is terminated, and the sample is marked as awaiting manual review.

[0050] In one optional implementation, after lightweight training, the system performs offline evaluation and security verification. The validation set is divided into 20% of the daily reserved samples plus a fixed monthly baseline set, which may contain 500 typical difficult cases from agricultural archives. Evaluation metrics include: character error rate must be less than or equal to 2% or not degraded, domain entity recall rate must be greater than or equal to 90%, and inference latency P95 must be less than or equal to 200ms. The overall decision logic is as follows: if all metrics are met, the deployment process begins; if any of the above metrics are exceeded, the channel is terminated, failure characteristics are recorded, and samples are automatically marked as requiring manual review.

[0051] The above describes the real-time fine-tuning logic for the online lightweight model. Based on this, the system has a large-scale model retraining process. For details on the model retraining process, please refer to [link / reference needed]. Figure 3 .

[0052] In step S103, the recognition model is retrained offline using the monthly aggregated new data in the training set to achieve comprehensive model retraining.

[0053] Specifically, based on daily online fine-tuning to achieve lightweight and rapid updates, the system summarizes all newly added samples every month, performs full model retraining in combination with the original training set, and releases a new version of the model based on this. The version number includes a timestamp and supports canary releases and A / B testing.

[0054] In step S1031, the recognition model is retrained offline using the monthly aggregated new data in the training set to achieve comprehensive retraining. This includes monthly triggering by the distributed scheduling center to obtain the full business flow within the second time period, including daily online fine-tuning samples, manual review and correction records, and historical original training sets; converting all data into a first format and generating a full dataset fingerprint; performing image deduplication and text editing distance verification for deduplication and cleaning; performing stratified sampling according to crop type, disease category, document age, and writing type, controlling the deviation of the sample ratio of each layer from the online real distribution to be less than a first ratio; calculating the cross-labeling coefficient, and triggering a two-person arbitration process if it is less than a second ratio; mining difficult examples from samples that have continuously failed to correct errors in the semantic correction module, samples with low recognition confidence but correct manual review, and archive fragments with low entity recall rate in the domain knowledge graph; and classifying and expanding common typical scenarios such as similar-looking characters confusion, continuous writing of professional terms, and variations in historical handwriting.

[0055] In an alternative implementation, a series of data governance and hard case mining steps are included, specifically in S10131 to S10133.

[0056] In step S10311, the distributed scheduling center triggers monthly to obtain the full business flow data for the second time period, including daily online fine-tuning samples, manual review and correction records, and historical original training sets; all data are uniformly converted into the first format, and a full dataset fingerprint is generated.

[0057] In one optional approach, a distributed scheduling center triggers monthly retrieval of the full business transaction logs for the past 30 days, including daily online fine-tuning samples and inference logs, manual review and correction records, and historical original training sets. All data is uniformly converted to a fixed format, with fields including image_id, etc. The system generates a fingerprint of the entire dataset, such as SHA-256(dataset_hash), and writes it to the version metadata table.

[0058] In step S10312, image deduplication and text editing distance verification are performed. Stratified sampling is carried out according to crop type, disease category, document age and writing type. The deviation between the sample ratio of each layer and the actual online distribution is controlled to be less than the first ratio. The cross-labeling coefficient is calculated. If it is less than the second ratio, the two-person arbitration process is triggered.

[0059] Optionally, the system performs data governance and consistency checks. For deduplication and cleaning, image-level deduplication is used, with Hamming distances less than or equal to 5 considered duplicates, retaining those with higher annotation quality; text-level deduplication uses edit distance checks to remove obvious errors in manual annotation, such as mixed pinyin input and non-agricultural terms.

[0060] In terms of stratified sampling and balancing, the system stratifies samples according to crop type (including rice, wheat, and fruit trees), disease type (including fungi, bacteria, viruses, and insect pests), document age (including less than 5 years, 5 to 15 years, and more than 15 years), and writing type (including printed, handwritten, and mixed writing types), controlling the deviation of the sample proportion of each stratum from the actual online distribution to be less than 5%. The system calculates the cross-labeling coefficient; if it is less than 0.8, a two-person arbitration process is triggered to ensure that the labeling quality meets the training baseline.

[0061] In step S10313, difficult cases are extracted from samples that fail to correct errors continuously by the semantic correction module, samples with low identification confidence but correct after manual verification, and archive fragments with low entity recall rate of the domain knowledge graph; common typical scenarios such as confusion of similar characters, concatenation of professional terms, and variation of historical handwriting are classified and expanded.

[0062] Optionally, the system performs difficult example mining and dynamic training set construction. Difficult examples include: samples that fail to correct errors twice consecutively by the semantic correction module, samples with an identification confidence level of less than 0.5 but correct after manual verification, and document fragments with an entity recall rate of less than 60% in the domain knowledge graph.

[0063] Common typical scenarios are categorized and expanded, such as confusion between similar-looking characters. In a practical example, rust disease is misidentified as a disease caused by other factors; technical terms are written in a connected or abbreviated manner; sentences are broken across pages or tables are misaligned or obscured by seals; and variations in historical handwriting are present, typically manifested as cursive script, cursive writing, or fading. The dataset ratio is 3:7 for difficult examples and 7 for regular examples, with the total size controlled between 100,000 and 200,000 records. Any records exceeding this range are downsampled according to stratified importance.

[0064] After the data governance and hard example mining steps are completed, the training architecture needs to be determined for model retraining.

[0065] In step S1032, a lightweight multimodal feature extractor is used as a shared backbone network, and three task heads are designed: a detection head, a recognition head, and a semantic alignment branch. A joint loss function is constructed, which includes detection loss, recognition loss, contrast loss, and spatial consistency regularization loss. Retraining is performed based on the joint loss function.

[0066] In this embodiment, a detection-recognition-semantic joint training architecture is adopted. Through multi-task joint optimization, the model's ability to recognize domain-specific terms and historical handwriting is improved.

[0067] In step S10321, a lightweight multimodal feature extractor is used as a shared backbone network, and three task heads are designed: a detection head, a recognition head, and a semantic alignment branch.

[0068] In an optional implementation, a lightweight multimodal feature extractor is used to output multi-scale feature maps. The multi-task head design includes: a detection head using DBNet++, outputting binary maps and probability threshold maps of text regions; a recognition head using an SVTR / Transformer architecture, outputting character-level probability sequences; and a lightweight semantic alignment branch that maps the recognition sequence to the domain knowledge graph node space through an embedding layer.

[0069] In step S10322, a joint loss function is constructed, which includes detection loss, recognition loss, contrast loss and spatial consistency regularization loss, and retraining is performed based on the joint loss function.

[0070] Optionally, a joint loss function is constructed, including: detection loss L_det, which is Dice plus BCE loss, used for text region detection; recognition loss L_rec, which is CTC plus Attention cross-entropy joint loss; contrast loss L_cont, based on InfoNCE, to narrow the embedding distance between the OCR output sequence and the corresponding entity node in the knowledge graph; and spatial consistency regularization loss L_spatial, which is the IoU penalty term between the detection box and the recognition sequence.

[0071] In step S1033, a fixed monthly benchmark set and a historical version comparison set are used for evaluation. Key metrics include a character error rate less than the third proportion, a domain entity recall rate greater than the fourth proportion, and an average detection precision greater than the fifth proportion. Dynamic routing is implemented based on file risk level and clarity, employing a phased gray-scale flow-cutting strategy, and circuit breaker and rollback conditions are set. Newly added high-confidence entities are extracted from the retrained model logs, validated according to rules, and injected into the domain dictionary and error correction path of the semantic correction module. Specifically, after completing joint training, the system performs offline evaluation and version packaging, and implements gray-scale release, A / B testing, and knowledge base synchronization.

[0072] In step S10331, a fixed monthly benchmark set and a historical version comparison set are used for evaluation. The core indicators include a character error rate less than the third proportion, a domain entity recall rate greater than the fourth proportion, and an average detection precision greater than the fifth proportion.

[0073] Optionally, the evaluation benchmark set is a fixed monthly benchmark set. In one optional implementation, the fixed monthly benchmark set contains 2000 difficult examples covering the entire scenario, plus a historical version comparison set. The core indicators and admission criteria are: character error rate less than 2%, and significantly lower than the previous stable version; domain entity recall rate greater than 95%; and average detection precision greater than 90%.

[0074] In step S10332, dynamic routing is performed according to the file risk level and resolution, a phased grayscale switching strategy is implemented, and circuit breaker and rollback conditions are set.

[0075] In one optional approach, the grayscale traffic redirection strategy dynamically routes traffic based on file risk level and clarity: Phase 1: 1% of traffic is directed to an internal sandbox or test account containing only clear scans; Phase 2: 5% of traffic is directed to low-risk files, i.e., printed text from the last 5 years with a confidence level greater than 0.8; Phase 3: 20% of traffic is directed to regular files, including mixed types with some handwritten text; Phase 4: 100% full traffic redirection is implemented after a 7-day stabilization period.

[0076] A / B testing monitoring metrics include real-time collected CER rolling average, semantic correction module call frequency, manual review interception rate, P99 latency, and abnormal rollback rate. Circuit breaking and rollback are triggered if any of the following conditions are met within 30 consecutive minutes: CER degradation greater than 2%, semantic correction failure rate increase greater than 5%, or P99 latency increase greater than 10%. In this case, the API gateway will automatically switch back to the previous stable version, and an alert will be pushed to the operations and algorithm teams.

[0077] In step S10333, newly added high-confidence entities are extracted from the retrained model logs, and after rule verification, they are injected into the domain dictionary and error correction path of the semantic correction module.

[0078] Optionally, the full synchronization process of the knowledge base is as follows: extract the top 50 newly added high-confidence entities from the retrained model logs, including new drug names, local disease common names, historical handwritten variants, etc.; after rule verification, inject the domain dictionary, DAG error correction path and knowledge graph nodes of the semantic correction module; update the weight of the correction rules, thereby completing a monthly closed loop of full data, joint optimization, version release and semantic knowledge base upgrade.

[0079] Please see Figure 4 , Figure 4 This is a schematic diagram of the incremental optimization device for an automated identification model of historical pest and disease records provided in an embodiment of this application. Figure 4 As shown, the generating apparatus 400 includes:

[0080] The closed-loop incremental learning module 410 is used to automatically collect the corrected samples after expert review and to construct a training set based on the corrected samples.

[0081] The online fine-tuning module 420 is used to fine-tune the recognition model online using the daily newly added high-value samples in the training set;

[0082] The offline retraining module 430 is used to retrain the recognition model offline using the monthly aggregated full new data in the training set.

[0083] Furthermore, when the online fine-tuning module 420 is used to fine-tune the recognition model online using the daily newly added high-value samples in the training set, the online fine-tuning module 420 is used to:

[0084] Logs are triggered periodically to retrieve the full OCR recognition log for the first time period from the distributed log storage;

[0085] A multidimensional weighted scoring model is used to filter log samples, retaining long-tail and conflicting samples;

[0086] The text region is cropped from the original image based on the text region coordinates and aligned with the annotation sequence;

[0087] The training set is augmented with domain-adaptive enhancements, including adding Gaussian noise, contrast stretching, micro-rotation, and simulating paper yellowing.

[0088] Furthermore, when the online fine-tuning module 420 is used to fine-tune the recognition model online, the online fine-tuning module 420 is used to:

[0089] Low-rank adapters are injected into the self-attention layer, cross-attention layer, and feedforward network layer of the OCR recognition head, respectively; historical high-confidence samples are retained through an empirical replay buffer, and elastic weights are used to consolidate the offset of the new weights by regularization constraints.

[0090] Furthermore, when the online fine-tuning module 420 is used to perform online fine-tuning of the recognition model, the online fine-tuning module 420 is also used for:

[0091] 20% of the daily sample and a fixed monthly baseline set are reserved as validation sets;

[0092] Evaluate character error rate, domain entity recall rate, and inference latency stability rate, and set acceptable thresholds;

[0093] If all indicators meet the criteria, the deployment process begins; if any indicator exceeds the criteria, the pipeline is terminated and the sample is marked for manual review.

[0094] Furthermore, when the offline retraining module 430 uses the monthly aggregated full new data in the training set to perform offline retraining on the recognition model to achieve comprehensive retraining, the offline retraining module 430 is used for:

[0095] Triggered monthly by the distributed scheduling center, the system retrieves the full business transaction log for the second time period, including daily online fine-tuning samples, manual review and correction records, and historical original training sets; all data is then uniformly converted into the first format, and a full dataset fingerprint is generated.

[0096] Deduplication and cleaning are performed at the image level and at the text level, and the editing distance is checked. Stratified sampling is carried out according to crop type, disease category, document age and writing type. The deviation between the sample ratio of each layer and the actual online distribution is controlled to be less than the first ratio. The cross-labeling coefficient is calculated. If it is less than the second ratio, a two-person arbitration process is triggered.

[0097] Difficult cases are extracted from samples that fail to correct errors continuously by the semantic correction module, samples with low identification confidence but correct after manual verification, and archival fragments with low entity recall rates in the domain knowledge graph; common and typical scenarios such as confusion of similar-looking characters, consecutive writing of professional terms, and variations in historical handwriting are classified and expanded.

[0098] Furthermore, when the offline retraining module 430 uses the monthly aggregated full new data in the training set to perform offline retraining on the recognition model to achieve comprehensive retraining, the offline retraining module 430 is also used for:

[0099] A lightweight multimodal feature extractor is used as a shared backbone network, and three task heads are designed: a detection head, a recognition head, and a semantic alignment branch.

[0100] A joint loss function is constructed, which includes detection loss, identification loss, contrast loss, and spatial consistency regularization loss. Retraining is then performed based on the joint loss function.

[0101] Furthermore, when the offline retraining module 430 uses the monthly aggregated full new data in the training set to perform offline retraining on the recognition model to achieve comprehensive retraining, the offline retraining module 430 is also used for:

[0102] Evaluation was conducted using a fixed monthly baseline set and a historical version comparison set. Key metrics included a character error rate less than the third proportion, a domain entity recall rate greater than the fourth proportion, and an average detection precision greater than the fifth proportion.

[0103] Dynamic routing is implemented based on the risk level and clarity of the archives, and a phased gray-scale switching strategy is adopted, with circuit breaker and rollback conditions set.

[0104] Newly added high-confidence entities are extracted from the model logs after retraining, and after rule verification, they are injected into the domain dictionary and error correction path of the semantic correction module.

[0105] The incremental optimization device for the automated identification model of historical pest and disease records provided in this application embodiment automatically collects corrected samples after expert review by constructing a closed-loop incremental learning mechanism, and builds a training set based on the corrected samples. It then uses daily new high-value samples in the training set to fine-tune the identification model online, achieving lightweight and rapid updates. Finally, it uses monthly aggregated new data in the training set to retrain the identification model offline, achieving comprehensive retraining. Thus, by combining an online fine-tuning and offline retraining dual-layer optimization strategy, it can quickly respond to changes in daily new samples and periodically upgrade the model comprehensively, thereby continuously improving the identification accuracy and robustness of historical pest and disease records and reducing the cost of manual review.

[0106] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.

[0107] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 The steps of the incremental optimization method for the automated identification model of historical pest and disease archives in the method embodiment shown are described in the method embodiment, and will not be repeated here.

[0108] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the incremental optimization method for the automated identification model of historical pest and disease archives in the method embodiment shown are described in the method embodiment, and will not be repeated here.

[0109] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An incremental optimization method for an automated identification model of historical pest and disease records, characterized in that, include: Construct a closed-loop incremental learning mechanism to automatically collect corrected samples after expert review, and build a training set based on the corrected samples; By utilizing the daily newly added high-value samples in the training set, the recognition model is fine-tuned online to achieve lightweight and fast model updates. Using the monthly aggregated new data in the training set, the recognition model is retrained offline to achieve comprehensive model retraining.

2. The method according to claim 1, characterized in that, The acquisition of newly added high-value samples daily includes: Logs are triggered periodically to retrieve the full OCR recognition log for the first time period from the distributed log storage; A multidimensional weighted scoring model is used to filter log samples, retaining long-tail and conflicting samples; The text region is cropped from the original image based on the text region coordinates and aligned with the annotation sequence; The training set is subjected to domain-adaptive augmentation, which includes adding Gaussian noise, contrast stretching, micro-rotation, and simulating paper yellowing.

3. The method according to claim 2, characterized in that, The online fine-tuning of the recognition model includes: Low-rank adapters are injected into the self-attention layer, cross-attention layer, and feedforward network layer of the OCR recognition head, respectively; historical high-confidence samples are retained through an empirical replay buffer, and elastic weights are used to consolidate the offset of the new weights by regularization constraints.

4. The method according to claim 3, characterized in that, The online fine-tuning of the recognition model also includes: The sample proportion reserved for the current day and a fixed monthly baseline set are used as the validation set, and the sample proportion is less than 50%. Evaluate character error rate, domain entity recall rate, and inference latency stability rate, and set acceptable thresholds; If all indicators meet the criteria, the deployment process begins; if any indicator exceeds the criteria, the pipeline is terminated and the sample is marked for manual review.

5. The method according to claim 1, characterized in that, The step of using the monthly aggregated full new data in the training set to perform offline retraining of the recognition model, thereby achieving comprehensive model retraining, includes: Triggered monthly by the distributed scheduling center, the system retrieves the full business transaction log for the second time period, including daily online fine-tuning samples, manual review and correction records, and historical original training sets; all data is then uniformly converted into the first format, and a full dataset fingerprint is generated. The process involves image deduplication and text editing distance verification, stratified sampling based on crop type, disease category, document age, and writing type, and ensuring that the deviation between the sample ratio of each stratum and the actual online distribution is less than the first ratio. The cross-labeling coefficient is calculated, and if it is less than the second ratio, a two-person arbitration process is triggered. Difficult cases are extracted from samples that fail to correct errors continuously by the semantic correction module, samples with low identification confidence but correct after manual verification, and archival fragments with low entity recall rates in the domain knowledge graph; common and typical scenarios such as confusion of similar-looking characters, consecutive writing of professional terms, and variations in historical handwriting are classified and expanded.

6. The method according to claim 5, characterized in that, The step of using the monthly aggregated new data in the training set to perform offline retraining of the recognition model, thereby achieving comprehensive model retraining, also includes: A lightweight multimodal feature extractor is used as a shared backbone network, and three task heads are designed: a detection head, a recognition head, and a semantic alignment branch. A joint loss function is constructed, which includes detection loss, identification loss, contrast loss, and spatial consistency regularization loss. Retraining is then performed based on the joint loss function.

7. The method according to claim 6, characterized in that, The step of using the monthly aggregated new data in the training set to perform offline retraining of the recognition model, thereby achieving comprehensive model retraining, also includes: Evaluation was conducted using a fixed monthly baseline set and a historical version comparison set. Key metrics included a character error rate less than the third proportion, a domain entity recall rate greater than the fourth proportion, and an average detection precision greater than the fifth proportion. Dynamic routing is implemented based on the risk level and clarity of the archives, and a phased gray-scale switching strategy is adopted, with circuit breaker and rollback conditions set. Newly added high-confidence entities are extracted from the model logs after retraining, and after rule verification, they are injected into the domain dictionary and error correction path of the semantic correction module.

8. An incremental optimization device for an automated identification model of historical pest and disease records, characterized in that, include: The closed-loop incremental learning module is used to automatically collect the corrected samples after expert review and to construct a training set based on the corrected samples; The online fine-tuning module is used to fine-tune the recognition model online using the daily newly added high-value samples in the training set; The offline retraining module is used to retrain the recognition model offline using the monthly aggregated new data in the training set.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the incremental optimization method for the automated identification model of historical pest and disease archives as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the incremental optimization method for the automated identification model of historical pest and disease archives as described in any one of claims 1 to 7.

Images