A large model-based data intelligent production and model self-evolution system and method

Abstract

The application relates to the field of artificial intelligence and computer vision technology, and discloses a data intelligent production and model self-evolution method based on a large model, which comprises the following steps: acquiring unlabeled original images, performing open-vocabulary zero-sample automatic pre-labeling by a visual teacher large model adapted to a field, and then performing multi-stage quality purification and man-machine collaborative fine correction for false detection and missed detection, to form a high-quality field labeled data set and divide an independent test set; training a field expert large model by using the data set and feeding back to improve the pre-labeling quality, while implementing supervised transfer learning and deployment-oriented compression for a lightweight detection model; and routing optimization instructions back to the data production or model training link according to multi-dimensional error attribution, to form a co-evolution closed loop with the data set as a link. The application can efficiently construct high-quality field data and continuously output a lightweight detection model which can be deployed in real time under the conditions of no initial labeling and limited edge deployment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and more specifically, to a data intelligence production and model self-evolution system and method based on large models. Background Technology

[0002] With the widespread adoption of deep learning object detection technology across various vertical industries, lightweight detection models, due to their low computational, memory, and inference latency overhead, are well-suited for deployment on resource-constrained edge devices, making them the preferred choice for real-time detection in practical scenarios. However, lightweight detection models have limited recognition and generalization capabilities in unfamiliar scenes. To achieve reliable performance in new domains, they heavily rely on training with sufficiently large-scale, domain-annotated data covering the target scene. Building an annotated dataset from scratch for a new domain involves high manual annotation costs, long cycles, and high professional barriers, becoming the main bottleneck restricting its rapid deployment.

[0003] On the other hand, large visual models have open-vocabulary zero-shot detection and cross-scene generalization capabilities, and can perform automatic pre-annotation on unlabeled images, providing a new way to reduce annotation costs; however, their parameters are huge and inference overhead is high, making it difficult to deploy directly on edge devices. Moreover, the automatic pre-annotation results inevitably have false detections, multiple detections and missed detections, and cannot be directly used for training without review, correction and quality purification.

[0004] Current technologies lack a systematic mechanism that combines the generalization and pre-annotation capabilities of large visual models with the deployability of lightweight models, and enables the two to co-evolve through data. This makes it difficult to simultaneously address the rapid construction of new scene data and the real-time deployment of models on edge devices. Therefore, how to leverage the generalization capabilities of large visual models to produce and correct high-quality domain-annotated datasets required for training lightweight detection models, and then train lightweight detection models that can be deployed in real time and enable data and models to co-evolve, is a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0005] This invention provides a data intelligence production and model self-evolution system and method based on large models, which solves the technical problems in related technologies such as lightweight detection models lacking the ability to recognize unfamiliar scenes and highly relying on large-scale domain labeled data, high cost, long cycle and high threshold for manual labeling from scratch in new fields, difficulty in directly deploying large visual models on resource-constrained edge devices and the existence of false detection and missed detection in their automatic pre-labeling.

[0006] This invention provides a data-driven intelligent production and model self-evolution method based on large models, comprising the following steps:

[0007] S1. Obtain unlabeled original images. First, use a domain-adapted visual teacher large model to perform open vocabulary zero-sample automatic pre-labeling. Then, perform multi-stage quality purification and human-machine collaborative fine-tuning to address both false detections and false negatives. Output the domain-labeled dataset and divide it into independent test sets accordingly.

[0008] S2 uses the domain-labeled dataset as training data. Based on the data size, it selects one of the visual base models to perform full training and parameter fine-tuning to obtain a domain expert model and sends it back to improve the quality of automatic pre-labeling.

[0009] S3 addresses the deployment constraints of edge devices in terms of computing power, memory, and real-time performance. It utilizes domain-labeled datasets to perform end-to-end supervised transfer learning on lightweight detection models selected from a pre-built repository and applies deployment-oriented compression optimization to produce ready-to-use lightweight detection models.

[0010] S4 performs standardized performance evaluation and multi-dimensional error attribution on the domain expert large model and the ready-to-use lightweight detection model on an independent test set, and routes optimization instructions back to the data production and model training stages according to the attribution conclusions.

[0011] Furthermore, the false detection dimension processing of the multi-stage quality purification in S1 includes: eliminating noise predictions with a first preset confidence threshold, marking those with confidence between the first preset confidence threshold and the second preset confidence threshold as pre-labeled for review, and marking those with confidence not lower than the second preset confidence threshold as highly reliable pre-labeled.

[0012] An undirected graph is constructed for the retained detection boxes. Two boxes with an intersection-union ratio (IU) not lower than a preset clustering threshold are connected and divided into detection clusters after traversal. When the category within a cluster is unique, the box with the highest confidence is retained. When the categories within a cluster conflict and the highest confidence is lower than a preset conflict resolution threshold, the smallest bounding rectangle of each box within the cluster is taken as a proxy bounding box and added to the manual review queue along with the conflict category. After completing the intra-cluster resolution, non-maximum suppression is performed to remove the residual overlap between different clusters, and a refined candidate annotation set is output.

[0013] Furthermore, the missed detection dimension processing in the multi-stage quality purification in S1 is to actively discover suspected missed detections in blind areas where the detector does not output a retained prediction box; the active discovery of suspected missed detections adopts at least one of the following: manual review, review of open-source or closed-source teacher models different from the visual teacher large model, screening of low confidence regions, deployment of backflow sample analysis, and feature distribution analysis, and submits the confirmed suspected missed detection candidates along with the relevant categories for manual review.

[0014] Furthermore, the suspected missed detection is actively detected using a three-way signal joint gating: the first way is a category-conditional weak hypothesis signal, which is based on the category-conditional response that still remains in the position where the visual teacher big model does not output a retained prediction box and points to one of the preset category sets. The category-conditional response is derived from the category-aligned feature response map exported from the cross-modal fusion layer of the open-source teacher big model with accessible internal structure and / or the low-score proposal pool that is truncated by the confidence threshold and not retained.

[0015] The second path is the visual deviation signal, which is given by a probability distribution model fitted after extracting local features of each image patch by a fixed self-supervised visual encoder. It is established when the deviation of the image patch features from the probability distribution model exceeds the threshold determined by the product of the mean deviation of the total number of image patches plus the sensitivity coefficient and the standard deviation.

[0016] The third path is the category prototype matching signal, which is established when the cosine similarity between the region aggregation feature and a certain category prototype is not lower than the preset prototype matching threshold. The region aggregation feature is the mean of the local features of each image block covered by the region. The category prototype is constructed by bootstrapping the local features of the image block covered by the pre-labeled box area obtained by the multi-stage quality purification in this batch and with a confidence level not lower than the preset high confidence threshold. The prototype is updated with the confirmed sample after the human-machine collaborative fine-tuning and confirmation of the sample.

[0017] Only when three signals are simultaneously established in a certain spatial area and are not covered by any reserved pre-labeled boxes, is it marked as a suspected missed detection candidate and submitted for manual review along with the relevant category.

[0018] Furthermore, in S1, the verification and correction of the automatic pre-labeling results are performed by human experts and / or open-source teacher models and closed-source teacher models that are different from the visual teacher model. For open-source teacher models with accessible internal structures, verification and correction are performed using their detection boxes, category results, and internal features. For closed-source teacher models that only return detection boxes and category results through interface calls, input-output verification and correction are performed using their output results.

[0019] The human-machine collaborative fine-tuning uses a graphical annotation editor to present three types of objects in a hierarchical manner: highly reliable pre-annotations, pre-annotations awaiting review, and manual review queues. These are used for batch confirmation, boundary and category correction, and key identification and completion, respectively. After fine-tuning, each annotation instance is accompanied by two metadata items: confidence level and annotation source. The adjustment range of the boundary of the manually corrected instance relative to the original pre-annotation box is also recorded. Independent test sets are divided by category from the full set of annotation samples, so that the distribution of each category in the independent test sets is consistent with that in the full set. The independent test sets are not used for training but are only used for performance evaluation and error analysis.

[0020] Furthermore, the data production process also includes data generation enhancement: identifying difficult samples based on the manual operation records of the fine-tuning stage. Difficult samples include suspected missed samples that are confirmed as real targets and corrected samples whose boundary adjustment range is not lower than a preset adjustment threshold.

[0021] Geometric and photometric transformations are applied to the identified difficult samples to perform directional amplification and maintain a balance in the number of categories; generative amplification is performed by constraining the target position with a conditional mask encoded by the bounding box and controlling the target category with semantic embedding of the category attribute; and then the synthetic samples are re-detected within the bounding box using a large visual teacher model to remove invalid samples that fail to detect consistent targets.

[0022] Furthermore, the domain annotation dataset and its generation process information are bound into a versioned data asset package, and its version lineage and the number statistics of annotation instances of each category are recorded;

[0023] The data asset package is fed back to S2 to perform targeted optimization on the visual base model to obtain the domain expert model. The domain expert model is then fed back to update the automatic pre-labeling, forming a positive feedback loop that promotes the improvement of data quality and the enhancement of teacher model capabilities. The loop is linked by the domain labeled dataset and does not rely on knowledge distillation that performs intermediate layer feature alignment between the large model and the lightweight detection model.

[0024] Furthermore, the deployment-oriented compression optimization is performed after supervised transfer learning is completed: a pseudo-quantization operator is inserted during forward propagation in quantization-aware training to truncate the weights and activations during the training phase, i.e., to simulate integer fixed-point operations.

[0025] For scenarios with stricter storage constraints, redundant channels are removed based on the contribution of each convolutional channel to the output activation, and channel-level structured pruning is performed. After pruning, quantization-aware training and calibration are redone. For scenarios with tighter inference latency constraints, the network input resolution is reduced. The quantization-aware training and structured pruning are used individually or in combination, with the convergence condition being that the measured single-frame latency and peak memory on the target edge device meet the deployment specifications. Based on this, a ready-to-use lightweight detection model is output.

[0026] Furthermore, in S4, after standardized performance evaluation, multidimensional error attribution is performed and optimization instructions are routed based on the attribution conclusions: missed detections caused by visual distribution not covered by training data are routed to the data acquisition and automatic pre-labeling stage; confusion caused by ambiguity in the category boundaries of existing labels is routed to the human-machine collaborative fine-tuning stage; systematic positioning offset caused by skewed target spatial distribution in training data is routed to the data generation and enhancement stage; and poor optimization status in the training process is routed to the model training stage.

[0027] The signal strength of each dimension is determined by the statistic of that dimension and the number of samples supporting that statistic. When only one dimension signal exceeds the preset routing trigger threshold, the corresponding link optimization is triggered. When multiple dimension signals are valid, they are merged into a comprehensive optimization instruction or a cascaded optimization instruction is generated and executed sequentially according to the signal strength. When all dimension signals are below the preset routing trigger threshold, a structured diagnostic report is generated and submitted to manual decision-making.

[0028] Furthermore, a data-intelligent production and model self-evolution system based on a large model is provided to execute the steps in the aforementioned data-intelligent production and model self-evolution method based on a large model, including:

[0029] The intelligent data annotation and quality self-inspection module is used to acquire unlabeled raw images. The open vocabulary zero-sample automatic pre-annotation is performed by a domain-adapted visual teacher model. Then, the images are reviewed and corrected by human experts and / or teacher models, and multi-stage quality purification and human-machine collaborative fine-tuning are carried out to address false detections and missed detections. The output domain-annotated dataset is then divided into independent test sets based on this dataset.

[0030] The visual pedestal model training and tuning module is used to select one of the following to perform full training, full parameter fine-tuning or efficient parameter fine-tuning on the visual pedestal large model, based on its data scale, using the domain labeled dataset as training data, to obtain the domain expert large model and send it back to improve the quality of the automatic pre-labeling.

[0031] The lightweight model transfer learning module is used to address the deployment constraints of edge devices in terms of computing power, memory, and real-time performance. It utilizes domain-labeled datasets to perform end-to-end supervised transfer learning on lightweight detection models selected from a pre-built repository and applies deployment-oriented compression optimization to produce ready-to-use lightweight detection models.

[0032] The model validation and closed-loop scheduling module is used to perform standardized performance evaluation and multi-dimensional error attribution on the domain expert large model and the ready-to-use lightweight detection model on the independent test set. Based on the attribution conclusion, the optimization instructions are routed back to the data production stage or the model training stage to form a closed loop of co-evolution of data and model.

[0033] The beneficial effects of this invention are as follows:

[0034] Starting with the open-vocabulary zero-sample pre-annotation of the visual teacher model, and through review and correction by human experts, stronger open-source teacher models and / or closed-source teacher models, as well as multi-stage quality purification to address false detections and false negatives, high-quality domain-specific labeled datasets are efficiently produced without initial annotations, significantly reducing the manual cost and cycle of constructing new domain data. It is compatible with both open-source teacher models with accessible internal structures and closed-source teacher models that can only be called through interfaces, and the correction method is compatible with internal feature utilization and input-output verification.

[0035] A positive feedback loop between data production and model training is established using high-quality domain-labeled datasets as a link. The quality of pre-labeling is continuously improved through targeted augmentation of difficult samples, optional generative semantic amplification, and iterative feedback from domain expert large models. The lightweight detection model can run in real time on edge devices through deployment-oriented compression such as supervised transfer learning and quantized perception training. In the model validation stage, optimization instructions are accurately routed back to the data production or model training stage based on multi-dimensional error attribution such as feature distribution distance, category confusion, and spatial offset. The entire process always uses the dataset as a link and does not rely on intermediate layer feature alignment knowledge distillation between large and lightweight models, thereby continuously producing deployable lightweight detection models without initial labeling and deployment constraints. Attached Figure Description

[0036] Figure 1 This is an overall structural diagram of a data-intelligent production and model self-evolution system based on a large model according to the present invention;

[0037] Figure 2 This is a flowchart of a data-intelligent production and model self-evolution method based on a large model according to the present invention;

[0038] Figure 3 This is a flowchart of a data-intelligent production and model self-evolution method based on a large model according to the present invention. Detailed Implementation

[0039] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0040] At least one embodiment of the present invention discloses a data-intelligent production and model self-evolution method based on a large model, such as Figure 1The diagram shows the overall structure of a data intelligence production and model self-evolution system based on a large model according to the present invention. In the diagram, ① represents the unlabeled original image, which enters the main process of data intelligence production and model self-evolution after data access. ④ represents the large visual teacher model, providing open vocabulary zero-sample pre-annotation to S1 in the main process. Within the main process, S1 sequentially completes pre-annotation, quality purification for false positives and false negatives, human-machine collaborative refinement, data augmentation and enhancement, and asset encapsulation, producing ③ a versioned domain-annotated dataset, which includes domain-annotated data for model training and an independent test set for evaluation. ③ respectively... S2 and S3 provide training data and supply it to S4 with an independent test set; S2 performs full training or parameter fine-tuning on the large visual pedestal model according to the data scale, obtaining a domain expert large model and sending it back to update the automatic pre-labeling of S1; S3 implements supervised transfer learning and deployment-oriented compression on the lightweight detection model, producing a ready-to-use lightweight detection model (⑤) for edge deployment and can form a field feedback loop; the models obtained from S2 and S3 enter S4, where standardized performance evaluation and multi-dimensional error attribution are performed on the independent test set, and optimization instructions are routed back to the data production stage or the model training stage based on the attribution conclusions; the specific steps are as follows. Figures 2 to 3 As shown, it includes:

[0041] S1. Obtain unlabeled original images, perform open vocabulary zero-sample automatic pre-labeling by a domain-adapted visual teacher model, and then have them reviewed and corrected by human experts, open-source teacher models and closed-source teacher models, as well as multi-stage quality purification and human-machine collaborative fine-tuning to address false detections and false negatives. Output the domain-labeled dataset and divide it into independent test sets accordingly.

[0042] This step leverages the generalization ability of the visual teacher model for unfamiliar scenes to generate pre-labels in batches. Then, human experts, stronger open-source teacher models, and / or closed-source teacher models review, correct, and supplement the pre-labeling results. Through multi-stage quality purification targeting false detections (including misdetections and multiple detections) and missed detections, a high-quality domain-specific labeled dataset is accumulated. Specifically, the visual teacher model performs fully automated pre-labeling on the original images, and through a sequentially executed multi-stage quality purification and review / correction process, outputs machine-labeled results with confidence scores, class conflict markers, and suspected missed detection markers. This includes:

[0043] Fully automated pre-annotation; the sources of raw images vary depending on the scene, including drone aerial photography modules, fixed monitoring point equipment, industrial cameras, and endoscope probes, all of which can serve as inputs. Images from different sources often differ in resolution, color space, and acquisition angle. All images undergo a unified preprocessing pipeline before entering subsequent processing, including standardizing the color space to a standard red-green-blue representation, scaling the long and short sides to a fixed resolution while maintaining the original aspect ratio and filling any insufficient parts with zero values, and normalizing pixel values ​​to the input range expected by the model. After preprocessing, the system selects a visual teacher model with open-vocabulary target detection capabilities as the detection executor, using a list of category names of the target business scene as detection prompts, and performs zero-sample detection inference on each image. The visual teacher model can be either an open-source visual model with accessible internal structure or a closed-source visual model that can only be called via an interface and only returns detection boxes and category results.

[0044] In one embodiment of the present invention, the visual teacher big model structurally includes, in sequence, an image input layer, an image backbone network, a category cue encoder, a cross-modal fusion layer, a target query decoding layer, and a detection output layer;

[0045] The image input layer tensors the preprocessed image into a three-channel tensor; the image backbone network consists of a multi-level downsampled visual backbone, outputting multi-scale visual feature maps with decreasing resolution and increasing channels at each level; the category prompt encoder encodes the names of each business category into a set of category semantic vectors after word segmentation and word embedding; the cross-modal fusion layer establishes a correspondence between visual features and category semantic vectors through cross-attention, enabling each visual region to obtain its matching response with each candidate category; the target query decoding layer iteratively decodes the location and category affiliation of potential targets on the fused features using a set of learnable target query vectors; the detection output layer regresses the bounding box coordinates and outputs the matching score and confidence score of the box with each category; each detection result is stored in the form of a structured record, including the pixel coordinates of the upper left and lower right corners of the bounding box, the category prediction label, and the confidence score; in the implementation where a feature distribution baseline is established, after the self-supervised visual encoder completes image segmentation, this record is also filled with the set of image block locations covered by the bounding box for subsequent deviation calculation and category prototype construction; this field is not included in simple tasks without a feature baseline.

[0046] In some embodiments, for application scenarios where the target may be highly similar to the background and the detector is prone to missing detections due to not traversing the bounding box, for open-source visual teacher models with access to their internal structure, category-aligned feature response maps (i.e., category-conditional feature maps output by the cross-modal fusion layer) covering all spatial locations in the entire image and for each candidate category can be exported from its cross-modal fusion layer and saved with the image. These feature response maps characterize the matching activation intensity of each spatial location with each category cue. Even if the detector does not output any predicted bounding boxes that reach the retention threshold at a certain location, that location still retains continuous category-conditional activation for the relevant category, thus providing a source of weak hypothesis signals that does not depend on bounding boxes. For closed-source visual teacher models that can only be accessed through an interface and whose internal features cannot be obtained, this process does not rely on the above feature response maps, but only uses its output detection boxes and category results, and completes subsequent missed detection and correction in an input-output manner (such as using a stronger open-source or closed-source teacher model for secondary verification, multi-model cross verification, manual verification, low-confidence region screening, etc.). All records are summarized into an initial candidate annotation set and then enter the quality purification process.

[0047] The system employs a dual-threshold confidence level stratification and cross-union ratio (CUI) graph conflict resolution. First, a first pre-set confidence threshold is established as a noise cutoff standard. Detection results with confidence levels below this threshold are considered highly uncertain noise predictions and removed from the main process, preventing them from participating in subsequent clustering and conflict resolution. Then, a second pre-set confidence threshold higher than the first threshold is set as a high-confidence judgment line. Results with confidence levels between the two thresholds are marked as pending pre-labeling, while results with confidence levels not lower than the second threshold are marked as high-confidence pre-labeling. This allows for differentiated review priorities to be assigned for subsequent fine-tuning.

[0048] It should be noted that in the implementation of the suspected missed detection proactive detection method, the low-score proposals truncated by the noise threshold are not completely discarded. Instead, they are stored in a separate low-score proposal pool along with the category classification determined by the detector. This pool is only used as a weak hypothesis for category conditionalization in the subsequent suspected missed detection proactive detection stage. Both thresholds can be configured by the user according to the characteristics of the domain, or they can be automatically calibrated based on historical annotation verification data. For application scenarios that require pixel-level output, after noise truncation, the retained detection box coordinates are used as spatial location prompts to call the instance segmentation model. The model uses the detection box as a box prompt, and outputs the pixel-level mask of the foreground target within the box after prompt encoding and mask decoding. The mask is associated with the corresponding box and stored together before entering subsequent processing. Tasks that only require bounding box output skip this segmentation step.

[0049] The system then constructs an undirected relation graph for all retained candidate detection boxes. The nodes of the graph represent the candidate boxes. An edge is connected between two boxes when their intersection-union ratio (IU) is not lower than a preset clustering threshold. Here, IU refers to the ratio of the intersection area to the union area of ​​the two boxes' covered regions, ranging from zero to one. A larger ratio indicates more severe overlap and a higher likelihood of pointing to the same target. After the relation graph is constructed, a depth-first search is used to traverse and identify all connected subgraphs. Each connected subgraph constitutes a detection cluster, representing a group of spatially highly overlapping candidate boxes that may describe the same target. For each detection cluster, the system first calculates the set of category labels for all boxes within the cluster and the highest confidence score within the cluster. If the cluster contains only one category, the box with the highest confidence score is directly retained as the final result for that cluster, and the rest are discarded. If multiple boxes contain different category labels, it indicates that different boxes are predicting the same region in conflict. In this case, a conservative decision is initiated. If the highest confidence score within the cluster is not lower than a preset conflict decision threshold, the model's best prediction for the region is considered reliable enough, and the box with the highest confidence score is adopted. If the highest confidence score within the cluster is lower than the threshold, the category assignment is considered to have high-risk ambiguity and should not be automatically decided. Therefore, the minimum value of the horizontal and vertical coordinates of the upper left corner of each box in the cluster and the maximum value of the horizontal and vertical coordinates of the lower right corner are taken to form a minimum bounding rectangle that completely covers all boxes in the cluster as a proxy bounding box. This bounding box, along with the conflicting categories involved in the cluster, is added to the manual review queue. After the intra-cluster decision is completed, non-maximum suppression is performed again to remove the residual slight overlap between different clusters. Finally, a refined candidate label set is output.

[0050] In some embodiments, for application scenarios where the target and background are highly similar and the detector may not leave any bounding boxes, the blind spots where the detector does not retain any predicted boxes can also be handled. The aforementioned two-stage cleanup can only make choices within the range of predicted boxes that the detector has already provided and retained through the threshold. It is powerless for regions where the detector is completely silent in the formal output and does not retain any predicted boxes. However, there are a lot of such blind spots in some scenarios, such as hidden targets that are highly similar to the background, targets with extremely low contrast, and local targets with very small scale. Open vocabulary detectors often do not output any formal predicted boxes that reach the retention threshold for them, and only leave low-score responses suppressed by the noise threshold inside. Such missed detections are difficult to salvage by raising or lowering the confidence threshold.

[0051] One seemingly straightforward approach is to apply general visual anomaly detection, which involves using a feature encoder to extract image region features and then identifying anomalous regions that deviate from the background distribution as missed detections. However, this approach fails in scenarios with highly heterogeneous backgrounds because the background itself may be composed of a mixture of various legitimate contents. Simple visual anomalies will identify a large number of rare but legitimate regions as anomalous, resulting in a flood of false alarms. At the same time, real targets of the same color as the background are missed because they do not disrupt the background texture and therefore have a low degree of anomalousness. In other words, visual anomalies do not equate to missed target detections.

[0052] In some embodiments, the proactive discovery of suspected missed detections in areas outside the detector's frame can be achieved through at least one of the following methods: manual review, review by a stronger open-source or closed-source teacher model, screening of low-confidence areas, deployment of backflow sample analysis, and feature distribution analysis. The following three-way signal joint gating based on weak category conditional assumptions, visual deviation, and category prototype matching is only one preferred implementation. It does not rely on a single abnormal signal, but integrates the three signals, each of which only covers a portion of the clues, to converge the judgment to areas that are both visually suspicious and category-related. When used in combination and required to be true simultaneously, it can further reduce false alarms.

[0053] The first approach is the category-conditional weak hypothesis signal, primarily derived from the category-aligned feature response map exported from the cross-modal fusion layer of the open-source visual teacher large model with accessible internal structure, supplemented by the low-score proposal pool retained during the aforementioned confidence stratification stage. The former covers all spatial locations in the entire image; even if the detector ultimately outputs no predicted bounding boxes at a certain location, the feature response map still provides continuous category matching activations for each preset business category prompt at that location. For a candidate spatial region, the maximum activation value of each spatial location within its coverage area in the corresponding category is taken as the aggregate activation of that region. If the aggregate activation is not lower than the preset weak hypothesis activation threshold in one of the preset categories, it is determined that a weak hypothesis pointing to that category exists in that region. The latter approach supplements the signal by utilizing the signal suppressed by the noise threshold, even if the detector does not output any predicted bounding boxes. Discrete low-scoring proposals that are officially out of bounding boxes but still classified as having a preset category are considered to be triggered by the signal in the corresponding category if either of them is true in a certain category. Thus, the weak hypothesis no longer depends on whether the detector has residual low-scoring boxes. It can also give a category tendency for silent blind spots with no out-of-bounds boxes. This is equivalent to listening to the suppressed low-voice judgment even when the detector is officially outputting a silent position. For closed-source visual teacher large models that cannot derive internal feature response maps, when their interface supports returning low-confidence candidate boxes, the low-scoring proposal pool can be used as the source of weak hypotheses. Otherwise, the detection of missed detections mainly depends on visual deviation signals, category prototype matching signals, or multi-model cross-verification and manual verification that are unrelated to the model's internal structure. The second path is a visual deviation signal that is independent of the detection confidence, given by the feature distribution baseline in the following text.

[0054] The third path is the category prototype matching signal. Considering that this step is performed before human-machine collaborative fine-tuning and there are no manually confirmed samples at this time, the prototype is constructed in a bootstrap manner. That is, the high-confidence pre-labeled boxes that have passed the first two stages of purification are taken from the current batch, and the average value of the image block features within the coverage area of ​​each box is taken to obtain the box-by-box features. Then, the average value of the features of each box of the same category is calculated according to the category to obtain the domain feature prototype of that category. After the human-machine collaborative fine-tuning confirms the samples, the confirmed samples are used to replace and update the prototype, and the prototype is continuously refined in subsequent rounds of closed loop using the confirmed data of the previous round, so that the prototype becomes more and more accurate with iteration. During matching, the average value of the image block features within the coverage area of ​​the candidate region is taken as the candidate region aggregation feature. The cosine similarity between the aggregation feature and a certain type of prototype is examined. The higher the similarity, the more the candidate region resembles the confirmed target of that category. When the cosine similarity between the candidate region aggregation feature and a certain type of prototype is not lower than the preset prototype matching threshold, the signal is considered to be valid in that category.

[0055] In some embodiments, when identifying suspected missed detection areas by combining category cues (weak hypotheses), visual deviations, and category similarity signals, the above signals can be used individually or in combination. When used in combination and a spatial area is required to simultaneously possess a weak hypothesis pointing to a preset category, obvious visual deviations, and proximity to a certain type of prototype, and the area is not covered by any reserved pre-labeled bounding boxes, it is marked as a suspected missed detection candidate and added to the manual review queue along with the relevant category. In this joint gating, category-conditional weak hypotheses and prototype matching keep rare but legitimate background areas out, while visual deviation signals ensure sensitivity to real abnormal targets. Prototype matching can also retrieve targets of the same color as the background based on their proximity to highly credible targets of the same type, thereby reducing false alarms while recalling hidden targets.

[0056] To obtain the aforementioned visual deviation signal, the system first extracts features from the image using a pre-trained self-supervised visual encoder. Structurally, this encoder includes an image segmentation layer, a linear projection and position encoding layer, several cascaded self-attention encoding layers, and an output layer. The image segmentation layer divides the image into non-overlapping square image blocks with a fixed stride. The block size matches the physical scale of the target of interest in the scene, and smaller block sizes are used for hidden and small targets to preserve fine spatial details. The linear projection and position encoding layer flattens each image block and projects it into a fixed-dimensional embedding vector, then superimposes the position encoding and adds a global classification label. The cascaded encoding layers use multi-head self-attention and feedforward sublayers to interactively update the embeddings of all image blocks layer by layer. The output of each layer is a feature sequence of equal length to the input.

[0057] The output layer takes the features corresponding to each image patch in the last layer as the local visual feature vector at that position, and takes the features corresponding to the global classification label as the global feature vector of the whole image. The encoder is pre-trained on a large-scale unlabeled image with self-supervised objectives such as mask image modeling or self-distillation comparison, predicting the features of occluded image patches or merging the representations of different views of the same image. Its parameters are not retrained with the scene in this process but are only used as a stable feature extractor. Scene adaptation is reflected in the statistical distribution modeling of its output features. This not only reuses the cross-domain generalization ability of general features, but also avoids the instability caused by retraining the encoder under small samples.

[0058] In one embodiment of the present invention, after obtaining the local visual features of all image blocks, the system uses these feature vectors as samples to fit a Gaussian mixture distribution model to characterize the statistical distribution baseline of the current domain visual feature space. The model represents the overall probability density of the feature space as a weighted superposition of several multivariate Gaussian components. Each component is described by a mean vector, a covariance matrix, and a mixture weight. Different components correspond to different types of visual content such as background texture, edge transition, and foreground target. The number of components is automatically selected within a preset range using the Bayesian information criterion to balance fitting accuracy and avoid overfitting. The model parameters are solved through expectation maximization iteration. In the expectation step, the posterior probability of each feature vector belonging to each component is calculated. In the maximization step, the mean, covariance, and mixture weight of each component are updated based on the posterior probability. The two steps alternate until the log-likelihood converges.

[0059] After the model is fitted, the overall deviation score is calculated for any image patch. The method is to first calculate the Mahalanobis distance of the image patch feature relative to the mean of the Gaussian component for each Gaussian component. The Mahalanobis distance is used to scale the feature dimensions with the covariance matrix of the component, so that the score is robust to the inconsistency of the variance of each dimension. Then, the Mahalanobis distances of each component are weighted by the posterior probability of the feature belonging to each component. The result is the overall deviation score of the image patch. In scenes where the target samples are relatively sparse, a large number of background image patches are clustered near a few compact components that represent regular texture and have low deviation. A few image patches that disrupt the consistency of background texture fall on the low-density edge of the distribution and have significantly higher deviation. This pattern is independent of whether the detector outputs a prediction box, thus providing a quantitative basis for the visual deviation signal.

[0060] The system then arranges the comprehensive deviation scores of each image patch in each image into a two-dimensional deviation heatmap according to their spatial coordinates. Connectivity analysis is performed on the heatmap to extract spatially connected regions with high deviation. The threshold used to determine high deviation is not a fixed empirical value, but is adaptively determined by the statistical distribution of the deviation scores of all image patches. Specifically, the threshold is the product of the mean of all scores plus a sensitivity coefficient and the standard deviation of the scores. This allows the threshold to be automatically calibrated according to the differences in deviation scales across different domains without requiring manual specification for each scene. For each high-deviation connected region, its minimum bounding box is calculated and compared to the boxes in the pre-annotated box set retained in the previous two stages. The intersection-union ratio is used, and the maximum value is taken as the degree to which the region has been labeled and covered. If the coverage degree is lower than the preset coverage threshold, and the connected region simultaneously satisfies the weak hypothesis condition (i.e., the aggregation activation of the category alignment feature response map within its coverage area is not lower than the weak hypothesis activation threshold in a certain category of the preset category, or there is a proposal in the low-score proposal pool that spatially overlaps with the region and points to one of the preset categories) and the prototype matching condition (i.e., the cosine similarity between the aggregation feature of the region and a certain prototype is not lower than the prototype matching threshold), then it is confirmed as a suspected missed candidate region, and is added to the manual review queue along with the comprehensive deviation score, the category involved and the spatial coordinates.

[0061] Considering that acquisition anomalies such as strong reflection and lens vignetting may also cause high deviations, the system does not make any automatic category assertions for these areas, but only leaves them to the annotators to identify based on their professional judgment during the fine-tuning stage. Compared with the conventional approach of filtering based solely on confidence, this step extends the effective boundary of quality control from the selection of existing predicted boxes to areas where the detector is completely silent, achieving targeted coverage of blind spots with limited human input. For simple tasks where there is no risk of such missed detections, this step and the establishment of the feature distribution baseline mentioned above can be omitted, and only the pre-annotation results from the first two stages will be output.

[0062] It should be noted that the characterization of the aforementioned feature distribution baseline is not limited to the Gaussian mixture model. A single multivariate Gaussian model, kernel density estimation, or density assessment based on nearest neighbor distance can also be used to characterize the background distribution and provide the deviation. These three methods can be substituted for each other when the background texture is relatively simple, differing only in their fitting ability to multimodal backgrounds and computational cost. When there are many types of background visual content, Gaussian mixture is preferred to avoid underfitting. Regarding category prototype construction, in addition to using the arithmetic mean of the confirmed target region features as the prototype, a confidence-weighted mean or retaining multiple sub-prototypes can also be used to characterize multiple appearances of the same category. Both are more robust when there are large differences in appearance within the category, and are fully compatible with the directional proximity judgment logic of the main scheme. Regarding the source of weak assumptions for category conditionalization, the open-source teacher model with accessible internal structure takes the category alignment feature response map output by its cross-modal fusion layer as the main source. In closed-source models or lightweight implementations where it is inconvenient to derive the feature response map, the discrete low-score proposals retained by the detector split can be reused as the source of weak assumptions. One or both can be selected. The difference is that the feature response map is more detailed in locating blind areas with no out-of-frame and the discrete proposals are simpler to implement. Both are compatible with the judgment logic of the above-mentioned multi-signal joint gating.

[0063] In summary, through pre-annotation by the visual teacher's large model and multi-stage quality cleanup targeting false detection, multiple detection, and missed detection, the original image is transformed into a structured machine pre-annotation result. Each target instance is associated with its bounding box coordinates, the final category label after adjudication, and the confidence score. In tasks requiring pixel-level output, the pixel-level mask of the instance is also associated. The pre-annotations are classified according to their quality status into high-confidence pre-annotations that can be confirmed in batches, pre-annotations that need to be quickly corrected and awaiting review, and a manual review queue that requires manual or teacher model review and correction. The latter includes both false detection and multiple detection category conflict areas generated in the conflict resolution stage and suspected missed detection candidate areas generated in the above-mentioned optional suspected missed detection active discovery stage.

[0064] In one embodiment of the present invention, based on the machine pre-annotation results of the aforementioned automatic pre-annotation and multi-stage quality purification output, an interactive fine-tuning environment is provided through a graphical annotation editor, allowing annotators to intervene as reviewers and correctors rather than creators from scratch. Through review, confirmation, correction, and supplementation operations, a high-quality domain annotation dataset is formed and divided into independent test sets; specifically including:

[0065] Interactive editing; the editor presents three types of labeled objects in a layered visual manner. Highly reliable pre-labeled objects support batch one-click confirmation without requiring box-by-box verification. Pre-labeled objects awaiting review are presented with highlighted boxes, allowing annotators to drag and adjust boundaries or correct categories using a dropdown menu. The manual review queue, including category conflict areas and suspected missed detection candidate areas, is highlighted with a prominent color, guiding annotators to focus on determining whether the target is indeed present, confirming the correct category, and completing the label boxes. In addition to manual review, a more robust open-source or closed-source teacher model can be invoked to perform secondary review of the false positives, multiple detection conflicts, and suspected missed detection candidates in the review queue using an input-output approach. The system provides correction suggestions, which are then manually confirmed, forming a fine-tuning method that combines manual review and / or teacher model review and correction. The editor also provides various annotation tools such as rectangles and polygons to adapt to both detection boxes and pixel-level masks. It also integrates a keyboard shortcut system covering four types of operations: confirmation, deletion, adjustment, and completion, to minimize mouse operation paths. Since the members of the manual review queue are generated by conflict detection and feature deviation statistics rather than randomly selected, the density of real annotation problems is significantly higher than that of full random sampling. Therefore, with the same amount of manual input, higher annotation coverage and accuracy can be achieved.

[0066] After fine-tuning, each labeled instance is accompanied by two pieces of metadata: confidence level and label source. The former distinguishes between high confidence, corrected, and manually completed, while the latter distinguishes between automatic confirmation, manual correction, and manual completion. For manually corrected instances, the adjustment range of their boundaries relative to the original pre-labeled boxes is also recorded. In implementations that have established feature baselines, the grid coordinates and comprehensive deviation score of the corresponding image patch can also be included for direct use in subsequent steps. The above metadata is fully retained with each dataset version and, together with the statistics of the number of labeled instances in each category, constitutes the high-quality domain-specific labeled dataset output in this step. This dataset supports one-click export to industry-standard labeling formats such as YOLO (You Only Look Once) and COCO (Common Objects in Context) for seamless integration with other computer vision frameworks and can be directly transmitted to downstream data augmentation and enhancement stages, as well as S2 and S3. The metadata such as label source, confidence level, boundary adjustment range, and comprehensive deviation retained with the dataset also serve as the basis for identifying difficult samples and formulating enhancement strategies in the data augmentation and enhancement stages.

[0067] Independent test set partitioning: Independent test sets are partitioned from the full set of labeled samples by stratified sampling according to category, with a sampling ratio of, for example, between 10% and 20%, to ensure that the distribution of each category in the test set is consistent with that in the full set; In the implementation method that establishes feature baselines, the images included in the test set have already completed feature extraction during the automatic pre-labeling stage, and their feature vectors and comprehensive deviation scores have been stored in the database. Therefore, when S4 performs error analysis, it can directly read the data without repeating inference on the test images; The labels in the test set do not participate in the data augmentation and reinforcement stages or the model training of S2 and S3, but are only used for the performance evaluation and error analysis of S4 to ensure that the evaluation conclusions are strictly independent of the training process;

[0068] In one embodiment of the present invention, based on the high-quality domain-specific labeled dataset produced by the aforementioned human-machine collaborative refinement, automated data augmentation and enhancement are performed to obtain a larger, more comprehensive, and more balanced enhanced training dataset; specifically including:

[0069] The main approach is to use conventional data augmentation such as geometric and photometric transformations and active augmentation targeting model weaknesses, and optionally introduce semantic-level augmentation based on generative models. Since the scale and diversity of high-quality annotation sets are still limited by the number of original collections and scene coverage, the coverage of difficult samples with complex visual appearance and low frequency of occurrence is often insufficient, and targeted expansion is needed on the premise of ensuring strict and effective annotation.

[0070] Difficult sample identification and traditional targeted augmentation: Difficult sample identification integrates two types of signals. The first type comes from manual operation records during the fine-tuning stage, namely, samples that have been confirmed as real targets by the annotator from the suspected missed candidates in the manual review queue. These belong to the visual subclass of systematically missed annotations by the detector, and samples that have triggered correction records due to significant boundary adjustments. These belong to the difficult appearances due to insufficient detector positioning accuracy. In the implementation method that has established a feature baseline, a second type of signal can also be introduced, namely, samples with high deviation but have been confirmed as valid targets. These are located near the detector's perception boundary and require more targeted augmentation. The union of the two types of signals yields the difficult sample set, which is used to describe the subspace with weak visual coverage in the current data. For the identified difficult samples, targeted augmentation is first performed using geometric transformations such as rotation, flipping, scaling, and random cropping, as well as photometric transformations such as brightness, contrast, and color jitter, to improve their sample density and diversity without relying on any generative model. Overall augmentation is performed on all categories to maintain the quantity balance between categories. The category to which the difficult sample belongs is augmented with an additional number of augmentations on top of the global baseline.

[0071] In some embodiments, when traditional augmentation is insufficient to compensate for semantic diversity, generative augmentation based on annotation conditions can be further employed. Its core is to directly inject existing annotation information into the generation process, ensuring that the spatial location and category semantics of the target in the generated image naturally correspond to the input annotations, rather than verifying them after generation. Specifically, a binary conditional mask encoded with the coordinates of the annotation box serves as the spatial structure condition. This constraint is embedded into the denoising process of the generative model through a feature layer fusion channel, thereby constraining the target's position in the generated image. Semantic embedding, encoded with category names and attributes such as color, texture, and shape, guides the appearance of the target region during denoising through cross-attention, thus controlling the target's category classification. Under dual conditional constraints, the generative model performs diverse random synthesis of background, lighting, hue, and viewpoint outside the target, resulting in new samples with diverse environments where the annotations still hold true.

[0072] The aforementioned diffusion-based generative model is pre-trained using the mean square error of the noise added during the forward noise generation process as the training objective. For highly specialized fields, this training objective can be further optimized using dozens of original refined samples for few-sample domain fine-tuning, injecting the appearance features of the domain target into its semantic concept space to improve the realism of the synthesis. To filter out invalid samples generated by the generative model when the conditions are not met, the system performs a detection and re-inspection on each synthesized sample. That is, the synthesized image is re-detected within the labeled bounding box using the current version of the visual teacher model. If no instance consistent with the target category and meeting the confidence standard is detected, it is determined that the target is missing or semantically drifted and is removed. In implementations where a feature baseline has been established, feature semantic consistency verification can also be added, i.e. The similarity in direction between the prototype features of this domain and the aggregated features of the target region of the synthetic sample is taken. Samples with a similarity lower than a preset threshold are considered to deviate from the real domain distribution and are removed. The synthetic samples retained after verification meet the requirements in terms of correspondence in the labeled space and consistency of semantic features, and can be used as reliable augmentation data. It should be noted that the above generative amplification is not limited to diffusion generation based on labeled conditions. Adversarial generation based on layout conditions or traditional copy-paste synthesis can also be used. As long as the target spatial position and category semantics can be kept to correspond with the input labels, it can be connected to subsequent detection re-examination and feature consistency verification. The three have different trade-offs in sample authenticity and controllability. For professional appearance fields with higher requirements for authenticity, the fine-tunable diffusion generation is preferred.

[0073] In one embodiment of the present invention, all generated data is uniformly assetized and version-managed, driving the entire data production system to continuously improve itself along with the evolution of teachers' large-scale model capabilities; specifically including:

[0074] Data asset encapsulation: The system binds the enhanced training dataset and its generation process information into a versioned data asset package. Records include the generation model used and its parameter configuration, the feature extraction model and feature library version used, various threshold settings, and the corresponding index between the enhanced samples and the original labeled dataset. The version hierarchy also records the quantity changes and the proportion of difficult samples for each category before and after enhancement, providing reference for S2 and S3 when configuring sample weights and for S4 when tracing data sources. Based on this, the system provides multi-dimensional quality quantification insights into the dataset, including the balance of the number of instances in each category, the distribution of target bounding box scales, and the composition of annotation sources. Data is archived and managed according to label systems for different domain scenarios, making data quality measurable and labels scalable. The asset package contains a mixed encapsulation of original, finely-calibrated samples and validated enhanced samples, with each sample labeled with its source for differentiated weighting during training.

[0075] Closed-loop evolution is the driving force. This involves establishing a correlation between the dataset version and its generation environment, thereby driving the system's self-evolution. This closed loop is only uniformly described here in this specification; all subsequent steps involving feedback refer to this mechanism. Specifically, this step automatically feeds back the high-quality dataset produced by S1 to S2 for targeted optimization and iteration of the teacher's large model. The teacher's large model optimized by S2 is then fed back to update the automatic pre-labeling. In the implementation where a feature baseline has been established, the feature distribution baseline and image feature library are also updated simultaneously, before being put into a new round of data production. This forms a positive feedback loop where improved data quality leads to enhanced capabilities of the teacher's large model, which in turn drives the production of higher-quality data. This ensures that the output benchmark of the entire data production system continuously and autonomously evolves with usage. Furthermore, this closed loop always uses the dataset as the link and does not rely on intermediate layer feature alignment knowledge distillation between the large model and the lightweight model.

[0076] S2 uses the domain-labeled dataset as training data. Depending on the data size, one of the visual base models is selected to perform full training and parameter fine-tuning to obtain a domain expert model, which is then sent back to improve the quality of automatic pre-labeling.

[0077] In addition to feeding back into data production, this domain expert large model can also serve as an optional data-level aid for training the lightweight S3 model (i.e., outputting pseudo-labels, soft category suggestions, and hard sample hints to supplement or weight training data), without requiring intermediate layer feature alignment-based knowledge distillation between this model and S3. The visual foundation large model used is structurally consistent with the visual teacher large model described in the aforementioned automatic pre-labeling step, and various cutting-edge visual foundation large models are pre-configured for users to choose from based on task objectives and computing resources (e.g., visual large models based on the Transformer architecture). This step aims to converge this structure from general visual cognition to domain expert cognition; specifically including:

[0078] Domain-adaptive training; training is aimed at the detection task, and the supervision signal is taken from the high-quality domain-labeled dataset. When data augmentation and enhancement are enabled, it is the bounding boxes and category labels in the enhanced training dataset after the data augmentation and enhancement process. The data flow during training is as follows: images are processed by an image backbone network to extract multi-scale features, category hints are encoded into semantic vectors by a category hint encoder, and after the two are aligned in the cross-modal fusion layer, the target query decoding layer decodes the predicted target. The detection output layer provides the predicted box and category matching score, and then calculates the loss with the ground truth labels and backpropagates. The loss consists of a weighted sum of three parts. The first part is the classification loss, which measures the correctness of the category prediction. When the ratio of positive and negative samples is severely imbalanced, the focus loss is used to reduce the dominance of easily classified samples on the gradient. When the ratio of positive and negative samples is relatively balanced, the cross-entropy is used. The second part is the bounding box regression loss, which measures the error of the bounding box position regression and uses the difference norm of the coordinates of the predicted box and the ground truth box. The third part is the localization loss, which measures the degree of fit of the bounding box shape and uses the generalized intersection-union loss of the predicted box and the ground truth box. When pixel-level output is required, the pixel-wise cross-entropy of the mask region is added.

[0079] Parameter update strategy selection: The strategy depends on the data scale. When the domain data scale is very small, low-rank adaptation fine-tuning is used. That is, the product of two low-dimensional matrices is introduced as the incremental parameter on the query projection and value projection of the attention module of the large model, and only this increment is trained, while the original weights are frozen. This achieves adaptation with very few trainable parameters and avoids catastrophic forgetting under small sample conditions. When the data scale is moderate, adapter fine-tuning is used. That is, a bottleneck adapter consisting of a dimension-reduced linear layer, a non-linear activation layer, and a dimension-increasing linear layer is inserted after the feedforward sub-layer of each coding block. Only the adapter is trained, while the main body is frozen. When the data scale is sufficient and the computing power allows, full parameter fine-tuning is performed with an initial learning rate much smaller than the pre-training learning rate. The learning rate is updated in a fine-grained manner for all parameters; when the domain data is extremely abundant, a complete full training can be performed starting from random initialization; all the above strategies are combined with cosine annealing learning rate scheduling with linear warm-up to avoid large gradients destroying pre-trained knowledge in the early stage of training and to achieve smooth convergence in the later stage; after training is completed, the training data version, category system, hyperparameters, weight version and evaluation index of the model are recorded, and the resulting domain expert big model is sent back to S1 according to the aforementioned closed-loop evolution drive mechanism. After being sent back, the model assumes the role of a new round of visual teacher big model in the automatic pre-labeling stage to improve the quality of subsequent pre-labeling, and can provide optional teacher guidance at the data level for S3 in the manner described in this section.

[0080] S3 addresses the deployment constraints of edge devices in terms of computing power, memory, and real-time performance. It utilizes a domain-labeled dataset to perform end-to-end supervised transfer learning on lightweight detection models selected from a pre-built repository and applies deployment-oriented compression optimization to produce ready-to-use lightweight detection models.

[0081] The high-quality domain-labeled dataset, when data augmentation and enhancement are enabled, is the augmented training dataset after the data augmentation and enhancement process. Taking a commonly used single-stage lightweight detection model as an example, its structure includes an input preprocessing layer, a backbone feature extraction layer, a neck feature fusion layer, and a detection output layer. The input preprocessing layer performs image scaling and normalization. The backbone feature extraction layer extracts multi-scale features, including edges, textures, shapes, and even high-level semantics, through cascaded convolutions. The neck feature fusion layer fuses features of different scales through top-down and bottom-up feature pyramid structures to accommodate both large and small targets. The detection output layer outputs bounding box coordinates, target presence confidence, and class probability on grids of multiple scales. Specifically, it includes:

[0082] Supervised transfer learning: First, load general pre-trained weights or weights from the previous version, and modify the output dimension of the detection output layer according to the number of domain categories. Then, use the augmented dataset output from the data augmentation and enhancement stages as a supervision signal to drive the model backbone to fully learn new domain features. The data flow is as follows: the image is preprocessed by input and multi-scale features are extracted by the backbone. After neck fusion, it is sent to the detection output layer to obtain predictions on grids at each scale. Then, it is matched with the real annotations according to scale and position to calculate the loss. The loss consists of three parts: the first part is the bounding box regression term using complete intersection-union loss, which considers the overlap of boxes, center distance, and aspect ratio. The second part is the target existence confidence term using binary cross-entropy. The third part is the category classification term using binary cross-entropy or cross-entropy. During the training process, the weights are selected based on the mean precision, precision, recall, harmonic mean score, inference speed, and model volume on the validation set.

[0083] Optional assistance can be introduced from the S2 domain expert big model, but this assistance is limited to teacher guidance at the data level, that is, using pseudo-labels, soft category suggestions, hard sample hints or annotation review results output by the teacher big model to supplement or weight the training data. It does not require intermediate layer feature alignment or similarity matrix distillation between the teacher big model and the lightweight model. The main line of lightweight model training is always supervised transfer learning based on high-quality domain labeled datasets.

[0084] Deployment-oriented compression optimization; after transfer learning, compression optimization is sequentially performed on the lightweight model; during the quantization-aware training phase, a pseudo-quantization operator is inserted into the model's forward propagation to simulate the numerical truncation effect of integer fixed-point operations on weights and activations during the training phase, thereby allowing the model to adapt to the accuracy range after quantization in advance and ensuring that the accuracy loss when formally converting to integer inference remains within an acceptable range for engineering purposes; for scenarios with stricter storage constraints, channel-level structured pruning can be performed before quantization, assessing the importance of each convolutional channel based on its contribution to the output activation and removing redundant channels according to a preset ratio to reduce the number of parameters and multiply-accumulate operations. After pruning, quantization needs to be redone. Quantization-based training and calibration are performed to restore accuracy. For scenarios with tighter inference latency requirements, the network input resolution can be appropriately reduced to decrease computation and achieve a balance between accuracy and speed. After compression optimization, end-to-end inference verification is run on the target edge hardware to confirm that the single-frame inference latency and peak memory usage meet the deployment specifications. Finally, a ready-to-use lightweight detection model is output. It should be noted that quantization-based training and structured pruning can be used alone or in combination. For scenarios with only volume constraints, only quantization can be performed. For scenarios with both volume and computing power constraints, pruning can be added. Both convergence conditions are based on the actual measured latency and memory usage on the target hardware.

[0085] S4 performs standardized performance evaluation and multi-dimensional error attribution on the domain expert large model and the ready-to-use lightweight detection model on the independent test set. Based on the attribution conclusion, the optimized instruction routing is routed back to the data production and model training stages.

[0086] This step involves performing pre-annotation of the large-scale teacher model and multi-stage quality cleansing (including proactive detection of suspected omissions) on the field images returned during deployment and operation, following the same process described in S1. Quality inspectors then quickly confirm the identified suspected omissions and conflict areas, resulting in a field evaluation set with ground truth annotations (its image patch features are simultaneously stored in the database). This set is then incorporated into the same evaluation process to detect performance degradation in new scenarios. Based on the evaluation and error analysis results, intelligent decision-making automatically triggers data optimization returning to S1 or model retraining returning to S2 or S3. It supports two modes: single-image testing for quick qualitative review and batch dataset testing for generating quantitative reports. It also supports parallel evaluation of multiple models with different architectures, parameters, or data versions on the same test set, generating horizontal comparison reports and visualizations to provide a basis for model selection and iteration decisions. Specifically, this includes:

[0087] Performance evaluation involves batch inference of the model to be validated on the test set. The matching between predicted and ground truth bounding boxes is statistically analyzed for each image, and the number of true positives, false positives, and false negatives for each category is recorded. Based on this, precision, recall, and harmonic mean score are calculated for each category. The average precision is calculated separately at multiple thresholds, including an intersection-union ratio (IU) threshold of 0.5 and thresholds ranging from 0.5 to 0.95 with a step size of 0.05, and then summarized as the mean precision. For lightweight models, single-frame latency and peak memory usage are also recorded on the target hardware to verify deployment specifications. These results form the quantitative basis of the performance validation report. The performance validation report includes basic information about the model and test set, a summary of performance metrics, and detailed analysis. The report comprises five parts: charts, error case analysis, and improvement suggestions. The detailed analysis charts visually present the performance differences and error distribution patterns of each category using precision-recall curves, performance curves at different confidence thresholds, confusion matrices for each category, and radar charts and bar charts for cross-model comparison. The error case analysis automatically identifies typical false positives and false negatives and provides a visual comparison of the detection results with the actual labeled data. The improvement suggestions provide preliminary optimization directions based on the indicators and error patterns. When the report shows instances where indicators fail to meet standards, this step further performs multi-dimensional error analysis to pinpoint the root cause to a specific stage, avoiding blind backtracking without understanding the underlying reasons.

[0088] In one embodiment of the present invention, an attribution mechanism covering at most four stages—data acquisition, annotation quality, generation enhancement, and model training—can also be established. The feature distribution distance attribution for data acquisition is only performed in implementations where a feature baseline has been established; otherwise, the other three dimensions are used. Optimization instructions are routed to the corresponding stages based on the attribution conclusions. This mechanism has a structural correspondence with upstream stages: visual distributions not covered by training data correspond to data acquisition and pre-annotation; ambiguous category boundaries in existing annotations correspond to human-machine collaborative fine-tuning; skewed spatial distribution of training data corresponds to directional enhancement; and poor optimization during training corresponds to model training parameter tuning. The first dimension, feature distribution distance attribution, is performed in implementations where a feature baseline has been established. This is achieved by performing a comprehensive analysis of all missed detections in the test set. The region retrieves existing features from the image feature library and calculates their comprehensive deviation using the established Gaussian mixture model. Then, it calculates the proportion of missed regions with deviations exceeding the out-of-distribution determination threshold among all missed regions as the signal strength of this dimension. The out-of-distribution determination threshold is the global mean of the deviation plus a large multiple of the standard deviation, making it higher than the threshold used in the above-mentioned optional suspected missed detection active discovery step, to ensure that only those with extremely significant deviations are identified as out of distribution. When this proportion exceeds the routing trigger threshold, it is determined to be a missed detection caused by insufficient coverage of out-of-distribution samples by the training data, triggering the supplementary acquisition and pre-labeling of S1. The global features of the entire image of the missed image are compared with the global features of existing images in the library to locate the visually closest reference sample to guide the acquisition direction of new scenes.

[0089] The second dimension is the attribution of category confusion patterns. This involves constructing a complete confusion matrix based on all predictions in the test set, identifying the category pairs with the most severe misclassifications. If high-frequency confusion is concentrated between categories with similar appearances (in implementations with established feature baselines, it can be further verified whether the feature distributions of this category pair significantly overlap in the training set to strengthen the criterion), then it is determined to be confusion caused by ambiguity in category boundary labeling. This triggers a human-machine collaborative fine-tuning process in S1, where the historical labels of the relevant categories are reviewed for consistency, the category distinction criteria are solidified, and samples with blurred boundaries are relabeled. The third dimension is the attribution of spatial offset patterns. This involves calculating the horizontal and vertical offsets of the predicted bounding box center relative to the ground truth bounding box center for all successfully matched ground truth examples in the test set. After normalization using the shorter side length of the ground truth bounding box as a benchmark, the range defined by the mean of the normalized offsets of all ground truth examples plus or minus a preset multiple of the standard deviation is taken as the normal range. Samples with offsets exceeding this range are filtered out. Then, the significance of the mean offset relative to its standard deviation in the horizontal and vertical directions, i.e., the degree of directional clustering, is examined, and the larger of the two directions is taken. As a signal in this dimension; when this signal exceeds the judgment threshold, it indicates that these positioning errors are not randomly distributed, but rather a systematic shift of the predicted box center in the same direction; the systematic shift of the same direction usually originates from the uneven spatial distribution of the target in the training data in this direction, causing the regressor to learn a directional positional prior rather than truly fitting the target, thus triggering the directional enhancement of the return data amplification and reinforcement stage, applying geometric transformations such as random translation and random cropping to the relevant category samples to redistribute the target's spatial position in the image, breaking the directional positional prior learned by the regressor, so that it regresses to truly fitting the target itself rather than relying on the positional prior; the fourth dimension is the training optimization state attribution, which is done by reading the task loss curve and validation curve recorded in the S2 or S3 training stage. If the task loss has converged but the validation performance is still insufficient, the difference between the training and validation losses is used to determine whether overfitting is present. If the training loss is always too high, it is determined that the hyperparameters such as learning rate and batch size are improperly configured, and the corresponding instructions are directed to the model training stage, triggering regularization strategy adjustment or hyperparameter search respectively;

[0090] The system integrates decision-making and self-evolutionary closed-loop iteration; it calculates the signal strength of each dimension, which is determined by the dimension's own statistics (i.e., the out-of-distribution missed detection rate of feature distribution distance attribution, the degree of confusion of the most severe class pair in category confusion attribution, the directional clustering degree of spatial offset attribution, and the difference between the loss and validation curve of training and optimization state attribution) and the number of samples supporting those statistics. The more significant the statistical deviation and the more supporting samples, the higher the strength, thus avoiding misjudgments based on occasional fluctuations of a small number of samples; when only one dimension's signal exceeds the routing trigger threshold and other dimensions do not support the opposite conclusion, the corresponding optimization step is automatically triggered and a new round of processes is started; when two or more dimensions simultaneously give valid signals, the system first checks... If the upstream links they point to overlap, they are merged into a single comprehensive optimization instruction. If they point to different links, cascaded optimization instructions are generated according to signal strength from high to low and executed sequentially. When all dimensions of the signal are below the threshold, the cause is still unclear. Instead, a structured diagnostic report containing quantitative indicators and analysis conclusions of each dimension is generated for manual decision-making to prevent the system from making meaningless iterations based on noisy signals. Based on this, the self-evolutionary closed-loop iterative operation is carried out. After each round of newly generated labeled and augmented data is added to the versioned training dataset, S2 uses the updated dataset to incrementally fine-tune the visual base model to obtain a more capable teacher model and sends it back to S1 to update the pre-labels. S3 then uses the updated dataset to retrain the lightweight detection model.

[0091] In the implementation where a feature baseline is established, as new images and confirmation samples are added, a fixed self-supervised visual encoder extracts additional features from the new images, refits the Gaussian mixture model with the full dataset, recalculates the deviation of each image patch, and covers the original image feature library with the new version. This update is completed before the next round of data production to ensure that the feature baselines used in missed detection, semantic consistency verification, and error analysis come from the same version of the feature extractor. Improvements in each step are transmitted through the training data stream to form positive feedback, enabling the system's data production capacity and model recognition performance to gradually increase with iterations without the need for continuous additional manual input.

[0092] In one embodiment of the present invention, the above method is applied to the intelligent identification of river and lake pollution in the field of river and lake management. The following is a complete process from raw images to deployable models and then to closed-loop triggering using a set of specific data. The system first links the drone and fixed monitoring point equipment through the pre-set field image acquisition module to automatically collect about 2,500 river and lake shoreline images covering different water types, shoreline topography and lighting conditions. No initial annotation is required from the user throughout the process. Examples of the sources and attributes of the collected images are shown in Table 1.

[0093] Table 1. Examples of image data collected along river and lake shorelines

[0094]

[0095] In the automatic pre-annotation and multi-stage quality cleanup process, open-vocabulary zero-sample pre-annotation is performed on the images using prompts such as "random occupation," "random sampling," "random stacking," and "random construction," followed by quality cleanup. The teacher's large-scale model can be either an open-source visual large-scale model that can export internal feature response maps or a closed-source visual large-scale model that only returns detection boxes and category results. For example, in an image, one overlapping region is judged as "random construction" with a confidence score of 0.85, and another is judged as "random stacking" with a confidence score of 0.78. These two categories conflict and fall into the same detection cluster. Since the highest confidence score within the cluster, 0.85, does not reach the conflict resolution threshold, the system classifies it as high-risk ambiguity and takes the smallest bounding rectangle of the two as a surrogate box. The data is then manually reviewed. Meanwhile, optionally, when using a multi-signal joint missed detection method, this optional suspected missed detection active detection step calculates a comprehensive deviation score for a region of the same color slag heap where the detector has not left the frame. This score is significantly higher than the threshold obtained adaptively from the mean and standard deviation of the full score. The category alignment feature response map of this region still maintains continuous activation above the weak hypothesis activation threshold for the random heap category (confirmed by a low-score proposal pointing to the random heap in the low-score proposal pool), and is close to the direction of the high-confidence random heap bootstrap prototype in this batch. All three signals are valid simultaneously, and the maximum crossover ratio with existing frames is zero and lower than the coverage threshold. Therefore, it is marked as a suspected missed detection candidate and submitted for review.

[0096] In the human-machine collaborative fine-tuning stage, maintenance and quality inspection experts only need to focus on processing about 150 high-conflict and suspected missed areas marked by the system. By using shortcut keys to confirm, delete, and adjust boundaries, they can obtain a batch of high-quality fine-tuned annotation examples while significantly reducing the workload of manual annotation. In the data augmentation and enhancement stage, the system screens out about 300 difficult samples with weak coverage of hidden piles and minor encroachments based on fine-tuning records and deviation distribution. First, traditional enhancement is performed using geometric and photometric transformations. Then, without changing the shoreline geographical features, annotation conditions can be optionally used to generate simulations of different water background colors, day and night lighting, and shooting angles. After detection, re-inspection, and feature consistency verification, invalid synthetic samples are removed, further expanding the scale and diversity of training samples. In the data asset encapsulation and closed-loop evolution-driven stage, all data, annotation versions, teacher large model versions used, and enhancement parameters are uniformly registered as a traceable data asset package river_lake_four_disorder_v1.0.

[0097] Then, in S2, the system inputs the data asset package into the visual base model training and optimization step. Given that the initial amount of self-generated data is relatively sufficient and compatible with the server's computing power, full parameter fine-tuning is automatically selected. End-to-end fine-tuning is performed on a medium-sized visual base model, with the loss calculated by a weighted sum of three factors: classification, bounding box regression, and generalized intersection-union-comparison localization. After training, a domain expert model is obtained that is deeply optimized for the characteristics of illegal activities in rivers and lakes. This model is then fed back to S1 in a closed loop to improve the pre-labeling quality of subsequent batches. In S3, the user uses the edge equipment of the field shoreline monitoring device. To address the constraints of computing power, memory, and latency, a single-stage lightweight detection model was selected from the pre-built repository of the S3 lightweight model transfer learning step as the base, and its detection output layer was modified according to the number of four random categories. After loading general pre-trained weights, supervised transfer learning was performed using a data asset package. The loss consisted of three components: complete intersection-union ratio, target confidence, and category classification. Subsequently, quantized perceptual training was integrated to adapt the weights to integer inference and compress the model size. The resulting ready-to-use lightweight detection model met the constraints of real-time field detection and edge deployment in terms of accuracy, single-frame inference latency, and model size on the test set.

[0098] Entering S4, after deploying the aforementioned lightweight model to the monitoring points and running it for a period of time, the system automatically collected approximately 500 shoreline images from the actual patrol environment, including complex weather, obstructions, and newly added scenes. These images were first pre-annotated and proactively identified as potentially missed by the large teacher model in S1. Quality control experts then quickly confirmed the identified potentially missed areas, resulting in a field evaluation set with ground truth annotations, and its image patch features were simultaneously added to the database. Subsequently, along with the lightweight model's predictions, this set was submitted for offline evaluation. It was found that the model had significant missed detections of newly emerging temporary illegal dumping on the tidal flats. Based on the first dimension's feature distribution distance attribution, it was statistically determined that the proportion of these missed areas exceeding the out-of-distribution judgment threshold exceeded the routing trigger threshold, confirming that they belonged to the category of... For visual scenes outside the distribution not covered by the training data, a report is automatically generated and a data optimization instruction is triggered to return to S1. The newly acquired images and model predictions are used as pre-labeled and fed back to S1. Experts only need to quickly refine the samples involving temporary illegal dumping on the mudflats. After merging the old and new data and enhancing them, a new version of the data asset package river_lake_four_disorder_v2.0 is generated. This version is then used for incremental fine-tuning of the teacher's large model and retraining of the lightweight model. After this round of retraining, the lightweight model's failure to detect temporary illegal dumping on the mudflats is improved, the overall recognition accuracy of various four types of disorder problems is improved, and the single-frame inference latency and model size are still maintained within the original edge deployment specifications.

[0099] Through the aforementioned self-evolutionary iteration, river and lake management departments can quickly construct a high-quality dedicated dataset for the "four illegal activities" (illegal dumping, illegal dumping, and illegal landfilling) of rivers and lakes without providing any initial seed data. This can be achieved with minimal manual intervention to refine high-conflict and suspected missed detection areas, and a lightweight detection model that can be directly deployed on resource-constrained field edge devices can be obtained. Simultaneously, it enables the automatic discovery of new scenarios such as temporary illegal dumping on tidal flats, rapid expansion of the dataset, and autonomous iteration of the model, forming a continuous adaptive capability to cope with changes in the shoreline environment. It should be noted that this method is not limited to the "four illegal activities" scenario of rivers and lakes, but is also applicable to visual inspection tasks in other vertical fields such as industrial defect detection and security monitoring.

[0100] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A data-intelligent production and model self-evolution method based on large models, characterized in that, Includes the following steps: S1. Obtain unlabeled original images, perform open vocabulary zero-sample automatic pre-labeling by a domain-adapted visual teacher model, and then have them reviewed and corrected by human experts, open-source teacher models and closed-source teacher models, as well as multi-stage quality purification and human-machine collaborative fine-tuning to address false detections and false negatives. Output the domain-labeled dataset and divide it into independent test sets accordingly. S2 uses the domain-labeled dataset as training data. Depending on the data size, one of the visual base models is selected to perform full training and parameter fine-tuning to obtain a domain expert model, which is then sent back to improve the quality of automatic pre-labeling. S3 addresses the deployment constraints of edge devices in terms of computing power, memory, and real-time performance. It utilizes a domain-labeled dataset to perform end-to-end supervised transfer learning on lightweight detection models selected from a pre-built repository and applies deployment-oriented compression optimization to produce ready-to-use lightweight detection models. S4 performs standardized performance evaluation and multi-dimensional error attribution on the domain expert large model and the ready-to-use lightweight detection model on an independent test set. Based on the attribution conclusions, the optimized instruction routing is routed back to the data production stage and the model training stage.

2. The data-intelligent production and model self-evolution method based on a large model according to claim 1, characterized in that, The false detection dimension processing in the multi-stage quality purification in S1 includes: eliminating noise predictions with a first preset confidence threshold, marking those with confidence between the first preset confidence threshold and the second preset confidence threshold as pre-labeled for review, and marking those with confidence not lower than the second preset confidence threshold as highly reliable pre-labeled. An undirected graph is constructed for the retained detection boxes. Two boxes with an intersection-union ratio (IU) not lower than a preset clustering threshold are connected and divided into detection clusters after traversal. When the category within a cluster is unique, the box with the highest confidence is retained. When the categories within a cluster conflict and the highest confidence is lower than a preset conflict resolution threshold, the smallest bounding rectangle of each box within the cluster is taken as a proxy bounding box and added to the manual review queue along with the conflict category. After completing the intra-cluster resolution, non-maximum suppression is performed to remove the residual overlap between different clusters, and a refined candidate annotation set is output.

3. The data-intelligent production and model self-evolution method based on a large model according to claim 1, characterized in that, The missing detection dimension processing in the multi-stage quality purification in S1 is to actively discover suspected missing detections in blind areas where the detector does not output a retained prediction box. The active discovery of suspected missing detections adopts at least one of the following methods: manual review, review by an open-source or closed-source teacher model different from the visual teacher model, screening of low-confidence regions, deployment of backflow sample analysis, and feature distribution analysis. The confirmed suspected missing detection candidates, together with the relevant categories, are submitted for manual review.

4. The data-intelligent production and model self-evolution method based on a large model according to claim 3, characterized in that, The suspected missed detection is actively detected using a three-way signal gating system: The first path is the category-conditional weak hypothesis signal, which is based on the category-conditional response that still remains in the position where the visual teacher big model does not output a retained prediction box and points to one of the preset category sets. The category-conditional response is derived from the category-aligned feature response map exported by the cross-modal fusion layer of the open-source teacher big model with accessible internal structure and / or the low-score proposal pool that is truncated by the confidence threshold and not retained. The second path is the visual deviation signal, which is given by a probability distribution model fitted after extracting local features of each image patch by a fixed self-supervised visual encoder. It is established when the deviation of the image patch features from the probability distribution model exceeds the threshold determined by the product of the mean deviation of the total number of image patches plus the sensitivity coefficient and the standard deviation. The third path is the category prototype matching signal, which is established when the cosine similarity between the region aggregation feature and a certain category prototype is not lower than the preset prototype matching threshold. The region aggregation feature is the mean of the local features of each image block covered by the region. The category prototype is constructed by bootstrapping the local features of the image block covered by the pre-labeled box area obtained by the multi-stage quality purification in this batch and with a confidence level not lower than the preset high confidence threshold. The prototype is updated with the confirmed sample after the human-machine collaborative fine-tuning and confirmation of the sample. Only when three signals are simultaneously established in a certain spatial area and are not covered by any reserved pre-labeled boxes, is it marked as a suspected missed detection candidate and submitted for manual review along with the relevant category.

5. The data-intelligent production and model self-evolution method based on a large model according to claim 2, characterized in that, In step S1, the verification and correction of the automatic pre-labeling results are performed by at least one of human experts, open-source teacher models, and closed-source teacher models, wherein the open-source teacher models and the closed-source teacher models are different from the visual teacher models; wherein the open-source teacher models with accessible internal structures are verified and corrected using their detection boxes, category results, and internal features, and the closed-source teacher models that are only called through the interface and only return detection boxes and category results are verified and corrected using their output results. The human-machine collaborative fine-tuning uses a graphical annotation editor to present three types of objects in a hierarchical manner: highly reliable pre-annotations, pre-annotations awaiting review, and manual review queues. These are used for batch confirmation, boundary and category correction, and key identification and completion, respectively. After fine-tuning, each annotation instance is accompanied by two metadata items: confidence level and annotation source. The adjustment range of the boundary of the manually corrected instance relative to the original pre-annotation box is also recorded. Independent test sets are divided by category from the full set of annotation samples, so that the distribution of each category in the independent test sets is consistent with that in the full set. The independent test sets are not used for training but are only used for performance evaluation and error analysis.

6. The data-intelligent production and model self-evolution method based on a large model according to claim 1, characterized in that, The data production process also includes data generation enhancement: identifying difficult samples based on the manual operation records of the fine-tuning stage. Difficult samples include suspected missed samples that are confirmed as real targets and corrected samples whose boundary adjustment range is not lower than a preset adjustment threshold. Geometric and photometric transformations are applied to the identified difficult samples to perform targeted amplification and maintain a balance in the number of samples between categories; The system uses conditional masks encoded with bounding boxes to constrain target positions and semantic embedding of category attributes to control target categories for generative augmentation. Then, a large visual teacher model is used to re-detect the synthetic samples within the bounding box range, eliminating invalid samples that fail to detect consistent targets.

7. The data-intelligent production and model self-evolution method based on a large model according to claim 1, characterized in that, Bind the domain annotation dataset and its generation process information into a versioned data asset package and record its version lineage and the number statistics of annotation instances in each category; The data asset package is fed back to S2 to perform targeted optimization on the visual base model to obtain the domain expert model. The domain expert model is then fed back to update the automatic pre-labeling, forming a positive feedback loop that promotes the improvement of data quality and the enhancement of teacher model capabilities. The loop is linked by the domain labeled dataset and does not rely on knowledge distillation that performs intermediate layer feature alignment between the large model and the lightweight detection model.

8. The data-intelligent production and model self-evolution method based on a large model according to claim 1, characterized in that, The deployment-oriented compression optimization is performed after supervised transfer learning is completed: a pseudo-quantization operator is inserted into the forward propagation during quantization-aware training to truncate the weights and activations during the training phase, i.e., to simulate integer fixed-point operations. For scenarios with stricter storage constraints, redundant channels are removed based on the contribution of each convolutional channel to the output activation, and channel-level structured pruning is performed. After pruning, quantization-aware training calibration is redone. For scenarios with tighter inference latency constraints, the network input resolution is reduced. The quantitative perception training and structured pruning, used individually or in combination, converge on the condition that the measured single-frame latency and peak memory on the target edge device meet the deployment specifications, thereby outputting a ready-to-use lightweight detection model.

9. The data-intelligent production and model self-evolution method based on a large model according to claim 1, characterized in that, In S4, after standardized performance evaluation, multidimensional error attribution is performed and optimization instructions are routed according to the attribution conclusions: missed detections caused by visual distribution not covered by training data are routed to the data acquisition and automatic pre-labeling stage; confusion caused by ambiguity of category boundaries in existing labels is routed to the human-machine collaborative fine-tuning stage; systematic positioning offset caused by skewed target spatial distribution in training data is routed to the data generation and enhancement stage; and poor optimization status in the training process is routed to the model training stage. The signal strength of each dimension is determined by the statistic of that dimension and the number of samples supporting that statistic. When only one dimension signal exceeds the preset routing trigger threshold, the corresponding link optimization is triggered. When multiple dimension signals are valid, they are merged into a comprehensive optimization instruction or a cascaded optimization instruction is generated and executed sequentially according to the signal strength. When all dimension signals are below the preset routing trigger threshold, a structured diagnostic report is generated and submitted to manual decision-making.

10. A data-intelligent production and model self-evolution system based on a large model, used to execute the steps of a data-intelligent production and model self-evolution method based on a large model as described in any one of claims 1-9, characterized in that, include: The annotation and self-checking module is used to acquire unlabeled raw images, which are automatically pre-annotated with open vocabulary zero samples by a domain-adapted visual teacher model. Then, the images are reviewed and corrected by human experts, open-source teacher models and closed-source teacher models, and undergo multi-stage quality purification and human-machine collaborative fine-tuning to address false detections and missed detections. The resulting domain-annotated dataset is divided into independent test sets. The training and tuning module is used to select one of the visual base large models to perform full training and parameter fine-tuning based on the data size of the domain labeled dataset, and then send it back to improve the quality of automatic pre-labeling. The transfer learning module is used to address the deployment constraints of edge devices in terms of computing power, memory, and real-time performance. It utilizes a domain-labeled dataset to perform end-to-end supervised transfer learning on lightweight detection models selected from a pre-built repository and applies deployment-oriented compression optimization to produce ready-to-use lightweight detection models. The verification and scheduling module is used to perform standardized performance evaluation and multi-dimensional error attribution on independent test sets for domain expert large models and ready-to-use lightweight detection models. Based on the attribution conclusions, the optimization instructions are routed back to the data production and model training stages.

Images