Target recognition method and device, electronic equipment, storage medium and program product

Abstract

The present disclosure relates to a target identification method and device, electronic equipment, storage medium and program product, comprising: obtaining an original image dataset, dividing a general training set and an expert sub-training set, and comparing the data leakage rate; training a general model and an expert model, adjusting the expert training strategy according to the leakage rate; evaluating the performance index and the generalization gap of the model, calculating the adaptive fusion weight combined with the leakage rate, and weighting and fusing the initial fusion model; fine-tuning the final fusion model through a low learning rate; inputting the to-be-identified image into the final model to obtain a candidate box, eliminating false positives through multi-modal large model semantic discrimination, and obtaining the final identification result. Among them, through the hierarchical architecture of the general sub-model and the multiple expert sub-model, the differences existing in the same type of target are modeled, and after adaptive weight fusion and unified fine-tuning, the recall rate of the model is significantly improved in the scene with large intra-class differences, the omission rate is significantly reduced, and high-precision detection and stable output of heterogeneous clusters are realized.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and in particular to a target recognition method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] This section is intended to provide background or context for embodiments of this disclosure. The description herein is not intended to imply that it is prior art simply because it is included in this section.

[0003] Object recognition technology is one of the core tasks in the field of computer vision, widely used in industrial inspection, autonomous driving, remote sensing image analysis, and intelligent security. With the development of deep learning technology, single-stage detectors, represented by the YOLO series, and Transformer-based detectors, represented by the DETR series, have made significant progress. The YOLO series models, with their high throughput and edge-deployability, have become a common choice for industrial applications. Meanwhile, the rise of Open Vocabulary Detection (OVD) and Multimodal Large Models (MLLM) has provided new approaches to solving problems of limited category labels and semantic ambiguity.

[0004] However, when faced with complex scenarios characterized by "large intra-class differences and blurred inter-class boundaries," existing technologies still have the following shortcomings: A single general detection model (such as YOLO) needs to cover multiple morphological, textural, or pose sub-distributions within the same category using a single parameter space. This trade-off leads to a decrease in the model's feature representation ability on specific subclusters, manifesting as unstable bounding boxes, fluctuating confidence levels, and a tendency to produce false positives and false negatives in extreme scales or dense scenes. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems in the related art, this disclosure provides a target recognition method, apparatus, electronic device, storage medium, and program product.

[0006] To achieve the above objectives, a first aspect of the exemplary embodiments of this disclosure provides a target recognition method, the method comprising: Obtain the original image dataset containing the target to be identified, divide the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences, and perform image content hash comparison between the general training set and each expert sub-training set to detect the data leakage rate of each expert sub-training set relative to the general training set. A general object detection model is trained based on the general training set. For each expert sub-training set, a corresponding expert object detection model is trained based on the expert sub-training set. When training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set. Obtain a validation set, evaluate the trained general object detection model and all expert object detection models based on the validation set, obtain the performance index of each model, and calculate the generalization gap of each model on the training set and validation set. Based on the performance index, generalization gap and corresponding data leakage rate of each model, calculate adaptive fusion weights for each model. Based on the adaptive fusion weights, perform weighted fusion of the network parameters of the general object detection model and all expert object detection models to obtain an initial fusion model. The initial fusion model is fine-tuned with a low learning rate based on the general training set to obtain the final fusion model. The image to be identified is input into the final fusion model to generate preliminary candidate target boxes. The image regions corresponding to the candidate target boxes are then input into the multimodal large model. Through semantic consistency discrimination, false positive target boxes that do not conform to the semantic description are eliminated to obtain the final recognition result.

[0007] In some exemplary embodiments, when training the expert object detection model, adjusting the training strategy based on the data leakage rate corresponding to the expert sub-training set includes: If the data leakage rate is lower than a preset threshold, then a standard training strategy is used for training. If the data leakage rate is higher than a preset threshold, a conservative training strategy is adopted for training. The conservative training strategy reduces the learning rate and / or reduces the number of training rounds compared to the standard training strategy.

[0008] In some exemplary embodiments, training a corresponding expert object detection model based on each expert sub-training set includes: The model is conservatively trained using the first learning rate and the first image resolution, and the first-stage model is output. Using the first-stage model, a second image resolution higher than the first image resolution, and / or a higher cross-union ratio threshold as input, progressive enhancement training is performed to output the second-stage model; Using the second-stage model, a third image resolution higher than the second image resolution, and increased regularization intensity as input, fine-tuning training is performed to finally output an expert object detection model.

[0009] In some exemplary embodiments, the step of evaluating the trained general object detection model and all expert object detection models based on the validation set to obtain the performance metric for each model and calculating the generalization gap of each model on the training set and validation set includes: Under the combination of multiple preset confidence thresholds and multiple intersection-union ratio thresholds, the performance index of each model is calculated respectively, wherein the performance index includes precision, recall and the harmonic mean of precision and recall; The generalization gap is the difference between the harmonic mean of precision and recall of the model on the training set and the validation set.

[0010] In some exemplary embodiments, the step of calculating adaptive fusion weights for each model based on its performance metrics, generalization gap, and corresponding data leakage rate includes: The performance metrics, generalization gap, and data leakage rate are jointly calculated using a preset weighting function, and adaptive fusion weights are output for each model. In the weighting function, the greater the generalization gap or the higher the data leakage rate, the smaller the dynamic fusion weight of the corresponding model.

[0011] In some exemplary embodiments, the step of inputting the image to be identified into the final fusion model to generate preliminary candidate bounding boxes, inputting the image regions corresponding to the candidate bounding boxes into a multimodal large model, and eliminating false positive bounding boxes that do not conform to the semantic description through semantic consistency discrimination to obtain the final recognition result includes: Using the image to be identified as input, the final fusion model is used to generate a preliminary set of candidate target boxes, including true positives and false positives, by setting a low confidence threshold below the standard detection threshold. Using each candidate box in the preliminary candidate target box set, the corresponding image region, and the preset semantic prompt words for querying the target category as input, the multimodal large model is invoked for inference; Receive the semantic judgment result of the multimodal large model on whether the target in the candidate box belongs to the preset category; If the judgment result is yes, then the candidate box is retained as part of the final recognition result; If the judgment result is negative, the candidate box is removed, and the cleaned high-confidence target recognition result is output as the final recognition result.

[0012] Based on the same inventive concept, a second aspect of the exemplary embodiments of this disclosure provides a target identification device, the device comprising: The training set construction module is configured to acquire an original image dataset containing the target to be identified, divide the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences, and perform image content hash comparison between the general training set and each expert sub-training set to detect the data leakage rate of each expert sub-training set relative to the general training set. The model training module is configured to train a general object detection model based on the general training set, and to train a corresponding expert object detection model based on each expert sub-training set, wherein when training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set. The initial fusion module is configured to acquire a validation set, evaluate the trained general object detection model and all expert object detection models based on the validation set, obtain the performance index of each model, calculate the generalization gap of each model on the training set and the validation set, calculate adaptive fusion weights for each model based on the performance index, generalization gap and corresponding data leakage rate, and perform weighted fusion of the network parameters of the general object detection model and all expert object detection models based on the adaptive fusion weights to obtain the initial fusion model. The final fusion module is configured to fine-tune the initial fusion model with a low learning rate based on the general training set to obtain the final fusion model; The recognition module is configured to input the image to be recognized into the final fusion model, generate preliminary candidate target boxes, input the image regions corresponding to the candidate target boxes into the multimodal large model, and eliminate false positive target boxes that do not conform to the semantic description through semantic consistency discrimination to obtain the final recognition result.

[0013] Based on the same inventive concept, a third aspect of the exemplary embodiments of this disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect.

[0014] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of this disclosure provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0015] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of this disclosure provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method as described in the first aspect.

[0016] The technical solution provided in this disclosure has the following advantages compared with the prior art: By employing a hierarchical architecture that combines a general sub-model with multiple expert sub-models, the model targets the differences that exist among similar targets. After adaptive weight fusion and unified fine-tuning, the recall rate is significantly improved and the false negative rate is significantly reduced in scenarios with large intra-class differences, achieving high-precision detection and stable output of heterogeneous subclusters. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic flowchart of a target recognition method provided for an exemplary embodiment of this disclosure; Figure 2 A schematic flowchart illustrating a training method for an expert object detection model provided as an exemplary embodiment of this disclosure; Figure 3 A schematic diagram of a large model re-evaluation process provided for an exemplary embodiment of this disclosure; Figure 4 Another flowchart illustrating a target recognition method provided as an exemplary embodiment of this disclosure; Figure 5 A schematic diagram of a target recognition device provided as an exemplary embodiment of the present disclosure; Figure 6 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0021] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0022] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0024] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0031] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0032] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0033] The inventors of this disclosure have discovered that, when faced with complex scenarios characterized by "large intra-class differences and blurred inter-class boundaries," the existing technology still has the following shortcomings: A single general detection model (such as YOLO) needs to cover multiple morphological, textural, or pose sub-distributions within the same category using a single parameter space. This trade-off leads to a decrease in the model's feature representation ability on specific subclusters, manifesting as unstable bounding boxes, fluctuating confidence levels, and a tendency to produce false positives and false negatives in extreme scales or dense scenes.

[0034] To address the aforementioned issues, this disclosure employs a hierarchical architecture combining a general sub-model and multiple expert sub-models to model the differences among similar targets. After adaptive weight fusion and unified fine-tuning, the model significantly improves recall and reduces false negative rate in scenarios with significant intra-class differences, achieving high-precision detection and stable output of heterogeneous subclusters.

[0035] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0036] To address the aforementioned problems, this disclosure provides a target recognition method, which will be described below with reference to specific embodiments.

[0037] Figure 1 This is a flowchart illustrating a target recognition method provided in an embodiment of the present disclosure. The method can be executed by a target recognition device, which can be implemented using software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method includes: Step 110: Obtain the original image dataset containing the target to be identified, divide the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences, and perform image content hash comparison between the general training set and each expert sub-training set to detect the data leakage rate of each expert sub-training set relative to the general training set.

[0038] The process begins by acquiring the original image dataset containing the target to be identified. Since similar targets may exhibit significant differences in morphology, texture, scale, pose, or lighting conditions, to improve the model's adaptability to these intra-class differences, this disclosure divides the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences. The general training set covers the overall distribution of all samples and is used to train a general model capable of learning common target features. Each expert sub-training set corresponds to a sub-sample cluster with similar feature patterns and is used to train an expert model capable of focusing on specific fine-grained features.

[0039] After data partitioning, to prevent inflated model performance and decreased generalization ability due to sample overlap between datasets, this disclosure further performs image content hashing comparison between the general training set and each expert sub-training set. Specifically, an image hashing algorithm (such as MD5 or pHash (perceptual hashing)) is used to extract the digital fingerprint of each image. By cross-comparing the image hash values ​​of the general training set and each expert sub-training set, duplicate or similar samples existing in both datasets are identified. Based on the comparison results, the data leakage rate of each expert sub-training set relative to the general training set is calculated according to the following formula: Data leakage rate = |General training set ∩ Expert sub-training set| / |Expert sub-training set| Wherein, |General Training Set ∩ Expert Sub-Training Set| represents the number of duplicate samples between the general training set and a specific expert sub-training set, and |Expert Sub-Training Set| represents the total number of samples in that expert sub-training set. This data leakage rate metric is used to quantify the degree to which the expert sub-training set is contaminated by the general training set, providing important data quality information for adjusting the expert model's training strategy and calculating adaptive fusion weights in subsequent steps. When the data leakage rate exceeds a preset threshold, the system will automatically adopt a conservative training strategy for the expert model corresponding to that expert sub-training set to suppress overfitting risk and ensure the true generalization performance of the fusion model.

[0040] Step 120: Train a general object detection model based on the general training set, and train a corresponding expert object detection model for each expert sub-training set. When training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set.

[0041] First, a general object detection model is trained based on a universal training set. This universal training set covers the overall distribution of all samples and aims to enable the general object detection model to learn the common visual features of various objects, providing the entire system with basic detection capabilities and stable feature representations. This universal model serves as the foundation for subsequent expert model initialization, ensuring that each expert model can further focus on fine-grained differences based on the universal features.

[0042] For each expert sub-training set, this disclosure trains a corresponding expert object detection model. Each expert sub-training set corresponds to a sample sub-cluster with similar feature patterns, such as a subset of objects under specific morphological, textural, or lighting conditions. During training, each expert object detection model initializes its parameters with the weights of a general object detection model, thereby inheriting the general feature representation. Based on this, it is fine-tuned using its corresponding expert sub-training set to enhance its sensitivity and recognition ability for fine-grained features of specific sub-clusters.

[0043] In some exemplary embodiments, when training the expert object detection model, adjusting the training strategy based on the data leakage rate corresponding to the expert sub-training set includes: If the data leakage rate is lower than a preset threshold, then a standard training strategy is used for training. If the data leakage rate is higher than a preset threshold, a conservative training strategy is adopted for training. The conservative training strategy reduces the learning rate and / or reduces the number of training rounds compared to the standard training strategy.

[0044] In the process of training the expert target detection model, this disclosure dynamically adjusts the training strategy based on the data leakage rate corresponding to the expert sub-training set.

[0045] This metric reflects the degree of sample overlap between the expert sub-training set and the general training set. When the data leakage rate is below a preset threshold, it indicates that the samples in the expert sub-training set have good sample independence. In this case, a standard training strategy is adopted, i.e., sufficient training is performed with a normal learning rate and training epochs. When the data leakage rate is above the preset threshold, it indicates that there are many samples in the expert sub-training set that overlap with the general training set. If the standard strategy is followed, the model may overfit to the duplicate samples, resulting in inflated performance and impaired generalization ability. Therefore, this disclosure triggers a conservative training strategy, specifically including reducing the initial learning rate and / or reducing the number of training epochs, to suppress the risk of overfitting and ensure that the features learned by the expert model have real gains rather than being memory of overlapping samples. Through this differentiated training strategy adjustment based on the data leakage rate, this disclosure ensures the independence and effectiveness of each expert model from the training source, laying a high-quality, high-generalization model foundation for the subsequent adaptive fusion step.

[0046] In some exemplary embodiments, training a corresponding expert object detection model based on each expert sub-training set includes: The model is conservatively trained using the first learning rate and the first image resolution, and the first-stage model is output. Using the first-stage model, a second image resolution higher than the first image resolution, and / or a higher cross-union ratio threshold as input, progressive enhancement training is performed to output the second-stage model; Using the second-stage model, a third image resolution higher than the second image resolution, and increased regularization intensity as input, fine-tuning training is performed to finally output an expert object detection model.

[0047] refer to Figure 2 For expert sub-training sets containing extremely difficult examples, this disclosure further employs a phased progressive training strategy to enhance the model's adaptability to difficult examples. This strategy includes: a first stage of conservative training with a low learning rate and low image resolution to stabilize basic features; a second stage of progressive enhancement training with higher image resolution and / or a stricter intersection-union (IU) threshold, loading the optimal model weights obtained in the first stage to strengthen mid-level semantic representation; and a third stage of fine-tuning training with higher image resolution and increased regularization intensity, ultimately outputting an expert object detection model optimized for the difficult example set.

[0048] Step 130: Obtain a validation set. Evaluate the trained general object detection model and all expert object detection models based on the validation set to obtain the performance index of each model. Calculate the generalization gap of each model between the training set and the validation set. Based on the performance index, generalization gap, and corresponding data leakage rate of each model, calculate adaptive fusion weights for each model. Based on the adaptive fusion weights, perform weighted fusion of the network parameters of the general object detection model and all expert object detection models to obtain an initial fusion model.

[0049] In some exemplary embodiments, the step of evaluating the trained general object detection model and all expert object detection models based on the validation set to obtain the performance metric for each model and calculating the generalization gap of each model on the training set and validation set includes: Under the combination of multiple preset confidence thresholds and multiple intersection-union ratio thresholds, the performance index of each model is calculated respectively, wherein the performance index includes precision, recall and the harmonic mean of precision and recall; The generalization gap is the difference between the harmonic mean of precision and recall of the model on the training set and the validation set.

[0050] First, a validation set, independent of the training set, is obtained to objectively evaluate the real performance of the trained general object detection model and all expert object detection models. To comprehensively measure the performance of each model under different levels of stringency, this disclosure employs a multi-threshold grid evaluation strategy. Specifically, performance metrics for each model are calculated separately under a combination of preset confidence thresholds and intersection-union (IU) thresholds. Performance metrics include precision, recall, and the harmonic mean of precision and recall, i.e., the F1 score. This gridded evaluation allows the optimal F1 score for each model to be selected as the performance benchmark for subsequent weight calculations.

[0051] Building upon this, this disclosure further calculates the generalization gap for each model on the training and validation sets. The generalization gap is defined as the difference between the F1 score of the model on the training set and the F1 score on the validation set. This metric reflects the difference between how well the model fits the training data and its ability to generalize on unseen data. A larger generalization gap indicates that the model is more likely to overfit and its performance stability in real-world scenarios is worse.

[0052] In some exemplary embodiments, the step of calculating adaptive fusion weights for each model based on its performance metrics, generalization gap, and corresponding data leakage rate includes: The performance metrics, generalization gap, and data leakage rate are jointly calculated using a preset weighting function, and adaptive fusion weights are output for each model. In the weighting function, the greater the generalization gap or the higher the data leakage rate, the smaller the dynamic fusion weight of the corresponding model.

[0053] This disclosure calculates adaptive fusion weights for each model based on its performance metrics, generalization gap, and the data leakage rate of the expert sub-training set relative to the general training set. The adaptive fusion weights are jointly calculated using a pre-defined weighting function. This function is designed so that the higher the model's detection performance, the greater its weight contribution; conversely, the greater the generalization gap or the higher the data leakage rate, the greater the penalty applied to the model's weights, thus suppressing the negative impacts of overfitting or data contamination. Specifically, one implementation of the weighting function can be expressed as a joint weighting formula based on performance metrics and generalization gap:

[0054] in, w i For the adaptive fusion weights of the i-th model, F 1 i This represents the optimal F1 score for the model. P i The accuracy of this model,R i The recall rate of this model. G i This is due to the model's generalization gap (poor performance on the training and validation sets). L i This represents the data leakage rate (the ratio of the intersection of the general training set and the expert training set) corresponding to this model. α and β The preset penalty coefficient (used to control the penalty intensity for generalization gap and data leakage rate).

[0055] The formula for the generalization gap is:

[0056] G For generalization gap, F 1 train The F1 score of the model on the training set. F 1 val This is the F1 score of the model on the validation set.

[0057] The formula for the data breach rate is:

[0058] L For data breach rate, H general For a general training set, H expert For expert training.

[0059] After calculating the original weights of all models, further processing is performed on all... w i Normalization is performed so that the sum of the weights of each model is 1, which facilitates the subsequent weighted fusion operation.

[0060] Finally, based on the normalized adaptive fusion weights, the network parameters of the general object detection model and all expert object detection models are weighted and fused. Specifically, for each layer of network parameters in the model, a weighted average is performed according to the corresponding weights to obtain a new parameter tensor, thereby generating an initial fusion model that integrates general features and various expert features. This initial fusion model integrates the advantages of each model in its respective domain, while suppressing the contribution of unreliable models through a penalty mechanism for generalization gap and data leakage rate, providing a high-performance starting point for subsequent fine-tuning and calibration.

[0061] Step 140: Based on the general training set, fine-tune the initial fusion model with a low learning rate to obtain the final fusion model.

[0062] Among them, the initial fusion model is fine-tuned with a low learning rate based on a general training set to eliminate parameter drift introduced during the model fusion process, so that the parameter space after fusion can reconverge under a unified optimization objective, and finally obtain a final fusion model with better generalization performance and more consistent feature representation.

[0063] Specifically, after obtaining the initial fusion model through adaptive weight fusion, although this model integrates the advantages of a general model and multiple expert models at the parameter level, there are inherent biases in their parameter spaces because each model was originally trained independently under different data distributions and optimization paths. Directly performing weighted averaging will disrupt the synergistic effect of features at each layer, manifesting as inconsistencies in feature responses and numerical drift, a phenomenon known as model drift. Without calibration, the initial fusion model may not perform as expected in actual deployment.

[0064] Therefore, this disclosure employs a general training set to fine-tune the initial fusion model using a low learning rate. A low learning rate refers to an initial learning rate significantly lower than that used in the preceding steps for training the general and expert models, typically set to 1×10⁻⁶. -5 Up to 5×10 -5 The magnitude is significant. Such a conservative learning rate is adopted to allow for fine-tuning of the fused parameters without drastically altering the model parameters, enabling feature representations from different sources to gradually align and reconverge within a shared optimization space.

[0065] Through several rounds of low-learning-rate iterative optimization, the model parameters gradually eliminate cross-model feature mismatch while retaining the original fusion advantages, thus re-establishing a balanced state of collaborative work among the feature extractor, classification head, and regression head.

[0066] The final fusion model, obtained after low-learning-rate fine-tuning training, has a uniformly calibrated parameter space. This retains the broad coverage of the general model and the fine-grained sensitivity of each expert model, while eliminating internal conflicts caused by direct parameter averaging. Experiments show that after this fine-tuning and calibration step, the model's average F1 score on the non-training distribution can be improved by 8% to 12%, significantly enhancing its generalization performance and providing a stable and high-precision model foundation for the subsequent inference and recognition stages.

[0067] Step 150: Input the image to be identified into the final fusion model to generate preliminary candidate target boxes. Input the image regions corresponding to the candidate target boxes into the multimodal large model. Through semantic consistency discrimination, eliminate false positive target boxes that do not conform to the semantic description to obtain the final recognition result.

[0068] In some exemplary embodiments, the step of inputting the image to be identified into the final fusion model to generate preliminary candidate bounding boxes, inputting the image regions corresponding to the candidate bounding boxes into a multimodal large model, and eliminating false positive bounding boxes that do not conform to the semantic description through semantic consistency discrimination to obtain the final recognition result includes: Using the image to be identified as input, the final fusion model is used to generate a preliminary set of candidate target boxes, including true positives and false positives, by setting a low confidence threshold below the standard detection threshold. Using each candidate box in the preliminary candidate target box set, the corresponding image region, and the preset semantic prompt words for querying the target category as input, the multimodal large model is invoked for inference; Receive the semantic judgment result of the multimodal large model on whether the target in the candidate box belongs to the preset category; If the judgment result is yes, then the candidate box is retained as part of the final recognition result; If the judgment result is negative, the candidate box is removed, and the cleaned high-confidence target recognition result is output as the final recognition result.

[0069] refer to Figure 3 In this process, the image to be recognized is input into the trained final fusion model, which first generates preliminary candidate bounding boxes. During the actual inference phase, to recall as many true targets as possible and avoid missed detections due to overly conservative confidence assessments by the fusion model itself, this disclosure uses a low confidence threshold (e.g., set to 0.05 to 0.1, significantly lower than the 0.5 threshold commonly used in conventional detection tasks) to filter the output of the final fusion model. With this low threshold setting, the final fusion model outputs a preliminary set of candidate bounding boxes containing a large number of candidate boxes. This set includes both correctly detected true positive bounding boxes and a large number of false positive bounding boxes introduced by the lower threshold. The core idea is: in the extremely fast, lightweight detection phase, priority is given to ensuring no missed detections (high recall), while the task of high precision (high accuracy) is left to the subsequent multimodal large model.

[0070] Subsequently, the image region corresponding to each candidate bounding box in the initial candidate bounding box set is cropped and fed as input to a pre-trained multimodal large model (MLLM). Simultaneously, it is coupled with pre-defined semantic prompts used to query the target category (e.g., "Please determine whether the object marked in the red box in the image is [target category]? Please answer yes or no."). Based on its powerful cross-modal semantic alignment capabilities and open-world knowledge, the multimodal large model performs semantic consistency determination on the image region. The multimodal large model outputs the probability that the candidate region matches the target category description or a binary classification result.

[0071] Finally, based on the semantic consistency discrimination results output by the multimodal large model, the initial candidate target box set is cleaned. Specifically, a re-judgment confidence threshold (e.g., 0.7) is set, and only candidate boxes with semantic consistency scores higher than this threshold are retained as the final, high-confidence recognition results. Candidate boxes with semantic consistency scores lower than the threshold are judged as false positive target boxes that do not conform to the semantic description and are removed. Through this cascade architecture of rapid recall by a lightweight detection model and accurate discrimination by a multimodal large model, this disclosure achieves a significant reduction in the false detection rate (approximately 30% to 40%) without significantly increasing the overall inference latency. The final output recognition results have the dual advantages of high throughput and high reliability.

[0072] In summary, for reference Figure 4 This disclosure selects a suitable basic detection small model (such as YOLO) as needed, trains it separately on a general training set and a well-classified expert training set (classified by intra-class difference distance), obtains the optimal model by generalization monitoring, then fuses the general model and the expert model by a set fusion strategy and fine-tunes the training to infer the result; then, the detection region of the fused model is re-judged by MLLM to remove false detection regions and generate the final output.

[0073] The technical solution provided in this disclosure has the following advantages compared with the prior art: More accurate detection results and stronger adaptability to intra-class differences: This disclosure adopts a hierarchical training architecture of "general sub-model + multi-expert sub-model" to specifically model differences in morphology, texture, reflection, scale, etc. among similar targets. After adaptive weight fusion and unified fine-tuning, the model significantly improves the recall rate and significantly reduces the false negative rate in scenarios with large intra-class differences, achieving high-precision detection and stable output of heterogeneous subclusters.

[0074] Significantly improves model generalization and robustness: Introduces a mechanism of "data leakage protection + generalization gap assessment + adaptive weight penalty" to automatically suppress the problem of inflated performance caused by sample overlap or subset overfitting. After fusing dynamic weight allocation and conservative fine-tuning calibration, the model's average F1 score on non-training distributions improves by 8%–12%, with stable generalization performance and less prone to performance regression after deployment.

[0075] Balancing inference speed and accuracy, throughput is increased rather than decreased: MLLM is used only for candidate re-judgment, rather than for whole-graph inference, significantly reducing inference time compared to end-to-end large models, while also reducing the number of false positives. The lightweight detection model is responsible for rapid candidate generation, while MLLM is responsible for high-level semantic error correction, forming a collaborative structure of "lightweight and efficient + semantically accurate", with overall throughput consistent with real-time detection on the edge.

[0076] Stable training process and strong adaptive capability for difficult examples: For extremely difficult subclasses such as those with weak reflections and subtle textures, a phased progressive training strategy (basic → augmentation → fine-tuning) is adopted, with self-adjustment of learning rate, resolution, and regularization parameters at each stage to achieve stable convergence for difficult examples. Real-world testing shows an F1 score improvement of over 20% on small sample difficult classes, effectively addressing the underfitting problem of traditional single-strategy training for extreme subclusters.

[0077] Data security and consistency assurance: Image content hashing (MD5 / pHash) detection is used to automatically detect leaks between the general set and the expert set, preventing training set contamination caused by sample reuse. This mechanism ensures that the data sources of the general model and the expert model are independent, and that the training results are traceable and reproducible, providing consistency assurance for subsequent model fusion and performance verification.

[0078] Scalable, replaceable, and easy to implement in engineering: The entire framework is compatible with mainstream detection architectures such as the YOLO series and supports multiple fusion strategies (weight fusion, output fusion, feature concatenation). The model training, fusion, and re-judgment modules are all implemented using standard Python / PyTorch, which can be quickly migrated to GPU / edge device environments and deployed without modifying the underlying inference engine.

[0079] Multimodal collaboration significantly reduces false detections: After the candidate boxes output by the fusion sub-model are semantically re-judged by MLLM, false positive samples that do not conform to semantic features can be automatically eliminated. In complex backgrounds and occluded scenarios, the false detection rate is greatly reduced, and the final output results are more consistent with the business semantics and human judgment standards.

[0080] Automated and traceable training and evaluation: This publication integrates log and metric storage mechanisms to record training hyperparameters, evaluation grids, fusion weights, generalization metrics, and other information throughout the entire process, forming an auditable performance tracing chain. The entire process of model fusion, retraining, and evaluation can be executed automatically, resulting in low engineering deployment costs and high efficiency in repeating experiments.

[0081] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0082] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] To achieve the above embodiments, this disclosure also proposes a target recognition device.

[0084] Figure 5 This is a schematic diagram of a target recognition device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is generally integrated into an electronic device. Figure 5 As shown, the device includes: The training set construction module 210 is configured to acquire an original image dataset containing the target to be identified, divide the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences, and perform image content hash comparison between the general training set and each expert sub-training set to detect the data leakage rate of each expert sub-training set relative to the general training set. The model training module 220 is configured to train a general object detection model based on the general training set, and to train a corresponding expert object detection model based on each expert sub-training set, wherein when training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set. The initial fusion module 230 is configured to acquire a validation set, evaluate the trained general object detection model and all expert object detection models based on the validation set, obtain the performance index of each model, calculate the generalization gap of each model on the training set and the validation set, calculate adaptive fusion weights for each model based on the performance index, generalization gap and corresponding data leakage rate, and perform weighted fusion of the network parameters of the general object detection model and all expert object detection models based on the adaptive fusion weights to obtain the initial fusion model. The final fusion module 240 is configured to perform low-learning-rate fine-tuning training on the initial fusion model based on the general training set to obtain the final fusion model; The recognition module 250 is configured to input the image to be recognized into the final fusion model to generate preliminary candidate target boxes, input the image regions corresponding to the candidate target boxes into the multimodal large model, and eliminate false positive target boxes that do not conform to the semantic description through semantic consistency discrimination to obtain the final recognition result.

[0085] In some exemplary embodiments, the model training module 220 is specifically configured as follows: If the data leakage rate is lower than a preset threshold, then a standard training strategy is used for training. If the data leakage rate is higher than a preset threshold, a conservative training strategy is adopted for training. The conservative training strategy reduces the learning rate and / or reduces the number of training rounds compared to the standard training strategy.

[0086] In some exemplary embodiments, the model training module 220 is specifically configured as follows: The model is conservatively trained using the first learning rate and the first image resolution, and the first-stage model is output. Using the first-stage model, a second image resolution higher than the first image resolution, and / or a higher cross-union ratio threshold as input, progressive enhancement training is performed to output the second-stage model; Using the second-stage model, a third image resolution higher than the second image resolution, and increased regularization intensity as input, fine-tuning training is performed to finally output an expert object detection model.

[0087] In some exemplary embodiments, the initial fusion module 230 is specifically configured as follows: Under the combination of multiple preset confidence thresholds and multiple intersection-union ratio thresholds, the performance index of each model is calculated respectively, wherein the performance index includes precision, recall and the harmonic mean of precision and recall; The generalization gap is the difference between the harmonic mean of precision and recall of the model on the training set and the validation set.

[0088] In some exemplary embodiments, the initial fusion module 230 is specifically configured as follows: The performance metrics, generalization gap, and data leakage rate are jointly calculated using a preset weighting function, and adaptive fusion weights are output for each model. In the weighting function, the greater the generalization gap or the higher the data leakage rate, the smaller the dynamic fusion weight of the corresponding model.

[0089] In some exemplary embodiments, the identification module 250 is specifically configured as follows: Using the image to be identified as input, the final fusion model is used to generate a preliminary set of candidate target boxes, including true positives and false positives, by setting a low confidence threshold below the standard detection threshold. Using each candidate box in the preliminary candidate target box set, the corresponding image region, and the preset semantic prompt words for querying the target category as input, the multimodal large model is invoked for inference; Receive the semantic judgment result of the multimodal large model on whether the target in the candidate box belongs to the preset category; If the judgment result is yes, then the candidate box is retained as part of the final recognition result; If the judgment result is negative, the candidate box is removed, and the cleaned high-confidence target recognition result is output as the final recognition result.

[0090] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0091] The target recognition device provided in this disclosure can execute the target recognition method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method execution.

[0092] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program / instructions, which, when executed by a processor, implements the target recognition method described above.

[0093] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0094] The following is a detailed reference. Figure 6 The diagram illustrates a structural schematic suitable for implementing the electronic device 300 in the embodiments of this disclosure. The electronic device 300 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0095] like Figure 6 As shown, the electronic device 300 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a memory 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processor 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0096] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0097] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a memory 308, or installed from a ROM 302. When the computer program is executed by the processor 301, it performs the functions defined in the target recognition method of embodiments of this disclosure.

[0098] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0099] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0100] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0101] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned target recognition method.

[0102] Electronic devices can be programmed with computer program code in one or more programming languages ​​or a combination thereof to perform the operations of this disclosure. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0105] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0106] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0107] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0108] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0109] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A target recognition method, characterized in that, The method includes: Obtain the original image dataset containing the target to be identified, divide the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences, and perform image content hash comparison between the general training set and each expert sub-training set to detect the data leakage rate of each expert sub-training set relative to the general training set. A general object detection model is trained based on the general training set. For each expert sub-training set, a corresponding expert object detection model is trained based on the expert sub-training set. When training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set. Obtain a validation set, evaluate the trained general object detection model and all expert object detection models based on the validation set, obtain the performance index of each model, and calculate the generalization gap of each model on the training set and validation set. Based on the performance index, generalization gap and corresponding data leakage rate of each model, calculate adaptive fusion weights for each model. Based on the adaptive fusion weights, perform weighted fusion of the network parameters of the general object detection model and all expert object detection models to obtain an initial fusion model. The initial fusion model is fine-tuned with a low learning rate based on the general training set to obtain the final fusion model. The image to be identified is input into the final fusion model to generate preliminary candidate target boxes. The image regions corresponding to the candidate target boxes are then input into the multimodal large model. Through semantic consistency discrimination, false positive target boxes that do not conform to the semantic description are eliminated to obtain the final recognition result.

2. The method according to claim 1, characterized in that, When training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set, including: If the data leakage rate is lower than a preset threshold, then a standard training strategy is used for training. If the data leakage rate is higher than a preset threshold, a conservative training strategy is adopted for training. The conservative training strategy reduces the learning rate and / or reduces the number of training rounds compared to the standard training strategy.

3. The method according to claim 1, characterized in that, The step of training a corresponding expert object detection model based on each expert sub-training set includes: The model is conservatively trained using the first learning rate and the first image resolution, and the first-stage model is output. Using the first-stage model, a second image resolution higher than the first image resolution, and / or a higher cross-union ratio threshold as input, progressive enhancement training is performed to output the second-stage model; Using the second-stage model, a third image resolution higher than the second image resolution, and increased regularization intensity as input, fine-tuning training is performed to finally output an expert object detection model.

4. The method according to claim 1, characterized in that, The process involves evaluating the trained general object detection model and all expert object detection models based on the validation set, obtaining performance metrics for each model, and calculating the generalization gap between each model on the training and validation sets, including: Under the combination of multiple preset confidence thresholds and multiple intersection-union ratio thresholds, the performance index of each model is calculated respectively, wherein the performance index includes precision, recall and the harmonic mean of precision and recall; The generalization gap is the difference between the harmonic mean of precision and recall of the model on the training set and the validation set.

5. The method according to claim 1, characterized in that, The adaptive fusion weights for each model are calculated based on its performance metrics, generalization gap, and corresponding data leakage rate, including: The performance metrics, generalization gap, and data leakage rate are jointly calculated using a preset weighting function, and adaptive fusion weights are output for each model. In the weighting function, the greater the generalization gap or the higher the data leakage rate, the smaller the dynamic fusion weight of the corresponding model.

6. The method according to claim 1, characterized in that, The process involves inputting the image to be recognized into the final fusion model to generate preliminary candidate bounding boxes, inputting the image regions corresponding to the candidate bounding boxes into a multimodal large model, and eliminating false positive bounding boxes that do not conform to the semantic description through semantic consistency discrimination to obtain the final recognition result, including: Using the image to be identified as input, the final fusion model is used to generate a preliminary set of candidate target boxes, including true positives and false positives, by setting a low confidence threshold below the standard detection threshold. Using each candidate box in the preliminary candidate target box set, the corresponding image region, and the preset semantic prompt words for querying the target category as input, the multimodal large model is invoked for inference; Receive the semantic judgment result of the multimodal large model on whether the target in the candidate box belongs to the preset category; If the judgment result is yes, then the candidate box is retained as part of the final recognition result; If the judgment result is negative, the candidate box is removed, and the cleaned high-confidence target recognition result is output as the final recognition result.

7. A target recognition device, characterized in that, The device includes: The training set construction module is configured to acquire an original image dataset containing the target to be identified, divide the original image dataset into a general training set and multiple expert sub-training sets based on intra-class feature differences, and perform image content hash comparison between the general training set and each expert sub-training set to detect the data leakage rate of each expert sub-training set relative to the general training set. The model training module is configured to train a general object detection model based on the general training set, and to train a corresponding expert object detection model based on each expert sub-training set, wherein when training the expert object detection model, the training strategy is adjusted based on the data leakage rate corresponding to the expert sub-training set. The initial fusion module is configured to acquire a validation set, evaluate the trained general object detection model and all expert object detection models based on the validation set, obtain the performance index of each model, calculate the generalization gap of each model on the training set and the validation set, calculate adaptive fusion weights for each model based on the performance index, generalization gap and corresponding data leakage rate, and perform weighted fusion of the network parameters of the general object detection model and all expert object detection models based on the adaptive fusion weights to obtain the initial fusion model. The final fusion module is configured to fine-tune the initial fusion model with a low learning rate based on the general training set to obtain the final fusion model; The recognition module is configured to input the image to be recognized into the final fusion model, generate preliminary candidate target boxes, input the image regions corresponding to the candidate target boxes into the multimodal large model, and eliminate false positive target boxes that do not conform to the semantic description through semantic consistency discrimination to obtain the final recognition result.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program / instruction thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Images