Identification method, device and equipment of intra-class difference target and storage medium

Abstract

The embodiment of the invention provides an intra-class difference target recognition method and device, equipment and a storage medium, and relates to the technical field of target recognition, and the method comprises the steps: constructing a difference training sample subset based on corresponding difference sample data for a difference class in each class of a recognized target; training a basic recognition model of a recognition target by using the general training sample set, and training a category recognition sub-model of an intra-class difference category by using the difference training sample subset; determining a sub-weight based on the model performance of the category identification sub-model; fusing the plurality of category identification sub-models with the basic identification model based on the sub-weights to obtain a target identification model; and identifying the identification target based on the target identification model to obtain a target identification result. According to the invention, the detection accuracy can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of target recognition technology, and in particular to a method, apparatus, device and storage medium for identifying intra-class differentiated targets. Background Technology

[0002] Object detection is one of the core tasks in the field of computer vision. Its core is to simultaneously locate and classify objects in images or video frames. It is widely used in scenarios such as security monitoring, autonomous driving, industrial quality inspection, and medical image analysis.

[0003] In complex real-world scenarios, targets of the same category exhibit significant differences in shape, texture, scale, pose, and working conditions. Current target detection models, when used to detect targets with diverse intra-class features, suffer from high false positive rates and high false negative rates for small or occluded targets due to significant intra-class differences, blurred inter-class boundaries, limited feature representation capabilities, and decreased bounding box prediction stability. This results in low overall detection accuracy. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides a method, apparatus, device and storage medium for identifying intra-class differential targets.

[0005] This disclosure provides a method for identifying intra-class differentiated targets. The method includes: constructing a general training sample set based on sample data of the identified targets; constructing a subset of differentiated training samples based on the corresponding differentiated sample data for each intra-class differentiated category of the identified targets; training a basic identification model for the identified targets using the general training sample set, and training a category identification sub-model for the intra-class differentiated categories using the subset of differentiated training samples; determining sub-weights based on the model performance of the category identification sub-models; fusing multiple category identification sub-models with the basic identification model based on the sub-weights to obtain a target identification model; and identifying the identified targets based on the target identification model to obtain a target identification result.

[0006] In one possible implementation, a category recognition sub-model for intra-class differential categories is trained using a subset of differential training samples, including: setting a low learning rate and a first resolution; performing a first-stage training of the category recognition sub-model using the subset of differential training samples to generate first-stage weights for the category recognition sub-model; setting a second resolution and an intersection-over-union (IoU) threshold; loading the first-stage weights for the category recognition sub-model; performing a second-stage training of the category recognition sub-model using the subset of differential training samples to generate second-stage weights for the category recognition sub-model; setting a third resolution and a regularization strength; loading the second-stage weights for the category recognition sub-model; performing a second-stage training of the category recognition sub-model using the subset of differential training samples to generate third-stage weights for the category recognition sub-model; and loading the third-stage weights for the category recognition sub-model to generate the trained category recognition sub-model. Wherein, the first resolution is less than the second resolution, the second resolution is less than the third resolution, the number of training epochs in the first stage is greater than the number of training epochs in the second stage, and the number of training epochs in the second stage is greater than the number of training epochs in the third stage.

[0007] In one possible implementation, a basic recognition model for identifying the target is trained using a general training sample set, and a category recognition sub-model for intra-class differences is trained using a subset of difference training samples. Prior to this, the method includes: calculating the cross ratio between the general training sample set and the subset of difference training samples using an image content hashing algorithm; and setting the learning rate and training epochs of the category recognition sub-model corresponding to the intra-class differences based on the cross ratio. The higher the cross ratio, the lower the corresponding learning rate and the fewer the training epochs.

[0008] In one possible implementation, determining sub-weights based on the model performance of the category recognition sub-model includes: evaluating the performance of the category recognition sub-model to obtain precision, recall, and a comprehensive index value; calculating the generalization gap value of the category recognition sub-model using the comprehensive index value; calculating the data leakage rate using the intersection ratio of the differential training sample subset and the general training sample set; and determining the sub-weights based on the model performance calculated using precision, recall, comprehensive index value, generalization gap value, and data leakage rate.

[0009] In one possible implementation, the sub-weights are determined based on model performance metrics such as precision, recall, composite index, generalization gap, and data leakage rate. This includes: for each category identification sub-model, calculating the average of precision, recall, and composite index, correcting the average using the generalization gap and data leakage rate to obtain the corresponding model performance index; calculating the ratio of the model performance index to the sum of the indices to obtain the sub-weights; wherein the sum of indices is the sum of the model performance indices of all category identification sub-models.

[0010] In one possible implementation, multiple category recognition sub-models are fused with the basic recognition model based on sub-weights to obtain a target recognition model. This process also includes fine-tuning the target recognition model based on sample data of the target.

[0011] In one possible implementation, the target is identified based on the target recognition model to obtain the target recognition result, including: setting a low detection threshold, using the target recognition model to identify the target and obtaining multiple target recognition boxes; using a multimodal large model to perform secondary target judgment on the target recognition boxes to determine whether the target recognition boxes contain the target; and when the target recognition boxes contain the target, the target recognition boxes are output.

[0012] This disclosure also provides an apparatus for identifying intra-class differential targets. The apparatus includes: a construction module for constructing a general training sample set based on sample data of the identified targets, and for each intra-class differential category of the identified targets, constructing a differential training sample subset based on the corresponding differential sample data; a training module for training a basic identification model of the identified targets using the general training sample set, and training a category identification sub-model of the intra-class differential categories using the differential training sample subset; a determination module for determining sub-weights based on the model performance of the category identification sub-models; a fusion module for fusing multiple category identification sub-models with the basic identification model based on the sub-weights to obtain a target identification model; and an identification module for identifying the identified targets based on the target identification model to obtain a target identification result.

[0013] This disclosure also provides a computing device, which includes: a processor; a memory for storing processor-executable instructions; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the intra-class difference target identification method provided in this disclosure.

[0014] This disclosure also provides a computer-readable storage medium storing a computer program for performing the intra-class differential target identification method provided in this disclosure.

[0015] This application employs a method combining a basic recognition model and multiple category recognition sub-models. The basic recognition model learns global features for identifying intra-category differential targets, while the multiple category recognition sub-models learn the differential features for identifying intra-category differential targets, achieving targeted modeling and reducing the false negative and false positive rates for intra-category differential targets. Furthermore, weights are assigned to the category recognition sub-models based on model performance, enabling the final target recognition model to improve its ability to represent differential features, reduce the false negative rate, and further enhance the detection accuracy and generalization ability for intra-category differential targets. Attached Figure Description

[0016] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0017] Figure 1 A flowchart illustrating a method for identifying intra-class differential targets provided in an embodiment of this disclosure; Figure 2 A schematic diagram illustrating a category recognition sub-model training method provided in an embodiment of this disclosure; Figure 3 A schematic diagram of a dual-target recognition method provided in an embodiment of this disclosure; Figure 4 A schematic diagram of the structure of a device for identifying intra-class differential targets provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] Traditional small-scale target detection models have limited feature representation capabilities when faced with diverse features within a class. They also suffer from low detection accuracy, high false positive and false negative rates, and insufficient inference efficiency when there are large differences among targets of the same class.

[0025] To address the aforementioned problems, embodiments of this disclosure provide a method for identifying intra-class differentiated targets. The method includes: constructing a general training sample set based on sample data of the identified targets; for each intra-class differentiated category of the identified targets, constructing a subset of differentiated training samples based on the corresponding differentiated sample data; training a basic identification model for the identified targets using the general training sample set, and training a category identification sub-model for the intra-class differentiated categories using the subset of differentiated training samples; determining sub-weights based on the model performance of the category identification sub-models; fusing multiple category identification sub-models with the basic identification model based on the sub-weights to obtain a target identification model; and identifying the identified targets based on the target identification model to obtain the target identification result. This application employs a method combining a basic identification model and multiple category identification sub-models. The basic identification model learns the global features for identifying intra-class differentiated targets, while the multiple category identification sub-models learn the differentiated features for identifying intra-class differentiated targets respectively, achieving targeted modeling and reducing the false negative and false positive rates for intra-class differentiated targets. Furthermore, by allocating weights to the category recognition sub-model based on model performance, the final target recognition model can improve its ability to represent differential features, reduce the false negative rate, and further improve the detection accuracy and generalization ability of intra-class differential targets.

[0026] The method will be described below with reference to specific embodiments.

[0027] Figure 1 This is a flowchart illustrating a method for identifying intra-class differentiated targets according to an embodiment of this disclosure. The method can be executed by an intra-class differentiated target identification device, which can be implemented in software and / or hardware, and is generally integrated into a computing device. Figure 1 As shown, the method includes: S101. Construct a general training sample set based on the sample data of the identified target. For each category of difference in the identified target, construct a difference training sample subset based on the corresponding difference sample data.

[0028] For the target to be detected, general sample data covering various scenarios is collected. The general sample data includes target images with various shapes, textures, lighting and reflection intensities, and a general training sample set is constructed using all target images.

[0029] For example, the target is identified as oil stains, and the sample data covers oil stain images of different carrier surfaces, common concentrations, and basic morphologies. A general training sample set is constructed using the oil stain images.

[0030] To address the differences in the target under different scenarios, multiple dimensions are divided according to the difference category. Difference sample data corresponding to each difference feature is collected. The difference sample data includes the target image corresponding to each shape, texture, illumination and reflection intensity. A corresponding difference training sample subset is constructed using each target image.

[0031] The number of differential training sample subsets is 1-10, preferably 2-4, which is used to control the number of divisions in the category recognition sub-model.

[0032] For example, the target is identified as oil stains. The difference categories include morphological differences such as dot-shaped seeping oil stains, strip-shaped flowing oil stains, and sheet-like diffused oil stains; texture differences such as dried oil stain textures and wet oil stain textures; light and reflection intensity such as oil stains under strong light, oil stains in backlight, oil stains in low light environments, and oil stains reflected from metal surfaces; and environmental interference differences such as oil stains covered by dust, oil stains with moisture and superimposed on moisture, and oil stains obscured by debris.

[0033] Furthermore, the general training sample set and multiple differential training sample subsets are organized, filtered, and standardized. Details will not be elaborated here.

[0034] In one possible implementation, after constructing a general training sample set and multiple differential training sample subsets, data leakage and duplicate sample detection are performed on the general training sample set and multiple differential training sample subsets.

[0035] Specifically, the hash values ​​of sample images in the general training sample set and the hash values ​​of sample images in each differential training sample subset are calculated using an image content hashing algorithm. The hash values ​​of the general training sample set are then matched with the hash values ​​of each differential training sample subset, and the number of successfully matched samples is recorded, and the crossover ratio is calculated. The formula for calculating the crossover ratio is: Crossover ratio = number of successfully matched samples / total number of samples in the differential training sample subset.

[0036] In some implementations, a hash table index is used to quickly locate candidate matches through a pre-built hash index, reducing the computational cost of a full traversal.

[0037] Furthermore, the learning rate and training epochs of the category recognition sub-model corresponding to the intra-class difference category are set based on the crossover ratio. Specifically, the higher the crossover ratio, the lower the learning rate and the fewer the training epochs for the corresponding category recognition sub-model.

[0038] Specifically, a crossover ratio threshold is preset. When the crossover ratio of a subset of differential training samples is greater than the threshold, it is determined that there is a high proportion of data overlap and data leakage. In this case, the learning rate and training rounds of the category recognition sub-model are reduced.

[0039] For example, the cross ratio threshold is 20%. When the cross ratio of the differential training sample subset is greater than 20%, the initial learning rate of the category recognition sub-model will be reduced by 50%-80%. The higher the cross ratio, the greater the reduction. The number of training rounds will be reduced by 30%-50%. The higher the cross ratio, the more rounds will be reduced.

[0040] This implementation reduces the learning intensity of the model on contaminated data by adjusting parameters linked to the cross-ratio, avoiding inflated generalization due to data leakage and ensuring the effectiveness and reliability of subsequent model training. Furthermore, this method ensures that the data sources of the base recognition model and the category recognition sub-model are independent, and the training results are traceable and reproducible, providing consistency assurance for subsequent model fusion and performance verification.

[0041] S102. Use a general training sample set to train a basic recognition model for identifying targets, and use a subset of differential training samples to train a category recognition sub-model for intra-class differential categories.

[0042] A basic recognition model for identifying targets is trained using a general training sample set to obtain general feature representations of the targets. A category recognition sub-model for intra-class differences is trained using a subset of difference training samples to enhance the ability to represent specific features of the targets.

[0043] In one possible implementation, the YOLO model is used as the base model, and a base recognition model and a category recognition sub-model are trained separately.

[0044] Understandably, the differential training sample subset used to train the category recognition sub-model contains difficult examples that the basic recognition model struggles to identify. Therefore, this application employs a three-stage progressive optimization training strategy for the category recognition sub-model. For details, please refer to... Figure 2 , Figure 2 This is a schematic diagram illustrating a category recognition sub-model training method provided in an embodiment of this disclosure. Figure 2 As shown, the method includes: The first stage employs conservative basic training. Specifically, the first stage sets a low learning rate, a small initial resolution, and a high number of training epochs. It uses a subset of differential training samples to train the category recognition sub-model in the first stage and performs generalization monitoring, generating and saving the first-stage weights of the category recognition sub-model.

[0045] The first stage stabilizes the learning of basic features by the category recognition sub-model, avoiding unstable model convergence due to excessively high initial training parameters or excessively large input resolution, thus laying a solid feature foundation for subsequent training.

[0046] The second stage employs progressive enhancement training. Specifically, based on the generalization monitoring results of the first stage, the image resolution is increased, the weight decay is accelerated, and an intersection-over-union (IoU) threshold, a higher second resolution, and a lower number of training epochs are set. The first-stage weights of the category recognition sub-model are loaded, and the category recognition sub-model is trained in the second stage using a subset of differential training samples, with generalization monitoring performed. The second-stage weights of the category recognition sub-model in the second stage are then generated and saved.

[0047] In one possible implementation, a multi-confidence and multi-cross-ratio grid search method is used to determine the cross-ratio threshold. Specifically, the grid combinations formed by the preset confidence interval and the cross-ratio threshold interval are traversed, the performance of each grid combination is verified, and the optimal threshold combination is selected for each subset of differential training samples to generate the cross-ratio threshold.

[0048] The second stage enhances the category recognition sub-model's learning of the semantic features of the target, laying a semantic foundation for subsequent fine-tuning. Examples include the distribution patterns of specific oil stains and texture hierarchy features.

[0049] The third stage employs refined optimization training. Specifically, based on the generalization monitoring results of the second stage, the image resolution is further increased, the weight decay is intensified, and the regularization intensity, high third resolution, and low training epochs are set. The second-stage weights of the category recognition sub-model are loaded, and the category recognition sub-model is trained in the third stage using a subset of differential training samples, while generalization monitoring is performed to generate the third-stage weights of the category recognition sub-model.

[0050] The third stage employs strict regularization techniques such as Dropout and L2 regularization to prevent overfitting, focusing on fine optimization of the boundary contour detection of the target and the recognition of reflective targets in special scenarios. Ultimately, this enables the category recognition sub-model to achieve higher feature sensitivity and accurate recognition in specific difference scenarios.

[0051] Furthermore, the third-stage weights of the category recognition sub-model are loaded to generate the trained category recognition sub-model.

[0052] For example, the first phase of training rounds accounts for 40% of the total training rounds, the second phase of training rounds accounts for 35% of the total training rounds, and the third phase of training rounds accounts for 25% of the total training rounds.

[0053] For example, the initial learning rate for the base recognition model and the category recognition sub-model is 1×10. -5 -5×10 -4 The resolution is 640-1536 pixels, and the weight decay is 5×10. -4 -1×10 -3 The confidence level range is 0.001-0.05, and the cross ratio threshold range is 0.4-0.65.

[0054] In this implementation, for extremely difficult subclasses such as weak reflection and fine texture, a three-stage training method is set up, and the learning rate, resolution and regularization parameters are automatically adjusted in each stage to achieve stable convergence of difficult examples. This avoids the problem of poor model accuracy caused by the inability to adjust parameters for intermediate training results during continuous training, and effectively solves the problem of underfitting extreme subclusters by traditional single-policy training.

[0055] S103. Determine sub-weights based on the model performance of the category recognition sub-model.

[0056] A comprehensive performance evaluation is performed on the trained category recognition sub-model.

[0057] Specifically, the precision, recall, and overall metric value are calculated for each category recognition sub-model. For example, the overall metric value is the overall F1 score.

[0058] The generalization gap value of the category recognition sub-model is calculated using a comprehensive index value. The specific formula for calculating the generalization gap value is as follows:

[0059] In the formula, This is the generalization gap value. The overall F1 score of the training set. The overall F1 score for the validation set.

[0060] The data leakage rate is calculated using the proportion of the intersection between the differential training sample subset and the general training sample set. The specific formula for calculating the data leakage rate is as follows:

[0061] In the formula, For data breach rate, For a general training sample set, This is a subset of the differential training samples.

[0062] Furthermore, sub-weights are determined based on model performance metrics such as precision, recall, overall performance index, generalization gap, and data leakage rate. Specifically, for each category recognition sub-model, the average of precision, recall, and overall performance index is calculated. This average is then corrected using the generalization gap and data leakage rate to obtain the corresponding model performance index. The ratio of the model performance index to the sum of the total indexes is calculated to obtain the sub-weight. The specific formula for calculating the sub-weight is as follows:

[0063] In the formula, For the first The sub-weights are determined by the model performance of the sub-model for each category identification. This is a comprehensive indicator value. For accuracy, For recall rate, This is the generalization gap value. For data breach rate, and The penalty coefficient is set to, for example, 1-5.

[0064] Based on the above formula, it can be understood that the sum of the weights of each category recognition sub-model is 1.

[0065] In this implementation, a mechanism combining data leakage protection, generalization gap assessment, and adaptive weight penalty is used to automatically suppress the problem of inflated performance caused by sample overlap or subset overfitting.

[0066] S104. Based on sub-weights, multiple category recognition sub-models are fused with the basic recognition model to obtain the target recognition model.

[0067] The sub-weights obtained in step S103 are used as the weights of the category recognition sub-model, and the weights of the basic recognition model are set to 1. Multiple category recognition sub-models are then fused with the basic recognition model.

[0068] Specifically, the weights are used as the weights of each layer of the corresponding model. For the same level parameters of the category recognition sub-model and the basic recognition model, they are weighted and fused according to the corresponding weights to obtain the target recognition model.

[0069] In this implementation, weights are allocated based on the differences in model performance, which can automatically reduce the impact of overfitted models or models with repeated data, achieving a dynamic balance between optimal performance and optimal stability, and finally integrating the parameters of each model to form a unified fusion model.

[0070] Furthermore, the target recognition model is fine-tuned based on sample data of the identified target. Specifically, the parameters of the fused target recognition model are used as initial values, a conservative learning rate is set, and the target recognition model is trained using a mixture of general and differential sample data. The feature distribution is recalibrated to eliminate the differences in feature distribution between different models, improve the parameter space stability of the fused target recognition model, further enhance the generalization ability of the target recognition model, and improve the recognition accuracy.

[0071] S105. Based on the target recognition model, the target is identified to obtain the target recognition result.

[0072] In one possible implementation, this application sets up a dual recognition method using both a target recognition model and a multimodal large model. Please refer to... Figure 3 , Figure 3 This is a schematic diagram of a dual-target recognition method provided in an embodiment of this disclosure.

[0073] Please see Figure 3 The image of the target to be identified is input into the target recognition model. A low detection threshold is set in the target recognition model to identify the target, resulting in preliminary recognition results of multiple target bounding boxes. Setting a low detection threshold improves the detection rate of the target recognition model and increases the number of target bounding boxes, which include true positive (TP) and false positive (FP) boxes. For example, the detection threshold is set between 0.5 and 0.9.

[0074] Furthermore, this application employs a multimodal large model to perform secondary target judgment on the target recognition box. Specifically, the target recognition box is input into the multimodal large model, which then determines whether the target recognition box contains a real target to distinguish between real target boxes and false detection boxes. When the target recognition box contains a target, it is output.

[0075] In one possible implementation, a multimodal large model (MLLM) performs semantic parsing on the image region of the target recognition box, performs cross-modal consistency matching with a preset target semantic feature library or semantic description, calculates a consistency score, and when the consistency score is greater than a preset threshold, it is determined that the target recognition box contains a real target, the target recognition box determined to be a real target is retained, the target recognition box determined to be a false target is proposed, and the final recognition result is saved.

[0076] In this implementation, by combining the target recognition model and the multimodal large model, false positive samples that do not conform to semantic features can be automatically eliminated, which can improve the detection rate of real targets and effectively filter out false detection interference, thus achieving accurate and reliable target recognition output.

[0077] Furthermore, this application integrates a log and metric storage mechanism to record information such as training hyperparameters, evaluation grids, fusion weights, and generalization metrics throughout the entire process, forming an auditable performance tracing chain. The entire process of model fusion, retraining, and evaluation can be executed automatically, resulting in low engineering deployment costs and high efficiency in repeating experiments.

[0078] This application employs a hierarchical training architecture combining a general basic recognition model and multiple expert category recognition sub-models to specifically model differences in morphology, texture, reflection, and scale among similar targets. After adaptive weight fusion and unified fine-tuning, the model significantly improves recall and significantly reduces false negatives in scenarios with significant intra-class differences, achieving high-precision detection and stable output for heterogeneous subclusters. Simultaneously, a multimodal large model performs semantic-level re-judgment of the candidate boxes detected by the recognition model, automatically eliminating false positives and improving the reliability of terminal recognition. The entire framework is compatible with mainstream detection architectures such as the YOLO series and supports multiple fusion strategies. The model training, fusion, and re-judgment modules are all implemented using standard Python / PyTorch, allowing for rapid migration to GPU / edge device environments and deployment without modifying the underlying inference engine.

[0079] To achieve the above embodiments, this disclosure also proposes a device for identifying intra-class differential targets.

[0080] Figure 4 This is a schematic diagram of a device for identifying intra-class differential targets provided in an embodiment of this disclosure. The device can be implemented by software and / or hardware, and is generally integrated into a computing device. Figure 4 As shown, the device for identifying differential targets within this category includes: The construction module 401 is used to construct a general training sample set based on the sample data of the identified target. For each category of difference in the identified target, a subset of difference training samples is constructed based on the corresponding difference sample data.

[0081] Training module 402 is used to train a basic recognition model for identifying targets using a general training sample set, and to train a category recognition sub-model for intra-class differences using a subset of difference training samples.

[0082] Module 403 is used to determine sub-weights based on the model performance of the category recognition sub-model; The fusion module 404 is used to fuse multiple category recognition sub-models with the basic recognition model based on sub-weights to obtain the target recognition model.

[0083] The recognition module 405 is used to recognize the target based on the target recognition model and obtain the target recognition result.

[0084] In one possible implementation, training module 402 includes: The first training unit is used to set a low learning rate and a first resolution, and to perform the first stage training of the category recognition sub-model using a subset of differential training samples to generate the first stage weights of the category recognition sub-model.

[0085] The second training unit is used to set the second resolution and the intersection-union ratio threshold, load the first-stage weights of the category recognition sub-model, perform second-stage training on the category recognition sub-model using a subset of differential training samples, and generate the second-stage weights of the category recognition sub-model.

[0086] The third training unit is used to set the third resolution and regularization intensity, load the second-stage weights of the category recognition sub-model, perform second-stage training on the category recognition sub-model using a subset of differential training samples, and generate the third-stage weights of the category recognition sub-model.

[0087] The generation unit is used to load the third-stage weights of the category recognition sub-model and generate the trained category recognition sub-model.

[0088] In one possible implementation, building module 401 includes: The first computing unit is used to calculate the cross ratio between the general training sample set and the differential training sample subset using an image content hashing algorithm.

[0089] The setting unit is used to set the learning rate and training epochs of the category recognition sub-model corresponding to the intra-class difference category based on the cross ratio. The higher the cross ratio, the lower the corresponding learning rate and the fewer the training epochs.

[0090] In one possible implementation, the determining module 403 includes: The evaluation unit is used to evaluate the performance of the category recognition sub-model and obtain precision, recall, and overall index values.

[0091] The second calculation unit is used to calculate the generalization gap value of the category identification sub-model using the comprehensive index value.

[0092] The third calculation unit is used to calculate the data leakage rate by utilizing the intersection ratio of the differential training sample subset and the general training sample set.

[0093] The determination unit is used to determine sub-weights based on the calculation of model performance, including precision, recall, comprehensive index value, generalization gap value, and data leakage rate.

[0094] In one possible implementation, the determining unit includes: The first calculation subunit is used to calculate the average of precision, recall, and comprehensive index values ​​for each category recognition submodel. The average value is then corrected using the generalization gap and data leakage rate to obtain the corresponding model performance index value.

[0095] The second calculation subunit is used to calculate the ratio of the model performance index to the sum of the indices, and obtain the sub-weights; where the sum of the indices is the sum of the model performance indices of all category recognition sub-models.

[0096] In one possible implementation, the fusion module 404 includes: The fine-tuning unit is used to fine-tune the target recognition model based on sample data of the target.

[0097] In one possible implementation, the identification module 405 includes: The first recognition unit is used to set a low detection threshold and use the target recognition model to recognize the target, thereby obtaining multiple target recognition boxes.

[0098] The second recognition unit is used to perform secondary target judgment on the target recognition box using a multimodal large model, to determine whether the target recognition box contains the target. When the target recognition box contains the target, the target recognition box is output.

[0099] The intra-class difference target identification device provided in this disclosure can execute the intra-class difference target identification method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method execution.

[0100] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for identifying intra-class difference targets in the above embodiments.

[0101] Figure 5 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0102] The following is a detailed reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the computing device 500 in the embodiments of this disclosure. The computing device 500 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The computing device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0103] like Figure 5As shown, the computing device 500 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computing device 500. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0104] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows computing device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 A computing device 500 with various devices is shown; however, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0105] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the intra-class difference target identification method of embodiments of this disclosure.

[0106] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0107] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0108] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0109] The aforementioned computer-readable medium carries one or more programs, which, when executed by the computing device, cause the computing device to perform the aforementioned method for identifying intra-class differential targets.

[0110] The computing device can be programmed with computer program code in one or more programming languages ​​or a combination thereof to perform the operations of this disclosure. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0112] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0113] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0114] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0115] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0116] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0117] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for identifying intra-class differential targets, characterized in that, The method includes: A general training sample set is constructed based on the sample data of the identified target. For each category of difference within the identified target, a subset of difference training samples is constructed based on the corresponding difference sample data. The basic recognition model of the target is trained using the general training sample set, and the category recognition sub-model of the intra-class difference category is trained using the difference training sample subset; The sub-weights are determined based on the model performance of the sub-models identified by the aforementioned categories; Based on the sub-weights, multiple category recognition sub-models are fused with the basic recognition model to obtain a target recognition model; The target is identified based on the target recognition model to obtain the target recognition result.

2. The method for identifying intra-class differential targets according to claim 1, characterized in that, The step of training the category recognition sub-model for the intra-class differential categories using the subset of differential training samples includes: A low learning rate and a first resolution are set, and the category recognition sub-model is trained in the first stage using the differential training sample subset to generate the first stage weights of the category recognition sub-model. Set a second resolution and an intersection-union ratio threshold, load the first-stage weights of the category recognition sub-model, and use the subset of differential training samples to perform a second-stage training on the category recognition sub-model to generate the second-stage weights of the category recognition model; Set the third resolution and regularization intensity, load the second-stage weights of the category recognition sub-model, and use the differential training sample subset to perform the second-stage training of the category recognition sub-model to generate the third-stage weights of the category recognition sub-model; Load the third-stage weights of the category recognition sub-model to generate the trained category recognition sub-model; Wherein, the first resolution is smaller than the second resolution, the second resolution is smaller than the third resolution, the number of training rounds in the first stage is greater than the number of training rounds in the second stage, and the number of training rounds in the second stage is greater than the number of training rounds in the third stage.

3. The method for identifying intra-class differential targets according to claim 1 or 2, characterized in that, The process of training the basic recognition model for the target using the general training sample set, and training the category recognition sub-model for the intra-class difference categories using the difference training sample subset, further includes: The cross ratio between the general training sample set and the differential training sample subset is calculated using an image content hashing algorithm; The learning rate and training epochs of the category recognition sub-model corresponding to the intra-class difference category are set based on the crossover ratio. The higher the crossover ratio, the lower the corresponding learning rate and the fewer the training epochs.

4. The method for identifying intra-class differential targets according to claim 1, characterized in that, The determination of sub-weights based on the model performance of the category recognition sub-model includes: The performance of the category recognition sub-model is evaluated to obtain precision, recall, and comprehensive index values. The generalization gap value of the category identification sub-model is calculated using the comprehensive index value; The data leakage rate is calculated using the intersection ratio of the differential training sample subset and the general training sample set; The model performance determinant weights are calculated based on the precision, recall, composite index value, generalization gap value, and data leakage rate.

5. The method for identifying intra-class differential targets according to claim 4, characterized in that, The calculation of the model performance determination sub-weights based on the precision, recall, composite index value, generalization gap value, and data leakage rate includes: For each category recognition sub-model, calculate the average of the precision, the recall, and the comprehensive index value, and correct the average value using the generalization gap value and the data leakage rate to obtain the corresponding model performance index value; The sub-weights are obtained by calculating the ratio of the model performance index to the sum of the indices; wherein the sum of the indices is the sum of the model performance indices of all the category recognition sub-models.

6. The method for identifying intra-class differential targets according to claim 1, characterized in that, The step of fusing multiple category recognition sub-models with the basic recognition model based on the sub-weights to obtain a target recognition model further includes: The target recognition model is fine-tuned based on the sample data of the target.

7. The method for identifying intra-class differential targets according to claim 1, characterized in that, The process of identifying the target based on the target recognition model to obtain the target recognition result includes: A low detection threshold is set, and the target recognition model is used to recognize the target to obtain multiple target recognition boxes; A multimodal large model is used to perform secondary target judgment on the target recognition box to determine whether the target recognition box contains the target. When the target recognition box contains the target, the target recognition box is output.

8. A device for identifying intra-class differential targets, characterized in that, The device includes: The construction module is used to construct a general training sample set based on the sample data of the identified target, and to construct a subset of differential training samples based on the corresponding differential sample data for each category of difference of the identified target. The training module is used to train the basic recognition model of the target using the general training sample set, and to train the category recognition sub-model of the intra-class difference category using the difference training sample subset; The determination module is used to determine sub-weights based on the model performance of the sub-models identified by the category; The fusion module is used to fuse multiple category recognition sub-models with the basic recognition model based on the sub-weights to obtain a target recognition model; The recognition module is used to recognize the target based on the target recognition model and obtain the target recognition result.

9. A computing device, characterized in that, The computing device includes: a processor; a memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the instructions to implement the method for identifying intra-class differential targets as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method for identifying intra-class differential targets according to any one of claims 1 to 7.

Images