A target detection model, a target detection method, a computing device, and a storage medium.

By constructing a mixed sample set and using class alignment hints to train the object detection model, the problem of class conflict in training on non-same-origin datasets is solved, and the model achieves high recall and high precision without changing the basic architecture.

CN122199902BActive Publication Date: 2026-11-17CORECHENG (BEIJING) TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610084748.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-01-21
Publication Date
2026-11-17
Estimated Expiration
2046-01-21

AI Technical Summary

Technical Problem

Existing object detection models are prone to class conflicts when trained on non-same-origin datasets, resulting in increased recall but decreased precision. It is difficult to adapt to non-same-origin class-labeled data and achieve high-quality object detection without changing the basic model architecture.

Method used

By constructing a first mixed sample set, which includes the first sample set and multiple non-homologous second sample sets, and using class alignment cues for model training, we ensure that the positive sample supervision signals are correct and avoid the negative sample supervision signals being incorrect. We then combine the first sample set with full class annotations for model training.

Benefits of technology

Without altering the model's fundamental architecture, it achieves dual stability in both recall and precision, balancing detection capabilities across all categories and performance reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122199902B_ABST
    Figure CN122199902B_ABST
Patent Text Reader

Abstract

This disclosure relates to a method for training an object detection model, an object detection method, a computing device, and a storage medium. The training method includes: acquiring a first mixed sample set; wherein the first mixed sample set consists of a first sample set and at least two second sample sets, the at least two second sample sets being associated with different second category sets, the first sample set being associated with a first category set, the first category set containing all categories in each of the second category sets; and training a first object detection model based on the first category set to perform an object detection task using the first mixed sample set; wherein each sample set in the first mixed sample set is supervised for the object detection task according to its associated category set during the training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of target detection technology, and more specifically, to a target detection model, a target detection method, a computing device, and a computer-readable storage medium. Background Technology

[0002] The core function of an object detection model is to identify and locate objects in an input image based on an associated class set, and output the corresponding detection results. In traditional training modes, the model usually relies on the assumption that all classes are known, that is, it is assumed that each sample contains information about all classes in the class set, and negative samples are constructed based on this full set of classes to optimize the model's ability to distinguish different target objects.

[0003] However, this assumption is significantly out of touch with real-world production scenarios. For example, data collected across datasets and projects often employs non-same-origin annotation systems. The target annotation rules of different datasets are independent of each other, with inconsistent category coverage. There may be complex relationships between categories, such as synonym mapping, hierarchical inclusion, or complete mutual exclusion, making it difficult to directly merge various datasets into a unified training sample set. If a full-class negative sampling strategy is forcibly adopted in training on non-same-origin datasets, it is very easy to cause category conflicts, generate incorrect negative supervision signals, and ultimately lead to a performance imbalance problem where the model's recall increases but its precision decreases.

[0004] Therefore, without altering the basic model architecture, how to effectively adapt to the characteristics of non-homogeneous category labeled data, achieve high-quality model training, and enable the model to have object detection capabilities across all categories has become a pressing technical challenge for the industry. Summary of the Invention

[0005] This disclosure provides a novel technical solution for training a target detection model using a non-homologous sample set, thereby improving the accuracy of the model in target detection.

[0006] According to a first aspect of this disclosure, a method for training an object detection model is provided, the method comprising: Obtain a first mixed sample set; wherein the first mixed sample set consists of a first sample set and at least two second sample sets, the at least two second sample sets are respectively associated with different second category sets, the first sample set is associated with the first category set, and the first category set contains all categories in each second category set; A first object detection model capable of performing object detection tasks based on the first category set is trained using the first mixed sample set; wherein, each sample set in the first mixed sample set is supervised for object detection tasks according to its associated category set during the training.

[0007] Optionally, at least some samples in the first sample set are first-class samples, which are generated by supplementing the original target annotations with the first-class set based on the sample data in any second sample set.

[0008] Optionally, the first category set is the union of all the second category sets, and the step of generating the first category sample includes: A subset of samples is extracted from the target sample set to form a target sample subset; wherein, the target sample set is any one of the at least two second sample sets; Determine each non-homologous second target detection model corresponding to the target sample subset; wherein, the non-homologous second target detection model is trained based on other second sample sets different from the target sample set; Each non-homologous second target detection model is used to perform target detection on the sample data in the target sample subset to obtain supplementary target annotations for the corresponding sample data on the second category sets associated with all other second sample sets. For each sample data in the target sample subset, the supplementary target label of the corresponding sample data and the original target label of the corresponding sample data are merged to obtain the first type of sample.

[0009] Optionally, the step of merging the supplementary target labels of the corresponding sample data with the original target labels of the corresponding sample data includes: filtering out supplementary target labels in the corresponding sample data whose confidence level is less than a first set threshold, and / or filtering out supplementary target labels in the corresponding sample data that have class conflicts; The process of filtering out supplementary target labels with class conflicts in the corresponding sample data includes: filtering out the corresponding supplementary target label when the overlap between the detection boxes of the supplementary target label and the original target label reaches a second set threshold and the supplementary target label has a class conflict with the original target label; and / or, filtering out the supplementary target label with low confidence when the overlap between the detection boxes of two supplementary target labels reaches a third set threshold and the two supplementary target labels have a class conflict.

[0010] Optionally, after obtaining the first target detection model, the method further includes: Obtain a second mixed sample set; wherein the second mixed sample set consists of at least two second sample sets, a third sample set, and a fourth sample set, the third sample set is associated with a third category set, the fourth sample set is associated with a fourth category set, the categories in the fourth category set do not belong to the first category set, and the third category set includes all categories in the first category set and the fourth category set; Using the second mixed sample set, the first object detection model is used as the initial model for model training to obtain a third object detection model that can perform object detection tasks based on the third category set; wherein, each sample set in the second mixed sample set is supervised for object detection tasks according to its associated category set during model training.

[0011] Optionally, at least some samples in the third sample set are second-class samples, which are generated by supplementing the fourth-class set with additional annotations on the sample data in the first sample set, based on the original target annotations corresponding to the first category set.

[0012] Optionally, after obtaining the first target detection model, the method further includes: Obtain a specified sample; wherein the target category of the specified sample belongs to the second category set; The specified sample is added to the second sample set to form an updated first mixed sample set; The first target detection model is updated by updating the first mixed sample set to obtain the updated first target detection model.

[0013] According to a second aspect of this disclosure, a target detection method is also provided, the method comprising: Acquire scene images captured by mobile devices; The object detection model is invoked to perform object detection on the scene image based on the first category set, and the object detection result is obtained; The target detection model is trained using a first mixed sample set, which consists of a first sample set and at least two second sample sets. The at least two second sample sets are associated with different second category sets, and the first sample set is associated with a first category set, which contains all categories in each of the second category sets. During training, each sample set in the first mixed sample set is supervised for the target detection task according to its associated category set.

[0014] According to a third aspect of this disclosure, a computing device is provided, including a memory for storing a computer program; and a processor configured to implement the method as described in any one of the first aspects when executing the computer program stored in the memory.

[0015] According to a fourth aspect of this disclosure, a non-volatile computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0016] According to embodiments of this disclosure, model training is performed using at least two non-homologous second sample sets combined with a first sample set with full class labeling. During training, correct positive sample supervision signals are formed through class alignment cues, while avoiding the generation of erroneous negative sample supervision signals. The class boundaries of non-homologous data are clearly defined, and the full supervision information of the first sample set serves as guidance. Thus, without modifying the basic model architecture, dual stability of model recall and precision can be achieved, taking into account both the detection capability of the full class and the reliability of detection performance. Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0018] Figure 1 This is a schematic diagram of a data processing system to which the methods provided in the embodiments of this disclosure can be applied; Figure 2 This is a flowchart illustrating a target detection model training method according to some embodiments; Figure 3 This is a schematic diagram illustrating the process of training object detection models based on some examples; Figure 4 This is a flowchart illustrating a target detection model training method according to some other embodiments; Figure 5 This is a schematic diagram illustrating the process of training object detection models based on other examples; Figure 6 This is a flowchart illustrating a target detection model training method according to some other embodiments; Figure 7 This is a flowchart illustrating a target detection method according to some embodiments; Figure 8 This is a schematic diagram of the hardware structure of a computing device according to some embodiments. Detailed Implementation

[0019] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0020] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0021] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0022] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0024] This disclosure relates to a training method for an object detection model. The object detection model is used to identify and locate objects in an input image based on an associated category set, and outputs corresponding detection results. The detection results include the category label of the detected object and the location information of the detected object in the input image. This location information can be represented by the corner coordinates of the detection box, or by the center coordinates of the detection box combined with the size (length, width) of the detection box.

[0025] In some application scenarios, the number of target categories is huge, the semantic boundaries are complex, and they continue to expand with business needs, which poses a severe challenge to the architecture design of target detection models and the annotation of training samples.

[0026] Taking the scene perception task of a Driving Automation System (DAS) as an example, due to the complexity of traffic scenarios, the targets to be detected are not only numerous and have ambiguous semantic boundaries, but the category system also needs to be continuously expanded with business iterations. This driving automation system can be an Advanced Driver Assistance System (ADAS) or an Autonomous Driving System (ADS). In such scenarios, there are often multiple sample sets generated based on non-same-origin annotation systems. The category coverage of different annotation systems varies, and the categories may also exhibit complex relationships such as synonym mapping, hierarchical inclusion, or complete mutual exclusion, making it difficult to directly merge various sample sets into a unified training sample set. If a full-class negative sampling strategy is forcibly adopted during the training process of non-same-origin sample sets, it is very easy to cause category conflicts, generate erroneous negative supervision signals, and ultimately cause a performance imbalance problem where the model's recall rate increases while its precision rate decreases.

[0027] In related technical fields, one approach combines vision-language pre-training with open vocabulary detection. Typical methods include Grounded Language-Image Pre-training (GLIP), Region-based Language-Image Pre-training (RegionCLIP), Vision and Language Detector (ViLD), and Detector with Image Classes (Detic). These methods achieve target detection under an open vocabulary through image-text alignment, category text encoding, and region matching. However, their training process still relies on a unified or mapped category set to construct negative samples, and the default sample annotations are complete. This requires unified standardization or fine mapping of each category set, which not only results in high engineering implementation costs and difficult maintenance but also easily introduces mapping errors that affect model performance.

[0028] Another related technology employs a multi-sample set joint training strategy. This approach attempts to unify multi-source data into the same category set by constructing a category mapping table, using hierarchical annotation, or calculating word vector similarity. This type of approach heavily relies on manual mapping, domain prior knowledge, or strong alignment assumptions, making it difficult to adapt to the business needs of dynamically expanding data. Furthermore, errors generated during the mapping process can further amplify noise in the supervision signal, reducing the model training effect.

[0029] Another related technique uses a partially labeled (also known as incompletely labeled) training mode, which alleviates class conflict by ignoring unknown categories or using a partial negative sample suppression strategy. However, this approach has a significant drawback of insufficient negative supervision, which is usually manifested as a decrease in the model accuracy (AP) metric.

[0030] In summary, existing technical solutions either require manually labeled uniform category sets, resulting in high engineering costs, or while reducing the labeling workload through partial labeling, they fail to train high-precision models due to insufficient negative supervision. To address the challenges related to sample labeling, this disclosure proposes a target detection model training scheme. This scheme trains the model based on a mixed sample set associated with different category sets, achieving dual stability of model recall and precision without altering the basic architecture of the target detection model.

[0031] The method of this disclosure embodiment can be implemented by the data processing system shown in FIG1. ​​As shown in FIG1, the data processing system includes a mobile device 101 and a computing device 102.

[0032] The mobile device 101 is equipped with an image acquisition device 1014, which is used to acquire multi-scene image data during the movement of the device 101, resulting in N datasets, where N is an integer greater than or equal to 2, namely datasets D21 to D2N. The image acquisition device 1014 may include at least one of a camera and a LiDAR. The image data may be data acquired by the camera or data obtained by mapping point cloud data acquired by the LiDAR onto a two-dimensional plane. The aforementioned N datasets are labeled with categories based on different annotation systems, thereby obtaining N sample sets S21 to S2N. Different annotation systems correspond to different category sets, and there are at least some category differences between each category set. For example, dataset D21 generates sample set S21 based on annotation system V21', and dataset D2N generates sample set S2N based on annotation system V2N'. Each sample in the sample set includes image data as sample data and target annotation corresponding to the sample data. The target annotation includes the target label for the category set under the corresponding annotation system and the position information of the corresponding target in the sample data. Datasets can be labeled manually, fully automatically using a model, or automatically using a model followed by manual calibration.

[0033] The computing device 102 is configured to train an object detection model that can perform object detection tasks under all categories based on the above non-homologous sample sets, where the full category includes all categories in the category sets associated with each non-homologous sample set.

[0034] The mobile device 101 can be a data collection vehicle specifically designed for data acquisition, or it can be a moving vehicle, robot, drone, etc., without limitation. Figure 1 As shown, in addition to the camera 1015, the mobile device 101 may also include a processor 1011, a memory 1012, a communication device 1013, etc., so that the mobile device 101 has the ability to perform set tasks and communicate with the outside world. Figure 1 Only one mobile device is shown schematically. The data processing system can acquire image data through multiple mobile devices, and is not limited to any one of them.

[0035] Figure 1 The computing device 102 can be any physically existing device, machine, or system capable of performing predefined computing tasks through hardware components, namely, receiving input images, processing images, and outputting recognition results. The computing device 102 can be, for example, a server, computer, controller, or system-on-a-chip. The computing device 102 can be set up independently of the mobile device 101, or it can be at least partially set up within the mobile device 101; this is not limited here.

[0036] like Figure 1As shown, the computing device 102 may include a processor 1021 and a memory 1022. The processor 1021 may include at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System-on-Chip (SOC), an Application Specific Integrated Circuit (ASIC), a Micro Controller Unit (MCU), or other processors. The memory 1022 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. In addition to storing instructions, the memory may also store input data, intermediate data, and result data, and the data stored in the memory can be retrieved and used by the processor.

[0037] In addition, the computing device 102 may also include a communication device 1023 and an interface device 1024, which can receive various sample sets, for example, through the communication device 1023 or the interface device 1024.

[0038] The following is combined Figure 1 The systems shown illustrate various embodiments of this disclosure.

[0039] <First Embodiment> Figure 2 This illustrates a training method for an object detection model according to some embodiments, which can be performed by... Figure 1 The computing device 102 is implemented in the middle. For example... Figure 2 As shown, the training method in this embodiment includes the following steps S210 and S220: Step S210: Obtain the first mixed sample set, which consists of the first sample set S1 and N second sample sets, where N is an integer greater than or equal to 2.

[0040] N second sample sets are non-homologous sample sets, corresponding to Figure 1There are N sample sets S21 to S2N in the dataset. These second sample sets are labeled with categories based on different labeling systems (i.e., non-homogeneous labeling systems), so that different second sample sets are associated with different second category sets. Moreover, the samples in each second sample set are labeled only within the scope of the second category set they are associated with. In other words, among the N second sample sets, the i-th second sample set S2i (1≤i≤N) is associated with the second category set V2i, and the target labeling of all samples in the second sample set S2i is limited to the category scope of the second category set V2i. That is, the samples in the second sample set S2i are only labeled with targets in the categories listed in the second category set V2i, and are not labeled with targets in other categories outside the second category set V2i.

[0041] In this embodiment, there are at least some category differences between different second category sets under non-homogeneous annotation systems. For example, the second category set V21 is adapted to the road scene annotation system and includes categories such as pedestrians, guardrails, cars, buses, and non-motorized vehicles, while the second category set V22 is adapted to the intersection scene annotation system and includes categories such as pedestrians, vehicles, traffic lights, zebra crossings, and signs.

[0042] In this embodiment, unlike the second sample set S2i, the first sample set S1 is a complete set of category samples, whose annotation range covers all categories of the second category sets. That is, the first category set V1 contains at least all categories of each second category set. For example, the first category set V1 can be the union of N second category sets, which can be expressed as: V1 = V21∪V22∪ ... ∪ V2N. Samples in the first sample set S1 are labeled within the scope of their associated first category set V1.

[0043] In this embodiment, each second sample set S2i mainly serves as a positive sample supervision unit during model training, ensuring the model's feature learning performance for each second sample set S2i's specific category. The first sample set S1 plays a role in standardizing the negative sample construction logic and calibrating cross-category boundaries during model training. Its sample size can be less than the total number of samples in the N second sample sets. For example, the sample size of the first sample set S1 accounts for 5%-25% of the total number of samples in the N second sample sets.

[0044] The target labels of the first sample set S1 conform to the first labeling standard, and the target labels of the second sample set S2i conform to the second labeling standard. The first labeling standard may be consistent with the second labeling standard, or it may be lower than the second labeling standard in some or all dimensions. The evaluation dimensions of the labeling standard include, for example, labeling accuracy and labeling completeness.

[0045] In some examples, the target annotations in the sample data of the second sample set S2i are ground truth annotations based on the corresponding annotation system. Ground truth annotation refers to the precise annotation of target objects belonging to the associated category set in the image data, ensuring that the annotation results meet the second annotation standard. Ground truth annotation can be achieved through direct manual annotation, that is, professional annotators review and annotate the category labels and locations of the targets frame by frame; or it can be achieved through automated model annotation combined with manual calibration, that is, automated annotation is first completed by a teacher's model, and then manual calibration is performed to correct mislabeled and missing samples, ensuring that the annotation results meet the second annotation standard.

[0046] For the first sample set S1, in some examples, at least a portion of its samples are ground truth samples obtained through ground truth annotation. The production method for these ground truth samples can be as follows: A certain proportion of samples are extracted from each second sample set S2i. For these sample data, while retaining the original category annotations, categories from other second category sets are manually annotated to ultimately form ground truth samples that are fully annotated within the first category set V1. For example, road scene samples extracted from the second sample set S21, whose original category annotations include "pedestrians, ordinary cars," etc., from the second category set V21, are supplemented with annotations for intersection scene-specific categories such as "traffic lights, zebra crossings," from the second category set V22, thus obtaining the corresponding ground truth samples.

[0047] For the first sample set S1, in some other examples, at least some of the samples are pseudo-truth value samples obtained through pseudo-truth value annotation. Pseudo-truth value annotation refers to fully automated annotation without human intervention, which can significantly improve annotation efficiency.

[0048] Furthermore, to improve the production efficiency of the first sample set, in some examples, at least some samples in the first sample set are first-class samples generated based on sample data from the second sample set. Here, the first-class samples are generated by supplementing the original target annotations of sample data in any second sample set S2i with annotations based on the first category set V1. The original target annotation of the sample data is the target annotation of that sample data in its respective second sample set S2i. In this example, for other categories in the first category set V1 that do not belong to the second category set V2i, supplementary annotations can be performed on the sample data in the second sample set S2i manually or using a model, thereby covering all categories of first-class samples.

[0049] In these examples, generating the first type of samples through the model may include the following steps S2101 to S2104: Step S2101: Extract a portion of samples from the target sample set to form a target sample subset, which is any one of the N second sample sets.

[0050] In this example, a subset of samples can be extracted from each second sample set S2i to form N target sample subsets. The target sample subset formed by the subset of samples extracted from the second sample set S2i is denoted as ΔS2i.

[0051] Step S2102: Determine the second target detection model corresponding to each non-homologous subset of the target samples.

[0052] For the target sample subset, the non-homologous second target detection model is trained on at least two other second sample sets that are different from the target sample set.

[0053] Taking the existence of three second sample sets as an example, for the target sample subset ΔS21, the second target detection models that are not homologous to it include: the second target detection model M22 that performs the target detection task based on the second category set V22, and the second target detection model M23 that performs the target detection task based on the second category set V23. Similarly, for the target sample subset ΔS22, the second target detection models that are not homologous to it include: the second target detection model M21 that performs the target detection task based on the second category set V21, and the second target detection model M23 that performs the target detection task based on the second category set V23, and so on.

[0054] The second object detection model M2i can be trained based on the corresponding second sample set S2i, which helps improve the consistency of annotation between the first and second sample sets. Taking the case of three second sample sets as an example, the second object detection model M21 is trained based on the second sample set S21, the second object detection model M22 is trained based on the second sample set S22, and the second object detection model M23 is trained based on the second sample set S23. The second object detection model M2i can also be trained based on other sample sets associated with the second category set V2i; this is not a limitation here.

[0055] Each second object detection model can have the same model structure but different model parameter values, thereby enabling each second object detection model to accurately perform object detection tasks within its associated second category set.

[0056] The initial models used to train each second object detection model can have the same model parameter values ​​or different model parameter values.

[0057] Step S2103: Perform target detection on the sample data in the target sample subset using each non-homologous second target detection model to obtain supplementary target annotations for the corresponding sample data on the second category sets associated with each of the other second sample sets.

[0058] Taking the existence of three second sample sets as an example, in step S2103, for the target sample subset ΔS21, target detection is performed on the sample data in the target sample subset ΔS21 by the second target detection model M22 and the second target detection model M23 respectively, to obtain the supplementary target annotations of the sample data on the second category set V22 and the second category set V23.

[0059] Step S2104: For each sample data in the target sample subset, the supplementary target label of the corresponding sample data is merged with the original target label of the corresponding sample data on the second category set associated with the second sample set to obtain the first category sample.

[0060] For each sample data in the target sample subset, when fusing its supplementary target annotation and original target annotation, the two types of target annotations can be merged directly; alternatively, inaccurate supplementary target annotations can be filtered out first, and then the filtered supplementary target annotations can be merged with the original target annotations to improve the accuracy of the supplementary target annotations.

[0061] When performing annotation filtering, at least one of the following filtering processes can be performed: The first step is to filter out supplementary target labels in the corresponding samples whose confidence level is less than the first set threshold, so as to ensure the accuracy of the supplementary target labels. The second step is to filter out supplementary target labels that have class conflicts in the corresponding samples.

[0062] Filtering out supplementary target labels with class conflicts in the corresponding samples can include: filtering out the corresponding supplementary target label when the overlap between the detection boxes of the original target label and the supplementary target label reaches a second preset threshold, and the supplementary target label has a class conflict with the original target label; and filtering out the supplementary target label with the lower confidence level when the overlap between the detection boxes of two supplementary target labels reaches a third preset threshold, and the two supplementary target labels have a class conflict. Class conflict here means that the two categories have different attributes. For example, vehicles and pedestrians constitute a conflicting category, and vehicles and traffic cones constitute a conflicting category. However, in the category of vehicles and buses, vehicles, because they include buses, share the same attributes as buses, and the two do not constitute a conflicting category. Any two of the first, second, and third preset thresholds can be the same or different, and this is not limited here.

[0063] In another example, the first sample set can also be generated based on other sample data that do not belong to any second sample set, without limitation here.

[0064] Step S220: Using the first mixed sample set, a first object detection model is trained that can perform object detection tasks based on at least the first category set. Different sample sets in the first mixed sample set are supervised for object detection tasks during training according to their respective associated category sets.

[0065] In this embodiment, the model is jointly trained by using different sample sets in the first mixed sample set and combining them with category alignment prompts, thereby obtaining the first target detection model.

[0066] In this embodiment, when training the model using the first mixed sample set, class alignment prompts are provided. For example, the second sample set S21 is associated with the second class set V21, the second sample set S22 is associated with the second class set V22, and the first sample set S1 is associated with the first class set V1. This class alignment prompt limits the model to learn features only within the specific class range of the second sample set S2i, ensuring the model's feature extraction accuracy for each specific class of the second sample set S2i, while avoiding false negative samples due to the introduction of unrelated classes. Furthermore, it enables the model to perceive cross-class negative samples and inter-class boundaries on the same sample data of the first sample set V1, avoiding misclassification of specific classes of different second sample sets as negative samples, and strengthening the model's ability to distinguish class boundaries, thereby achieving dual stability of model recall and precision.

[0067] In this embodiment, the first object detection model and the second object detection model M2i described above can have the same model structure but different model parameter values. The initial model used to train the first object detection model and the initial model used to train the second object detection model M2i can have the same model parameter values ​​(i.e., initial parameter values) or different model parameter values.

[0068] Figure 3 An example of the object detection model training process is shown. Figure 3 In the example shown, there are three non-homologous second sample sets S21, S22, and S23. Second sample set S21 is associated with second category set V21, second sample set S22 is associated with second category set V22, and second sample set S23 is associated with second category set V23. These three second sample sets S21, S22, and S23 are labeled with truth values. Figure 3 As shown, the corresponding model training method includes three stages: training the second object detection model, constructing the first mixed sample set, and training the first object detection model.

[0069] Phase 1: A second object detection model M21 is trained using the second sample set S21 to perform object detection tasks based on the second category set V21, with the category alignment hints during training containing only the second category set V21; A second object detection model M22 is trained using the second sample set S22 to perform object detection tasks based on the second category set V22, with the category alignment hints during training containing only the second category set V22; and A second object detection model M23 is trained using the second sample set S23 to perform object detection tasks based on the second category set V23, with the category alignment hints during training containing only the second category set V23.

[0070] After training three object detection models, threshold calibration or temperature scaling can be performed on their respective validation sets to make the confidence scores of the three models comparable. Threshold calibration involves adjusting or optimizing the classification confidence threshold based on the model's performance on the validation set, so that the model achieves the desired precision-recall tradeoff during inference, or meets expected business metrics (such as false positive rate and false negative rate requirements). Temperature scaling involves dividing the raw scores of the model output by a temperature coefficient T and then normalizing them to adjust the smoothness of the class probability distribution, making the probabilities closer to the true likelihood.

[0071] The second stage involves constructing the first mixed sample set, which includes three second sample sets S21, S22, and S23, as well as a first sample set S1 that can be used for full-scale category hint training within the first category set. The first category set V1 is the union of the second category sets V21, V22, and V23. The steps for generating the first sample set S1 include sample extraction, cross-model supplementation, and annotation fusion. Sample extraction involves drawing a certain proportion of samples from the second sample sets S21, S22, and S23 respectively, forming second sample subsets ΔS21, ΔS22, and ΔS23. The extraction proportion is, for example, 5%–20% of each of the second sample sets. By controlling the number of samples in the first sample set, the engineering cost of supplementary annotation can be controlled. During sampling, samples covering multiple scene conditions (such as weather, lighting, road type, etc.) can be extracted stratified from each second sample set to improve the model's target detection performance under different scene conditions.

[0072] Cross-model supplementation includes: for each sample data in the second sample subset ΔS21, performing target detection using the second target detection models M22 and M23 based on their respective associated second category sets to obtain supplementary target labels for the corresponding sample data corresponding to the second category sets V22 and V23; for each sample data in the second sample subset ΔS22, performing target detection using the second target detection models M21 and M23 based on their respective associated second category sets to obtain supplementary target labels for the corresponding sample data corresponding to the second category sets V21 and V23; and for each sample data in the second sample subset ΔS23, performing target detection using the second target detection models M21 and M22 based on their respective associated second category sets to obtain supplementary target labels for the corresponding sample data corresponding to the second category sets V21 and V22.

[0073] By using cross-model supplementation, the sample data of each sample extracted from the three second sample sets will have the original target label on the second category set associated with its own second sample set, as well as supplementary target labels on the second category set associated with other second sample sets.

[0074] The annotation fusion includes: for each sample data extracted from the three second sample sets, filtering the supplementary target annotations, and merging the filtered supplementary target annotations with the original target annotations to obtain the full target annotations of the first category set corresponding to the sample data, thereby producing the first sample set S1.

[0075] After obtaining the first sample set S1, a first mixed sample set can be constructed, which includes the first sample set S1 and three second sample sets S21, S22 and S23.

[0076] The third stage: Training a first object detection model adapted to the first category set based on the first mixed sample set. During training, each sample set in the first mixed sample set provides targeted supervision signals based on its associated category set, enabling the first object detection model to have high recall and precision, at least when performing object detection based on the first category set.

[0077] According to the method of this embodiment, by combining multiple existing non-homologous second sample sets with a small number of first sample sets fully labeled by manual or model annotation, and with category alignment hints, the model can avoid false negative samples due to the introduction of unrelated categories while ensuring the feature extraction accuracy of each second sample set S2i specific category, and enhance the model's ability to distinguish category boundaries. Compared with related technologies that use manual annotation of a unified category set or partial annotation for model training, this embodiment can effectively control engineering costs and provide effective negative supervision, thereby ensuring the recall and precision of the trained first object detection model.

[0078] According to the method of this embodiment, based on multiple existing non-homologous second sample sets, combined with a small number of first sample sets with full category annotation completed manually or by model, and with the addition of a category alignment suggestion strategy, the problem of false negative samples caused by the introduction of unrelated categories can be avoided from the source while ensuring the model's accuracy in extracting the S2i-specific category features of each second sample set. At the same time, the model's ability to distinguish between different category boundaries is enhanced. Compared with related technologies that use manual annotation of a unified category set or model training based on partial annotations, the method of this embodiment can effectively control the engineering annotation cost and provide accurate and reliable negative sample supervision signals, thereby ensuring that the trained first object detection model has both high recall and precision.

[0079] All object detection models in this embodiment can adopt an open vocabulary architecture to give the model stronger category generalization ability; the corresponding category alignment prompts can be implemented by inputting category alignment text into the model.

[0080] <Second Embodiment> The first target detection model generated through the first embodiment has the ability to expand its categories and data domains as business needs evolve. This embodiment provides a method for expanding the categories and data domains based on the first target detection model. Figure 4 As shown, the method of this embodiment includes the following steps S410 to S440: Step S410: Obtain a first mixed sample set; wherein the first mixed sample set consists of a first sample set and at least two second sample sets, the at least two second sample sets are respectively associated with different second category sets, the first sample set is associated with a first category set, and the first category set contains all categories in each second category set.

[0081] Step S420: A first object detection model is trained using the first mixed sample set to perform object detection tasks based on the first category set. During training, each sample set in the first mixed sample set is supervised according to its associated category set for object detection tasks.

[0082] Step S430: Obtain the second mixed sample set.

[0083] In this embodiment, the second mixed sample set consists of at least two second sample sets S21-S2N, a third sample set S3, and a fourth sample set S4. The third sample set S3 is associated with the third category set V3, and the fourth sample set S4 is associated with the fourth category set V4. Categories in the fourth category set V4 do not belong to the first category set V1. The third category set V3 includes all categories from both the first category set V1 and the fourth category set V4. In other words, in this embodiment, the categories in the fourth category set V4 are newly added categories relative to the first category set V1, and the fourth category set V4 can include one or more categories. The third category set V4 can be the union of the first category set V1 and the fourth category set V4.

[0084] In some examples, at least some samples in the third sample set S3 are second-class samples generated based on samples in the first class set V1. Similar to generating the first-class samples, the second-class samples are generated by supplementing the fourth-class set V4 with the original target annotations corresponding to the first class set V1, based on the sample data in the first sample set.

[0085] Step S440: Using the second mixed sample set, the first object detection model is used as the initial model for model training to obtain a third object detection model that performs object detection tasks based on the third category set. Each sample set in the second mixed sample set is supervised for object detection tasks according to its associated category set during model training.

[0086] Figure 5 An example of an expanded category for object detection models is shown. Figure 5 In the example shown, there are three non-homologous second sample sets S21, S22 and S23, and one fourth sample set S4. The second sample set S21 is associated with the second category set V21, the second sample set S22 is associated with the second category set V22, the second sample set S23 is associated with the second category set V23, and the fourth sample set is associated with the fourth category set V4.

[0087] like Figure 5 As shown in this example, a fourth object detection model M4 capable of performing object detection tasks within the fourth category set S4 can first be trained using the fourth sample set. Then, the fourth object detection model M4 is used to supplement the sample data in the first sample set S1 with additional annotations, and the supplemented object annotations are merged with the original object annotations corresponding to the first sample set S1 to generate second-class samples, thus obtaining a third sample set S4 covering the third category set. Finally, based on a second mixed sample set consisting of the third sample set S3, the fourth sample set S4, and three second sample sets S21, S22, and S23, the first object detection model M1 is used as the initial model for training, resulting in a third object detection model capable of performing object detection tasks based on the third category set V3.

[0088] As can be seen, the method of this embodiment can quickly expand the categories of the target detection model and improve the ability of the target detection model to adapt to different business needs.

[0089] <Third Embodiment> The first target detection model generated through the first embodiment can quickly improve its detection performance for one or more existing categories according to business needs. For example... Figure 4 As shown, the method of this embodiment includes the following steps S610 to S650: Step S610: Obtain a first mixed sample set; wherein the first mixed sample set consists of a first sample set and at least two second sample sets, the at least two second sample sets are respectively associated with different second category sets, the first sample set is associated with a first category set, and the first category set contains all categories in each second category set.

[0090] Step S620: A first object detection model is trained using the first mixed sample set to perform object detection tasks based on the first category set. Each sample set in the first mixed sample set is supervised for object detection tasks during training according to its associated category set.

[0091] Step S630: Obtain a set sample, the target category of which belongs to N sets of second categories.

[0092] The target category of the sample is set to the existing category for which the detection performance needs to be improved.

[0093] Step S640: Add the set sample to the second sample set to form an updated first mixed sample set.

[0094] For example, if a sample is set to have a target category belonging to the second category set V21, then the set sample is added to the second category set V21 to form an updated first mixed sample set.

[0095] Step S650: Update the first target detection model using the updated first mixed sample set to obtain the updated first target detection model.

[0096] In step S650, the updated first target detection model can be further trained using the first mixed sample set and the first target detection model obtained in step S620 as the initial model to obtain the updated first target detection model.

[0097] According to the method of this embodiment, after obtaining the first target detection model, if it is necessary to improve the detection performance of the first target detection model for one or more specific categories in the first category set based on business needs, the labeled samples of that category can be directly added, while maintaining the original category alignment hint strategy. This can quickly improve the model's performance in that category without having to regenerate the first category set, thereby quickly improving the model's performance.

[0098] The model training method of any embodiment of this disclosure can still achieve excellent application results in complex application scenarios such as assisted driving, autonomous driving, and medical image target detection, where there are many target categories, ambiguous semantic boundaries, or the category system needs to be continuously expanded with business iteration. However, the model training method of this disclosure is not limited to the above scenarios. It can be applied to other fields that need to perform target detection tasks, such as industrial quality inspection and security monitoring. It is not limited here.

[0099] <Fourth Embodiment> This disclosure also provides a target detection method for the field of motion control, which performs scene perception inference based on a first target detection model obtained from any of the above embodiments. Figure 7 As shown, the target detection method in this embodiment includes the following steps S710 and S720: Step S710: Acquire scene images captured by the mobile device.

[0100] Mobile devices can be vehicles, drones, robots, etc., equipped with automated motion control systems.

[0101] Step S720: Call the object detection model to perform object detection on the scene image based on the first category set, and obtain the object detection result.

[0102] The target detection results include the category label of the detected target and the location information of the detected target in the scene image.

[0103] The target detection model invoked in step S720 is the target detection model obtained through the model training method of any of the above embodiments. For example, it can be the first target detection model obtained according to the first embodiment, the updated first target detection model obtained according to the third embodiment, or the third target detection model obtained according to the second embodiment.

[0104] In this embodiment, the automated motion control system on the mobile device can control the mobile device to move in the current scene based on the target detection results.

[0105] <Fifth Embodiment> This embodiment provides a computing device, such as Figure 8As shown, the computing device 800 includes a memory 820 and a processor 810. The memory 820 is used to store a computer program executed by the processor 810. The processor 810 is configured to implement the method according to any embodiment of the present disclosure when executing the computer program stored in the memory 820.

[0106] The computing device can be deployed independently of the mobile device, or it can be deployed at least partially on the mobile device.

[0107] The various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0108] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0109] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0110] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.

[0111] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0112] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0113] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0114] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0116] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for training an object detection model, characterized in that, include: Obtain a first mixed sample set; wherein, the first mixed sample set is a multi-scene image data sample set, the first mixed sample set consists of a first sample set and at least two second sample sets, the first sample set is associated with a first category set, the at least two second sample sets are respectively associated with different second category sets, the first category set contains all categories in each second category set; the first sample set is a full category sample set, and its annotation range covers all categories in all second category sets; A first object detection model capable of performing object detection tasks based on the first category set is trained using the first mixed sample set; wherein, each sample set in the first mixed sample set provides targeted supervision signals according to its associated category set during the training, thereby supervising the object detection task.

2. The method according to claim 1, characterized in that, At least some samples in the first sample set are first-class samples. The first-class samples are generated by supplementing the original target annotations with the first-class set based on the sample data in any second sample set.

3. The method according to claim 2, characterized in that, The first category set is the union of all the second category sets. The steps for generating the first category samples include: A subset of samples is extracted from the target sample set to form a target sample subset; wherein, the target sample set is any one of the at least two second sample sets; Determine each non-homologous second target detection model corresponding to the target sample subset; wherein, the non-homologous second target detection model is trained based on other second sample sets different from the target sample set; Each non-homologous second target detection model is used to perform target detection on the sample data in the target sample subset to obtain supplementary target annotations for the corresponding sample data on the second category sets associated with all other second sample sets. For each sample data in the target sample subset, the supplementary target label of the corresponding sample data and the original target label of the corresponding sample data are merged to obtain the first type of sample.

4. The method according to claim 3, characterized in that, The step of merging the supplementary target labels of the corresponding sample data with the original target labels of the corresponding sample data includes: filtering out supplementary target labels in the corresponding sample data whose confidence level is less than a first set threshold, and / or filtering out supplementary target labels in the corresponding sample data that have class conflicts; The supplementary target annotations for filtering out class conflicts in the corresponding sample data include: When the overlap between the detection bounding boxes of the supplementary target annotation and the original target annotation reaches a second preset threshold, and the supplementary target annotation conflicts with the original target annotation in terms of category, the corresponding supplementary target annotation is filtered out; and / or, When the overlap of the detection boxes of two supplementary target labels reaches the third set threshold, and there is a class conflict between the two supplementary target labels, the supplementary target labels with low confidence are filtered out.

5. The method according to any one of claims 1 to 4, characterized in that, After obtaining the first target detection model, the method further includes: Obtain a second mixed sample set; wherein the second mixed sample set consists of at least two second sample sets, a third sample set, and a fourth sample set, the third sample set is associated with a third category set, the fourth sample set is associated with a fourth category set, the categories in the fourth category set do not belong to the first category set, and the third category set includes all categories in the first category set and the fourth category set; Using the second mixed sample set, the first object detection model is used as the initial model for model training to obtain a third object detection model that can perform object detection tasks based on the third category set; wherein, each sample set in the second mixed sample set is supervised for object detection tasks according to its associated category set during model training.

6. The method according to claim 5, characterized in that, At least some samples in the third sample set are second-class samples. The second-class samples are generated by supplementing the fourth-class set with the sample data in the first sample set based on the original target annotations corresponding to the first category set.

7. The method according to any one of claims 1 to 4, characterized in that, After obtaining the first target detection model, the method further includes: Obtain a specified sample; wherein the target category of the specified sample belongs to the second category set; The specified sample is added to the second sample set to form an updated first mixed sample set; The first target detection model is updated by updating the first mixed sample set to obtain the updated first target detection model.

8. A target detection method, characterized in that, include: Acquire scene images captured by mobile devices; The object detection model is invoked to perform object detection on the scene image based on the first category set, and the object detection result is obtained; The target detection model is trained using a first mixed sample set, which consists of a first sample set and at least two second sample sets. The at least two second sample sets are associated with different second category sets, and the first sample set is associated with a first category set. The first category set contains all categories in each second category set. The first sample set is a full category sample set, and its annotation range covers all categories in all second category sets. Each sample set in the first mixed sample set provides targeted supervision signals according to its associated category set during the training, thereby supervising the target detection task.

9. A computing device, characterized in that, include: Memory, used to store computer programs; as well as, A processor configured to implement the method of any one of claims 1 to 8 when executing a computer program stored in the memory.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Heterogeneous data centralized access analysis method based on heterogeneous platform

    CN114154572A

  • Model pre-training method and related equipment

    CN118097179A