A target detection method, device, storage medium, sensor and controller

Abstract

The application belongs to the technical field of artificial intelligence AI (Artificial Intelligence), and particularly relates to a target detection method and device, a storage medium, a sensor and a controller; based on a supervised target detection framework, the learning ability of a category feature vector (Embedding) is strengthened by introducing a self-supervised contrast learning process; and then the feature vectors of the same category are pulled closer while the feature vectors of other categories are pushed away; it is beneficial to distinguish a small number of instances that are difficult to identify in a small data amount scene; at the same time, the target detection method based on the feature contrast learning process also introduces a contrast loss function adaptive to the quality of the candidate frame, which can further improve the target detection effect and improve the convergence speed of the learning process; the product can be used in the field of industrial defect detection, saves manpower and improves the detection speed while ensuring the recall rate; in addition, the method and product can be integrated into an existing target detection system, which is beneficial to upgrading the existing system.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence (AI) technology, and particularly relates to a target detection method, device, storage medium, sensor and controller. Background Technology

[0002] Object detection is an important branch of artificial intelligence (AI). Machine vision based on intelligent algorithms has made great strides as a key support. Related applications have partially or to a certain extent replaced human work in fields such as image classification, object detection, and semantic segmentation.

[0003] Taking industrial applications as an example, deep learning-based automated inspection has been able to partially replace the human visual inspection process, thereby saving human resources, increasing inspection speed, and effectively avoiding detection rate fluctuations caused by human fatigue and emotional fluctuations.

[0004] Unlike conventional object detection, most industrial defect detection tasks require a 100% recall rate for defect categories. The difficulty in identifying industrial defects stems from both the limited sample space and the inherent complexity of the identification process itself.

[0005] For the reasons mentioned above, how to achieve more reliable and stable target detection in scenarios such as limited training data, imbalanced defect categories, and difficulty in distinguishing defects has become a key technical problem that urgently needs to be overcome in the process of industrial defect detection. Summary of the Invention

[0006] This invention discloses a target detection method, including a first positive and negative sample construction step and a second supervised contrastive learning step. The first positive and negative sample construction step obtains a positive and negative sample set based on a label allocation process. This label allocation process determines the anchor boxes used for output results based on the anchor-based structure and assigns labels to the anchor boxes that should output results. In a first training dataset preset during model training, anchor boxes matched by the same type of target are considered positive samples to each other, and anchor boxes matched by different types of targets are considered negative samples to each other.

[0007] Furthermore, its second supervised contrastive learning step uses anchor boxes matched with the actual target for contrastive learning; this contrastive learning process uses the target's category information to assist in representation learning.

[0008] Specifically, in the implementation of this target detection method, the label assignment process assigns multiple anchor boxes to a label for prediction, in order to ensure that the detection rate of the target detection process reaches 100% or as high as possible.

[0009] In the detection process, a decoupling head can be used as the detection head; the decoupling head uses a separate classification feature map to extract feature vectors.

[0010] Specifically, the second supervised contrastive learning step uses a feature pyramid structure FPN, and contrastive learning is performed only within the same layer of the FPN.

[0011] Furthermore, a loss function L is constructed based on the layer index of the feature pyramid, the number of all anchor boxes assigned to the labels, the total number of anchor boxes assigned to all layers of the feature pyramid, and the index of all anchor boxes matching the actual target in the first training dataset, combined with anchor boxes of the same category and the remaining anchor boxes. This loss function L, combined with the quality coefficient Qi, is used to characterize the overlap between the regression coordinates of the anchor box output and the corresponding real region of the target, thereby filtering the detection data. The contrastive learning process of this target detection method can use the following loss function Lcontrasive, such that:

[0012]

[0013] In the formula, l∈L is the layer number of the feature pyramid, Nl is the number of all anchor boxes assigned to the label of that layer, Ntotal is the total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all matched anchor boxes in the first training dataset, P9i) is the anchor box of the same category as i, A(i) is the other anchor box besides i; Qi and Qp are the quality coefficients Qi between 0 and 1; the quality coefficient Qi is used to characterize the overlap between the regression coordinates of the anchor box output and the real region of the corresponding target.

[0014] Furthermore, based on the regression coordinates predicted by the anchor boxes and the true positions of the samples corresponding to the anchor boxes, the optimization process of the quality coefficient Qi is given; where i represents the anchor box number; used to introduce the information of the regression branch of the quality coefficient Qi into the loss function of the contrastive learning, dynamically adjusting the weights between different positive samples; wherein, this target detection method can also introduce a third sample quality screening step; the third sample quality screening step uses the following quality coefficient Qi; such that:

[0015]

[0016] In the formula, i is the number of the corresponding anchor box, Pi is the regression coordinate predicted based on the anchor box, and Gi is the true position of the sample corresponding to the anchor box; its quality coefficient Qi introduces the information of the regression branch into the loss function of contrastive learning, and dynamically adjusts the weights between different positive samples.

[0017] Furthermore, a detection box category loss function and a coordinate loss function are employed for the learning process; the detection box category loss function and the coordinate loss function are integrated with the loss function L to construct a global loss function, thereby superimposing the detection box information and coordinate information according to preset weights; wherein, the learning process may also employ a detection box category loss function Lcls and a coordinate loss function Lcor; such that the loss function Loveraall for the target detection process satisfies:

[0018] Loverall=wcls·Lcls+wcor·Lcor+wcontrasive·Lcontrasive;

[0019] In the formula, wi represents the weight of each loss function.

[0020] Accordingly, this invention also discloses a target detection device, including a first positive and negative sample construction unit and a second supervised contrastive learning unit; the first positive and negative sample construction unit obtains a set of positive and negative samples based on a label allocation process, the label allocation process is based on anchor points of an anchor-free structure to allocate labels, or the anchor boxes used for output results can be determined in an anchor-based structure and labels can be assigned to the anchor boxes that should output results; in a first training dataset preset during model training, the anchor boxes matched by the same type of target are positive samples to each other, and the anchor boxes matched by different types of targets are negative samples to each other.

[0021] The second supervised contrastive learning unit uses anchor boxes matched with the actual target for contrastive learning; its contrastive learning process uses the target's category information to assist in representation learning.

[0022] Similarly, to improve the detection rate, the label assignment process can assign multiple anchor boxes to a single label for prediction. Specifically, a decoupling head can be used as the detection head; this type of decoupling head extracts feature vectors using a separate classification feature map. Wherein: if the second supervised contrastive learning unit uses a Feature Pyramid Network (FPN) structure, then the contrastive learning is performed only within the same layer of the FPN.

[0023] Furthermore, a loss function L is constructed based on the layer index of the feature pyramid, the number of all anchor boxes assigned to the labels, the total number of anchor boxes assigned to all layers of the feature pyramid, and the index of all matched anchor boxes in the first training dataset, combined with anchor boxes of the same category and the remaining anchor boxes. This loss function L, combined with the quality coefficient Qi, is used to characterize the overlap between the regression coordinates of the anchor box output and the corresponding real region of the target, thereby filtering the detection data. The contrastive learning can employ the following loss function Lcontrasive, such that:

[0024]

[0025] In the formula, l∈L is the layer number of the feature pyramid, Nl is the number of all anchor boxes assigned to the label of that layer, Ntotal is the total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all matched anchor boxes in the first training dataset, P(i) is the anchor box of the same category as i, and A(i) is the other anchor box besides i; Qi and Qp are the quality coefficients Qi between 0 and 1; the quality coefficient Qi is used to characterize the degree of overlap between the regression coordinates of the anchor box output and the corresponding real area of ​​the target.

[0026] Furthermore, based on the regression coordinates predicted by the anchor boxes and the true positions of the samples corresponding to the anchor boxes, the optimization process of the quality coefficient Qi is given; where i represents the anchor box number; used to introduce the information of the regression branch of the quality coefficient Qi into the loss function of the contrastive learning, dynamically adjusting the weights between different positive samples; wherein, the target detection device can also be equipped with a third sample quality screening unit, and the following quality coefficient Qi is introduced; such that:

[0027]

[0028] In the formula, i is the number of the corresponding anchor box, Pi is the regression coordinate predicted based on the anchor box, and Gi is the true position of the sample corresponding to the anchor box; the quality coefficient Q introduces the information of the regression branch into the loss function of contrastive learning, and dynamically adjusts the weights between different positive samples.

[0029] Specifically, a detection box category loss function and a coordinate loss function may also be used in the learning process; the detection box category loss function and the coordinate loss function are integrated with the loss function L to construct a global loss function, thereby superimposing the detection box information and coordinate information according to preset weights; wherein, the learning process may also use a detection box category loss function Lcls and a coordinate loss function Lcor; such that the loss function Loveraall of the target detection process satisfies:

[0030] Loverall=wcls·Lcls+wcor·Lcor+wcontrasive·Lcontrasive;

[0031] In the formula, wi represents the weight of each loss function.

[0032] Accordingly, the above methods and products can also be implemented in computer storage media, sensors, or controllers.

[0033] Specifically, its computer storage medium can be configured as a storage medium body for storing computer programs; when its computer program is executed by a microprocessor, it can implement any of the above target detection methods; its sensor can be configured as any of the above target detection devices and / or computer storage media, and implement the same detection scheme; the implementation process of its controller is similar to that of the sensor; it will not be described in detail here.

[0034] In summary, this invention, based on a supervised object detection framework, enhances the learning ability of category feature vectors (Embeddings) by introducing a self-supervised contrastive learning process. This brings feature vectors of the same category closer together while pushing away feature vectors of other categories. This is beneficial for distinguishing a small number of instances that are difficult to identify in scenarios with limited data. Furthermore, the object detection method based on the feature contrastive learning process introduces a contrastive loss function that adapts to candidate box quality, further improving object detection performance and accelerating the convergence of the learning process. Its products can be used in the field of industrial defect detection, ensuring recall while saving manpower and increasing detection speed. In addition, its methods and products can be integrated into existing object detection systems, facilitating upgrades to existing systems.

[0035] It should be noted that the terms "first," "second," and similar terms used in this article are merely for describing the constituent elements of the technical solution and do not constitute a limitation on the technical solution, nor should they be interpreted as an indication or implication of the importance of the corresponding elements; elements with terms such as "first," "second," or similar terms indicate that at least one of the elements is included in the corresponding technical solution. Attached Figure Description

[0036] To more clearly illustrate the technical solution of the present invention and facilitate a further understanding of its technical effects, features, and objectives, the present invention will be described in detail below with reference to the accompanying drawings. The drawings constitute an essential part of the specification and are used together with Embodiment 1 of the present invention to illustrate the technical solution of the present invention, but do not constitute a limitation on the present invention.

[0037] The same reference numerals in the attached diagrams represent the same parts, specifically:

[0038] Figure 1 This is a schematic diagram comparing the detection processes of the coupling head and the decoupling head.

[0039] Figure 2 This is a schematic diagram of the positive and negative sample construction process based on label allocation in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram comparing the anchoring quality of an embodiment of the present invention.

[0041] Figure 4This is an example of a defect category image from the LPS (Low Pressure Sensor) dataset of the present invention.

[0042] Figure 5 This is an example of comparing the visualization results of principal component analysis (PCA) in the LPS data feature implementation of the present invention.

[0043] Figure 6 The results of the PCA variance interpretability comparison of LPS data features in embodiments of the present invention are shown.

[0044] Figure 7 This is a comparison of the embodiments of the present invention with existing methods.

[0045] Figure 8 This is an example of comparing the improvement in target detection capability with a small number of samples in the embodiments of the present invention.

[0046] Figure 9 This is a schematic diagram of the process of an embodiment of the method of the present invention.

[0047] Figure 10 This is a schematic diagram of the structural composition of an embodiment of the device of the present invention.

[0048] Figure 11 This is a schematic diagram of the layout structure of an embodiment of the product of the present invention. Figure 1 .

[0049] Figure 12 This is a schematic diagram of the layout structure of an embodiment of the product of the present invention. Figure 2 .

[0050] Figure 13 This is a schematic diagram of the layout structure of an embodiment of the product of the present invention. Figure 3 .

[0051] in:

[0052] 010 - Detailed feature data;

[0053] 110-Coupled Head (CH) shares the Feature Map (FM) mode;

[0054] 111 - First Detection Feature Set (DF);

[0055] 113 - First and third detection result set DR (Detection Result);

[0056] 120-Decoupled Head (DH) Classification Feature Map CFM (Classified FM) Pattern;

[0057] 121 - Second Classifier Feature Set CF21;

[0058] 123 - Second and third position feature set PF23;

[0059] 125 - Second and Fifth Test Results;

[0060] 127 - Data for the second and seventh objects;

[0061] 129 - Data for the 29th object;

[0062] 210 - Mini-batch (MiB) input data during training;

[0063] Schematic diagram of anchor boxes (AB Anchor Boxes) matching different targets in feature map FM (220-).

[0064] 230 - Schematic diagram of the positive and negative sample generation process;

[0065] 311, 312, 313, 314, 315, 316 - Third type of anchoring frame;

[0066] 321, 322 - Third type II anchoring frames;

[0067] 410 - LPS dataset to be detected;

[0068] Magnified view of the 411-LPS dataset Figure 1 ;

[0069] Magnified view of the 413-LPS dataset Figure 3 ;

[0070] 420-Bend image sample of bond wire;

[0071] 430 - Image sample with positional deviation;

[0072] 440 - Sample images of contaminated parts;

[0073] The 501-LPS data is a visualization of the first layer anchor features of the Feature Pyramid Networks (FPN) matched by the original X version of the YoloX (You Only Look Once X) object detection method.

[0074] Visualization of FPN second-layer Anchor features matched using the original YoloX method in 502-LPS data;

[0075] Visualization of FPN third-layer Anchor features matched using the original YoloX method in 503-LPS data;

[0076] 511-LPS data is a visualization of the first-layer Anchor features of the FPN matched by embodiments of the present invention;

[0077] 512-LPS data is a visualization of the second-layer Anchor features of FPN matched by embodiments of the present invention;

[0078] 513-LPS data is a visualization of the third-layer Anchor features of FPN matched by embodiments of the present invention;

[0079] 610 - First Sample Construction Steps;

[0080] 620 - Second comparative learning step;

[0081] 630 - Third Tag Optimization Steps;

[0082] 700 - Target Detection Device;

[0083] 710 - First Sample Construction Unit;

[0084] 720 - Second Contrastive Learning Unit;

[0085] 730 - Third Label Optimization Unit;

[0086] 800-sensor;

[0087] 900 - Vehicles;

[0088] 901 - Controller;

[0089] 903 - Computer storage media. Detailed Implementation

[0090] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described below are merely illustrative of the technical solutions of the present invention, and not intended to limit the invention. Furthermore, the parts described in the embodiments or drawings are merely illustrative examples of relevant parts of the present invention, and not the entirety of the invention.

[0091] like Figure 9The target detection method shown includes a first positive and negative sample construction step 610 and a second supervised contrastive learning step 620. The first positive and negative sample construction step 610 obtains a set of positive and negative samples based on a label assignment process. The label assignment process determines the anchor boxes used for outputting results within pre-placed anchor boxes and assigns labels to the anchor boxes that should output results. During model training, in a pre-set first training dataset, anchor boxes matched by targets of the same class are considered positive samples, while anchor boxes matched by targets of different classes are considered negative samples. The second supervised contrastive learning step 620 uses anchor boxes matched with actual targets for contrastive learning. This contrastive learning process uses target category information to assist in representation learning.

[0092] Specifically, the label assignment process can assign multiple anchor boxes to a label for prediction; a decoupling head can be used as the detection head; the decoupling head uses a separate classification feature map to extract feature vectors.

[0093] Furthermore, if the second supervised contrastive learning step 620 adopts the Feature Pyramid (FPN) structure, then contrastive learning is performed only within the same layer of the FPN.

[0094] Specifically, a loss function L is constructed based on the layer index of the feature pyramid, the number of all anchor boxes assigned to the labels, the total number of anchor boxes assigned to all layers of the feature pyramid, and the index of all matched anchor boxes in the first training dataset, combined with anchor boxes of the same category and the remaining anchor boxes. This loss function L, along with the quality coefficient Qi, is used to characterize the overlap between the regression coordinates of the anchor box output and the corresponding ground truth region of the target, thereby filtering the detection data. The contrastive learning can employ the following loss function Lcontrasive, such that:

[0095]

[0096] In the formula, l∈L is the layer number of the feature pyramid, Nl is the number of all anchor boxes assigned to the label of that layer, Ntotal is the total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all matched anchor boxes in the first training dataset, P(i) is the anchor box of the same category as i, and A(i) is the other anchor box besides i; Qi and Qp are the quality coefficients Qi between 0 and 1; the quality coefficient Qi is used to characterize the degree of overlap between the regression coordinates of the anchor box output and the real region of the corresponding target.

[0097] Furthermore, based on the regression coordinates predicted by the anchor boxes and the true positions of the samples corresponding to the anchor boxes, the optimization process of the quality coefficient Qi is given; where i represents the anchor box number; used to introduce the information of the regression branch of the quality coefficient Qi into the loss function of the contrastive learning, dynamically adjusting the weights between different positive samples; wherein, the target detection method also includes a third sample quality screening step 630; and introduces the following quality coefficient Qi; such that:

[0098]

[0099] In the formula, i is the number of the corresponding anchor box, Pi is the regression coordinate predicted based on the anchor box, and Gi is the true position of the sample corresponding to the anchor box; its quality coefficient Q introduces the information of the regression branch into the loss function of contrastive learning, and dynamically adjusts the weights between different positive samples.

[0100] Specifically, a detection box category loss function and a coordinate loss function may also be used in the learning process; the detection box category loss function and the coordinate loss function are integrated with the loss function L to construct a global loss function, thereby superimposing the detection box information and coordinate information according to preset weights; wherein, the learning process may also employ a detection box category loss function Lcls and a coordinate loss function Lcor; such that the loss function Loveraall of the target detection process satisfies:

[0101] Loverall=wcls·Lcls+wcor·Lcor+wcontrasive·Lcontrasive;

[0102] In the formula, wi represents the weight of each loss function.

[0103] Accordingly, such as Figure 10 The target detection device 700 shown includes a first positive and negative sample construction unit 710 and a second supervised contrastive learning unit 720. The first positive and negative sample construction unit 710 obtains a set of positive and negative samples based on a label allocation process. The label allocation process determines the anchor boxes used for outputting results in a pre-placed anchor box and assigns labels to the anchor boxes that should output results. In the first training dataset preset during model training, the anchor boxes matched by the same type of target are positive samples to each other, and the anchor boxes matched by different types of targets are negative samples to each other.

[0104] The second supervised contrastive learning unit 720 uses anchor boxes matched with the actual target for contrastive learning; this contrastive learning process uses the target's category information to assist in representation learning.

[0105] To ensure a high detection rate, the label assignment process can assign multiple anchor boxes to a single label for prediction. Furthermore, to avoid conflicts caused by shared feature maps between classification and regression tasks, a decoupling head can be used as the detection head; this decoupling head extracts feature vectors from a separate classification feature map.

[0106] Furthermore, if the second supervised contrastive learning unit 720 adopts a feature pyramid structure FPN, then contrastive learning is performed only within the same layer of the FPN.

[0107] Specifically, a loss function L is constructed based on the layer index of the feature pyramid, the number of all anchor boxes assigned to the labels, the total number of anchor boxes assigned to all layers of the feature pyramid, and the index of all matched anchor boxes in the first training dataset, combined with anchor boxes of the same category and the remaining anchor boxes. This loss function L, combined with the quality coefficient Qi, is used to characterize the overlap between the regression coordinates of the anchor box output and the corresponding real region of the target, thereby filtering the detection data. The contrastive learning process can use the following loss function Lcontrasive, such that:

[0108]

[0109] In the formula, l∈L is the layer number of the feature pyramid, Nl is the number of all anchor boxes assigned to the label of that layer, Ntotal is the total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all matched anchor boxes in the first training dataset, P(i) is the anchor box of the same category as i, and A(i) is the other anchor box besides i; Qi and Qp are the quality coefficients Qi between 0 and 1; the quality coefficient Qi is used to characterize the degree of overlap between the regression coordinates of the anchor box output and the corresponding real area of ​​the target.

[0110] Furthermore, based on the regression coordinates predicted by the anchor boxes and the true positions of the samples corresponding to the anchor boxes, an optimization process for the quality coefficient Qi is given; where i represents the anchor box number; this is used to introduce the information of the regression branch of the quality coefficient Qi into the loss function of the contrastive learning, dynamically adjusting the weights between different positive samples; wherein, the target detection device 700 is also provided with a third sample quality screening unit 730, and the following quality coefficient Qi is introduced, such that:

[0111]

[0112] In the formula, i is the number of the corresponding anchor box, Pi is the regression coordinate predicted based on the anchor box, and Gi is the true position of the sample corresponding to the anchor box; its quality coefficient Q introduces the information of the regression branch into the loss function of contrastive learning, and dynamically adjusts the weights between different positive samples.

[0113] Specifically, a detection box category loss function and a coordinate loss function may also be used in the learning process; the detection box category loss function and the coordinate loss function are integrated with the loss function L to construct a global loss function, thereby superimposing the detection box information and coordinate information according to preset weights; wherein, the learning process may also use a detection box category loss function Lcls and a coordinate loss function Lcor; such that the loss function Loveraall of the target detection process satisfies:

[0114] Loverall=wcls·Lcls+wcor·Lcor+wcontrasive·Lcontrasive;

[0115] In the formula, wi represents the weight of each loss function.

[0116] In practical applications, the object detection method disclosed in this invention introduces self-supervised contrastive learning into the existing supervised object detection algorithm framework, thereby enhancing the algorithm's ability to learn class feature vectors (Embedding), bringing feature vectors of the same class closer together while pushing feature vectors of other classes further apart, making it easier for the algorithm to distinguish difficult cases and perform better with a small amount of data.

[0117] Meanwhile, based on this object detection method that introduces a feature contrast learning process, the object detection effect is improved by using a supervised, candidate box quality-adaptive contrast loss function.

[0118] Furthermore, the framework of the method of the present invention can continue to incorporate most existing target detection processes without requiring modifications to the original algorithms.

[0119] This invention includes a target detection algorithm framework based on supervised contrastive learning. Unlike classic self-supervised contrastive learning methods that primarily rely on image classification data for representation learning and then freeze weights for downstream tasks, this invention avoids using the same image enhancement method to generate positive sample pairs or assigning samples from other images as negative samples. The goal of contrastive learning in this invention is to narrow the gap between positive samples while simultaneously widening the gap between positive and negative samples. Specifically, for target detection tasks, this invention addresses situations where multiple targets exist in an image and their categories are uncertain. Furthermore, it requires locating the target positions. Therefore, this invention improves upon existing supervised contrastive learning-based target detection algorithm frameworks.

[0120] By introducing a decoupled head (DH), the efficiency of feature extraction in the detection algorithm is improved, avoiding the conflict problem caused by the shared feature map between classification and regression tasks in a coupled detection head.

[0121] Specifically, as follows: Figure 1 The non-decoupled head, i.e. the coupled head CH (coupled head), uses the same feature map FM (feature map) mode 110 for regression and classification. In contrast, the decoupled head DH (decoupled head), i.e. the decoupled head CFM (classified FM) mode 120, uses a separate classification feature map, which can be used to extract feature vectors.

[0122] Furthermore, such as Figure 2 The diagram illustrates the process of constructing positive and negative samples based on label assignment. Typically, existing contrastive learning methods construct positive and negative samples based on classification tasks, where positive samples are the same image enhanced in two different ways, and negative samples are images other than the image itself. However, this embodiment uses a positive and negative sample construction method based on object detection label assignment.

[0123] Specifically, since the location and number of targets in object detection tasks are often uncertain, existing algorithms mainly place many anchor boxes in advance on the image, while the label assignment process is used to determine the anchor boxes that should be output and assign them corresponding labels.

[0124] Among them, such as Figure 2 As shown, in order to ensure the detection rate of the algorithm, multiple anchor boxes are often assigned to a label for prediction during label allocation. The positive and negative sample construction method in this embodiment is based on the label allocation result. In a batch of images during training, anchor boxes matched by the same type of target are considered positive samples, while anchor boxes matched by different types of targets are considered negative samples.

[0125] Specifically, for algorithms using the Feature Pyramid Network (FPN) structure, the contrastive learning is performed only within the same layer.

[0126] Furthermore, this embodiment introduces a supervised contrastive learning loss function based on anchor box quality.

[0127] Among them, such as Figure 3As shown, its positive and negative sample construction method uses all anchor boxes matched with the actual target for comparative learning; however, since the label assignment usually assigns as many anchor boxes as possible to the target to ensure the detection rate, specifically, the target in the figure is matched with 8 anchor boxes 311 to 316, namely anchor boxes 321 and 322; but the difficulty of prediction based on different anchor boxes is significantly different; among them, anchor boxes 321 and 322 are more representative of the target than other anchor boxes; other positions, such as the anchor box 312 in the upper left corner, have a very small overlap area with the target, which leads to possible quality differences based on the positive and negative sample construction method mentioned above, which may cause the network to fail to converge during training.

[0128] Meanwhile, the object detection algorithm framework based on contrastive learning uses the category information of the target to assist in representation learning compared with traditional contrastive learning methods, which makes the similarity of representations of the same class higher and the similarity between different classes lower.

[0129] Therefore, this embodiment discloses a supervised contrastive learning loss function Lcontrasive based on anchor box quality, such that:

[0130]

[0131] In the formula, l∈L is the layer number of the feature pyramid, Nl is the number of all anchor boxes assigned to the label of the layer, Ntotal is the total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all matched anchor boxes in the entire dataset, P(i) is the anchor box of the same category as i, and A(i) is the other anchor box besides i.

[0132] In the loss function, Qi∈[0,1] represents the quality coefficient corresponding to the anchor box, which is expressed by the overlap between the regression coordinates output by the anchor box and the corresponding target ground truth region, i.e.:

[0133]

[0134] In the formula, i is the corresponding anchor box number, Pi is the regression coordinate predicted based on the anchor box, and Gi is the true position of the sample corresponding to the anchor box. That is, the quality coefficient Qi can introduce the information of the coordinate regression branch in the object detection algorithm into the contrastive learning loss function, dynamically adjusting the weights between different positive samples and helping the algorithm converge.

[0135] Therefore, its overall loss function can be expressed as:

[0136] Loverall=wcls·Lcls+wcor·Lcor+wcontrasive·Lcontrasive;

[0137] Where Lcls and Lcor are the loss functions for the detection box categories and coordinates in the object detection process, and wi is the weight of each loss function.

[0138] like Figure 5 As shown, the visualization results of Principal Component Analysis (PCA) of the LPS data feature embodiment are presented. Based on the You Only Look Once YoloX object detection method, the structure of this embodiment is used, and validation tests are performed on two component defect detection datasets, LPS (966 training images, 119 validation images) and DSS (467 training images, 89 validation images).

[0139] Among them, such as Figure 4 The two industrial testing datasets shown have fewer data volumes and are more difficult to distinguish between categories compared to publicly available datasets, further verifying the effectiveness of this embodiment.

[0140] Specifically, such as Figure 5 As shown, taking the LPS (Low Pressure Sensor) dataset as an example, its feature visualization diagram shows the effect of this embodiment on representation learning: two models, the original YoloX and the method of this embodiment, were trained based on the LPS dataset, and the anchor box category features matched under the three-layer feature pyramid in YoloX were visualized by PCA respectively.

[0141] The corresponding PCA variance interpretability is as follows: Figure 6 As shown, this embodiment can make the representations of the same category more similar; at the same time, it can extend the features of different categories; the effect is more obvious in the first two feature pyramid layers, making it easier to distinguish different target categories.

[0142] like Figure 7 As shown, comparative data on target detection performance are also provided, and the results are verified on two product defect detection datasets. It can be seen that this embodiment improves the detection accuracy to varying degrees, especially for categories that are difficult to distinguish in the dataset represented by HardClass, where significant improvements have been achieved.

[0143] In addition, such as Figure 8 As shown, the performance based on the DSS dataset is further presented; for the few-shot object detection task, only half of the data was used for training and validation; the figure shows the accuracy of the difficult example categories. It can be seen that in the few-shot object detection task, this embodiment can still help the algorithm improve accuracy, and the accuracy improvement is greater for the more difficult-to-distinguish categories.

[0144] Furthermore, the quality of the anchor box can be incorporated into the label assignment stage to add a new optimization term for building a new label assignment method.

[0145] Correspondingly, such as Figures 11 to 13 Embodiments and corresponding integrated structures of computer storage medium 903, sensor 800, and controller 901 are also provided; wherein, computer storage medium 903 includes a storage medium body for storing computer programs; when the computer program is executed by a microprocessor, it can implement any of the above target detection methods; its sensor 800 includes any of the above target detection devices 700 and / or computer storage medium 903; the implementation method of its controller 901 is similar and will not be described in detail.

[0146] It should be noted that the above embodiments are only for more clearly illustrating the technical solution of the present invention. Those skilled in the art will understand that the implementation of the present invention is not limited to the above content. Any obvious changes, substitutions or replacements made based on the above content do not exceed the scope of the technical solution of the present invention. Other implementations will also fall within the scope of the present invention without departing from the concept of the present invention.

Claims

1. A target detection method, characterized in that, The model includes a first positive and negative sample construction step (610) and a second supervised contrastive learning step (620). The first positive and negative sample construction step (610) obtains a set of positive and negative samples based on a label assignment process. This label assignment process determines the anchor boxes used for outputting results based on the anchor-based structure and assigns labels to the anchor boxes that should output results. During model training, in a pre-set first training dataset, anchor boxes matched by targets of the same class are considered positive samples, while anchor boxes matched by targets of different classes are considered negative samples. The second supervised contrastive learning step (620) uses the anchor boxes matched with actual targets for contrastive learning. This contrastive learning process uses the target's category information to assist in representation learning. The second supervised contrastive learning step (620) adopts a feature pyramid structure FPN, and the contrastive learning is performed only within the same layer of the FPN; The contrastive learning process constructs a loss function based on the layer number of the feature pyramid, the number of all anchor boxes assigned to that layer's label, the total number of anchor boxes assigned across all layers of the feature pyramid, and the indices of all anchor boxes in the first training dataset that match the actual target, combined with anchor boxes of the same category and the remaining anchor boxes. , In the formula, l∈L is the layer number of the feature pyramid, and N l N represents the number of all anchor boxes assigned to this layer's label. total The total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all anchor boxes that match the actual target in the first training dataset, P(i) is the anchor box of the same category as i, and A(i) is the other anchor box besides i. and The quality coefficient is between 0 and 1; this loss function Combined with quality coefficient quality coefficient This is used to characterize the overlap between the regression coordinates output by the anchor box and the corresponding real area of ​​the target, thereby filtering the detection data; It also includes a third sample quality screening step (630); the third sample quality screening step (630) gives the quality coefficient based on the regression coordinates predicted by the anchor box and the true position of the sample corresponding to the anchor box. The optimization process; where i represents the anchor box number; used to optimize the quality coefficient. Information from the regression branch is incorporated into the loss function of the contrastive learning, dynamically adjusting the weights between different positive samples; It also includes a bounding box category loss function and a coordinate loss function for the contrastive learning process; the bounding box category loss function and the coordinate loss function are used to sum with the loss function. A global loss function is constructed, and then the detection box information and coordinate information are superimposed according to the preset weights.

2. The target detection method as described in claim 1, wherein: The label assignment process assigns multiple anchor boxes to a label for prediction.

3. The target detection method as described in claim 2, wherein: A decoupling head is used as the detection head; the decoupling head extracts feature vectors using a separate classification feature map.

4. A target detection device (700), comprising a first positive and negative sample construction unit (710) and a second supervised contrastive learning unit (720); wherein, The first positive and negative sample construction unit (710) obtains a positive and negative sample set based on the label allocation process. The label allocation process determines the anchor boxes used for outputting results based on the anchor box based structure and assigns labels to the anchor boxes that should output results. In the first training dataset preset during model training, the anchor boxes matched by the same type of target are positive samples to each other, and the anchor boxes matched by different types of targets are negative samples to each other. The second supervised contrastive learning unit (720) performs contrastive learning using the anchor boxes matched with the actual targets. The contrastive learning process uses the target's category information to assist in representation learning. The second supervised contrastive learning unit (720) adopts a feature pyramid structure FPN and performs the contrastive learning only within the same layer of the FPN; The contrastive learning process constructs a loss function based on the layer number of the feature pyramid, the number of all anchor boxes assigned to that layer's label, the total number of anchor boxes assigned across all layers of the feature pyramid, and the indices of all anchor boxes in the first training dataset that match the actual target, combined with anchor boxes of the same category and the remaining anchor boxes. , In the formula, l∈L is the layer number of the feature pyramid, and N l N represents the number of all anchor boxes assigned to this layer's label. total The total number of anchor boxes assigned to all layers of the feature pyramid, i∈I is the index of all anchor boxes that match the actual target in the first training dataset, P(i) is the anchor box of the same category as i, and A(i) is the other anchor box besides i. and The quality coefficient is between 0 and 1; this loss function Combined with quality coefficient quality coefficient This is used to characterize the overlap between the regression coordinates output by the anchor box and the corresponding real area of ​​the target, thereby filtering the detection data; It also includes a third sample quality screening unit (730); the third sample quality screening unit (730) provides the quality coefficient based on the regression coordinates predicted by the anchor box and the true position of the sample corresponding to the anchor box. The optimization process; where i represents the anchor box number; used to optimize the quality coefficient. The regression branch information is incorporated into the loss function of the contrastive learning to dynamically adjust the weights between different positive samples; It also employs a bounding box category loss function and a coordinate loss function for comparing the learning process; the bounding box category loss function and the coordinate loss function are used to compare with the loss function. A global loss function is constructed, and then the detection box information and coordinate information are superimposed according to the preset weights.

5. The target detection device (700) as described in claim 4, wherein: The label assignment process assigns multiple anchor boxes to a label for prediction.

6. The target detection device (700) as described in claim 5, wherein: A decoupling head is used as the detection head; the decoupling head extracts feature vectors using a separate classification feature map.

7. A computer storage medium (903) comprising a storage medium body for storing a computer program; wherein the computer program, when executed by a microprocessor, implements the target detection method as described in any one of claims 1 to 3.

8. A sensor (800) comprising a target detection device (700) as claimed in any one of claims 4 to 6 and / or a computer storage medium (903) as claimed in claim 7.

9. A controller (901) comprising a target detection device (700) as claimed in any one of claims 4 to 6; and / or a computer storage medium (903) as claimed in claim 7; and / or a sensor (800) as claimed in claim 8.

Images