Target detection method and system

Abstract

A method and system for object detection. The method includes: after receiving an input image, performing object detection on the input image using two or more pre-set detection network models; fusing the intermediate detection results output by each of the detection network models to obtain a target object detection result; the intermediate detection result includes target object category information, target object location information, and confidence information; wherein, the training samples corresponding to any two detection network models are of different categories. Application of this solution can improve the accuracy of target detection.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a target detection method and system. Background Art

[0002] Deep learning, a prototype that uses computer system architecture to simulate the human brain, is widely used in fields such as security inspection. The main application of deep learning in security inspection is object detection, which involves detecting the location and classification of objects in input images.

[0003] Existing deep learning-based target detection solutions usually use a single convolutional neural network (CNN) or a fully connected deep neural network (DNN) for target detection.

[0004] However, the accuracy of target detection using the above scheme is poor. Summary of the Invention

[0005] The problem to be solved by the present invention is to improve the accuracy of target detection.

[0006] To solve the above problems, an embodiment of the present invention provides a target detection method, which includes:

[0007] After receiving the input image, use two or more preset detection network models to detect the target object in the input image;

[0008] The intermediate detection results output by each detection network model are integrated to obtain the detection result of the target object; the intermediate detection result includes: target object category information, target object location information and confidence information;

[0009] The categories of the training samples corresponding to any two detection network models are different.

[0010] Optionally, the categories of the training samples corresponding to any two of the detection network models are completely different, or partially different.

[0011] Optionally, the difference in the number of categories of training samples corresponding to any two of the detection network models is less than a preset first category difference; in the training samples corresponding to each of the detection network models, the difference in the number of training samples of any two categories is less than a first sample difference threshold.

[0012] Optionally, the two or more detection network models are trained by the following method:

[0013] Count the number of training samples in each category among all training samples, and determine the first category with the most training samples and the second category with the least training samples;

[0014] The training sample interval defined by the number of training samples of the first category to the number of training samples of the second category is evenly divided into K parts to obtain K-1 sample number separation values, where K is a positive integer greater than 1;

[0015] Based on the K-1 sample quantity separation value and the number of training samples in each category, the categories of the obtained training samples are separated to obtain K training samples of the detection network model and then train them.

[0016] Optionally, fusing the intermediate detection results output by each of the detection network models to obtain the detection result of the target object includes:

[0017] When the target item category information of the same target item exists in only one of the intermediate detection results, the intermediate detection result is used as the detection result of the target item.

[0018] When the target item category information of the same target item exists in two or more intermediate detection results, the detection result of the target item is obtained by combining the target item location information and confidence information in the two or more intermediate detection results.

[0019] Optionally, combining the target object location information and the confidence information in the two or more intermediate detection results to obtain the detection result of the target object includes:

[0020] For any intermediate detection result with a non-lowest confidence level among the two or more intermediate detection results, respectively calculate the first area, and perform the following operations on each of the first areas: calculate the overlapping area between the first area and each second area, and perform a rejection operation on the corresponding intermediate detection result based on the size of the overlapping area, and use the intermediate detection result after the rejection operation among the two or more intermediate detection results as the detection result of the target object in the current round;

[0021] The detection result of the target item in each round and the intermediate detection result with the lowest confidence among the two or more intermediate detection results are collectively used as the detection result of the target item;

[0022] Among them, the first area is the target item area corresponding to the target item position information of the intermediate detection result, the second area is the target item area corresponding to the target item position information of the remaining intermediate detection results among the two or more intermediate detection results, and the confidence level of the intermediate detection result where the first area is located is higher than the confidence level of the intermediate detection result where the second area is located.

[0023] Optionally, the removing of corresponding intermediate detection results according to the size of the overlapping area includes:

[0024] Based on the size of the overlapping area, determining the difference between the area of ​​each target object and the overlapping area in the two corresponding intermediate detection results;

[0025] When the difference between the target object area and the overlapped area is smaller than a first area threshold, performing a rejection operation on the intermediate detection results where the target object area difference between the target object area and the overlapped area is smaller than the first area threshold;

[0026] When the difference between the target object area and the overlapping area is greater than the first area threshold, a rejection operation is performed based on the difference between the target object areas in the two corresponding intermediate detection results.

[0027] Optionally, performing a rejection operation based on a difference between target object areas in the two corresponding intermediate detection results includes:

[0028] Calculating the difference between the target object areas in the two corresponding intermediate detection results;

[0029] When the difference between the areas of the target objects is smaller than the second area threshold, a rejection operation is performed on the intermediate detection results with lower confidence.

[0030] Optionally, performing a rejection operation based on a difference between target object areas in the two corresponding intermediate detection results further includes:

[0031] The target object position information of the discarded intermediate detection results is used to update the target object position information of the retained intermediate detection results.

[0032] Optionally, each first area in the intermediate detection results with a non-lowest confidence level is calculated in descending order of confidence level, and a elimination operation is performed.

[0033] Optionally, the method further includes:

[0034] The detection results of the target items are classified using a preset classification network model to obtain classification results of the target items; the classification results of the target items include: target item category information and confidence information.

[0035] Optionally, the classifying the detection result of the target object using a preset classification network model to obtain the classification result of the target object includes:

[0036] When the detection result of the target object is received, the detection result of the target object is classified using two or more preset classification network models;

[0037] fusing the intermediate classification results output by each of the classification network models to obtain a classification result of the target object;

[0038] The number of the classification network models is more than two, and the categories of the training samples corresponding to any two of the classification network models are different.

[0039] Optionally, the categories of the training samples corresponding to any two of the classification network models are completely different, or partially different.

[0040] Optionally, in the training samples corresponding to each of the classification network models, the difference in the number of training samples between any two categories is less than a second sample difference threshold.

[0041] Optionally, the training samples corresponding to the two or more classification network models are determined by the following method:

[0042] Selecting training samples with the same visual features from all training samples, and using the training samples with the same visual features as training samples corresponding to the same classification network model;

[0043] Selecting training samples with the same item usage characteristics from the remaining training samples after the selection based on the visual features, and using the training samples with the same item usage characteristics as training samples corresponding to the same classification network model;

[0044] From the remaining training samples after selection according to the visual features and the usage characteristics of the items, training samples with the same item material are selected, and the training samples with the same item material are used as training samples corresponding to the same classification network model.

[0045] Optionally, the training samples corresponding to the two or more classification network models are further determined by the following method:

[0046] Two categories of training samples with different visual features but a difference in the number of training samples less than a preset third sample difference threshold are used as training samples corresponding to the same classification network model; the third sample difference threshold is less than the second sample difference threshold.

[0047] Optionally, fusing the intermediate classification results output by each of the classification network models to obtain the classification result of the target item includes:

[0048] When the target item category information of the same target item exists in only one intermediate classification result, the intermediate classification result is used as the classification result of the target item;

[0049] When the target item category information of the same target item exists in only two intermediate classification results, the intermediate classification result with a higher confidence level among the two intermediate classification results is used as the classification result of the target item;

[0050] When the target item category information of the same target item exists in more than three intermediate classification results, a preset number of intermediate classification results are selected as the classification result of the target item based on the confidence information in the three or more intermediate classification results.

[0051] Optionally, the method further includes: using the classification result of the target item to determine whether the detection result of the target item is valid, and using the valid detection result of the target item as the identification result of the target item.

[0052] Optionally, the determining whether the detection result of the target object is valid by using the classification result of the target object includes:

[0053] Using the preset first training sample category mapping relationship information, performing category conversion on the target item category information in the detection result of the target item, and judging whether the detection result of the target item is valid based on the category conversion result, thereby obtaining an identification result of the target item;

[0054] Among them, the first training sample category mapping relationship information is the correspondence relationship information between the category of the same training sample when it is used as the training sample corresponding to the detection network model and the category of the same training sample when it is used as the training sample corresponding to the classification network model.

[0055] Optionally, judging whether the detection result of the target object is valid based on the category conversion result includes:

[0056] Based on the category conversion result, determining whether the target item category information in the target item detection result matches the target item category information in the target item classification result, to obtain a first determination result;

[0057] Based on the first judgment result, it is determined whether the detection result of the target object is valid in combination with the detection result of the target object and the confidence information in the classification result of the target object.

[0058] Optionally, determining whether the detection result of the target item is valid based on the first judgment result and in combination with confidence information in the detection result of the target item and the classification result of the target item includes:

[0059] When the first judgment result is that the two match, if the confidence level in the detection result of the target item is higher than the first confidence threshold, or the confidence level in the classification result of the target item is higher than the second confidence threshold, then the detection result of the target item is valid; otherwise, it is invalid;

[0060] When the first judgment result is that the two do not match, if the confidence level in the detection result of the target item is higher than a third confidence threshold, or the confidence level in the classification result of the target item is higher than a fourth confidence threshold, then the detection result of the target item is valid; otherwise, it is invalid;

[0061] The third confidence threshold is greater than the first confidence threshold, and the fourth confidence threshold is greater than the second confidence threshold.

[0062] Optionally, the method further includes:

[0063] When the detection result of the target object is invalid, performing category conversion on the target object category information in the detection result of the target object using the preset second training sample category mapping relationship information, and obtaining a similar object recognition result of the target object based on the category conversion result;

[0064] Among them, the second training sample category mapping relationship information is the correspondence information between the training sample category in which the same training sample is located when it serves as the training sample corresponding to the detection network model, and the similar training sample category in which it is located when it serves as the training sample corresponding to the classification network model; the similar training sample category is a training sample category with the same visual features, or one or more training sample categories at the same level in the training sample category table.

[0065] An embodiment of the present invention further provides a target detection system, comprising:

[0066] Two or more object detection units, each of which is adapted to perform object detection on an input image using a preset detection network model after receiving the input image;

[0067] a first fusion unit adapted to fuse the intermediate detection results output by each of the detection network models to obtain a detection result of the target object; the intermediate detection result includes: target object category information, target object location information, and confidence information;

[0068] The categories of the training samples corresponding to any two detection network models are different.

[0069] Optionally, the system further includes a classification unit adapted to classify the detection results of the target items using a preset classification network model to obtain classification results of the target items; the classification results of the target items include: target item category information and confidence information.

[0070] Optionally, the classification unit includes:

[0071] Two or more target classification subunits, each of which is adapted to classify the detection result of the target object using a preset classification network model upon receiving the detection result of the target object;

[0072] a second fusion subunit, adapted to fuse the intermediate classification results output by each of the classification network models to obtain a classification result of the target item;

[0073] The categories of the training samples corresponding to any two classification network models are different.

[0074] Optionally, the system further comprises:

[0075] The judgment unit is adapted to judge whether the detection result of the target object is valid by using the classification result of the target object, and use the valid detection result of the target object as the recognition result of the target object.

[0076] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0077] By applying the solution of the present invention, two or more detection network models are used to perform target detection at the same time. Compared with using a single detection network model for target detection, since the categories of training samples corresponding to any two detection network models are different, different detection network models can be trained based on training samples of different categories to obtain different detection network models. As a result, when two or more detection network models perform target detection at the same time, the detection effect is better and the detection accuracy is higher.

[0078] Furthermore, since the difference in the number of categories of training samples corresponding to any two of the detection network models is less than the preset first category difference, and in the training samples corresponding to each of the detection network models, the difference in the number of training samples between any two categories is less than the first sample difference threshold, by adjusting the first category difference, the categories of training samples used for each detection network model can be made more balanced, and by adjusting the first sample difference threshold, the number of training samples in each category can be made more balanced, thereby avoiding the impact of sample imbalance on target detection accuracy and improving target detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A flow chart of a target detection method according to an embodiment of the present invention;

[0080] Figure 2 is a schematic diagram of the detection results of the target object in an embodiment of the present invention;

[0081] Figure 3 2 is a schematic diagram of calculating the overlapping area of ​​the frame to be inspected and other frames in an embodiment of the present invention;

[0082] Figure 4 is a schematic diagram of the detection results of multiple target objects in an embodiment of the present invention;

[0083] Figure 5 is a flow chart of another target detection method according to an embodiment of the present invention;

[0084] Figure 6 is a flow chart of a method for classifying detection results of target objects in an embodiment of the present invention;

[0085] Figure 7 is a flow chart of another target detection method in an embodiment of the present invention;

[0086] Figure 8 Schematic diagram of the structure of a target detection system in an embodiment of the present invention. DETAILED DESCRIPTION

[0087] Existing deep learning-based target detection solutions usually use a single CNN or DNN for target detection, which has poor accuracy.

[0088] In response to the above problems, an embodiment of the present invention provides a target detection method. By applying the solution in the embodiment of the present invention, two or more detection network models are used to perform target detection respectively. Compared with using a single detection network model for target detection, since the categories of training samples corresponding to any two detection network models are different, different detection network models can be trained based on training samples of different categories to improve the accuracy of detection.

[0089] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0090] Reference Figure 1 , an embodiment of the present invention provides a target detection method, which may include the following steps:

[0091] Step 11: After receiving the input image, use two or more preset detection network models to perform target object detection on the input image.

[0092] The categories of the training samples corresponding to any two detection network models are different.

[0093] In a specific implementation, the number of the two or more detection network models is not limited. For example, it can be two, or three or more, as long as the categories of the training samples corresponding to any two detection network models are different. The training samples corresponding to the detection network model are the training samples used to train the network model to obtain the detection network model.

[0094] Each detection network model can be obtained by training the same network model using training samples of different categories, or can be obtained by training different network models using training samples of different categories. That is, the model structures corresponding to each detection network model can be the same or different, and there is no specific limitation. For example, each detection network model can be obtained by training a convolutional neural network model using training samples of different categories.

[0095] It is understood that each detection network model can use the same detection method to detect the input image, or it can use different detection methods to detect the input image, which is specifically determined by the model structure corresponding to each detection network model. If the model structure corresponding to each detection network model is the same, the detection method for the input image is also the same, while if the model structure corresponding to each detection network model is different, the detection method for the input image is also different.

[0096] In a specific implementation, the categories of the training samples corresponding to any two detection network models may be completely different or partially different, thereby making the training samples corresponding to the detection network models not completely the same.

[0097] Take the example of two detection network models, namely detection network model Q1 and detection network model Q2. The total categories of training samples of the two detection network models are N, an integer of N≥2, and the set of N categories is C1. Among them, the set of total categories of training samples of detection network model Q1 is CQ1, and the set of total categories of training samples of detection network model Q2 is CQ2, CQ1∈C1, CQ2∈C1. Set CQ1 and set CQ2 can contain the same category, that is, the elements of set CQ1 and set CQ2 can be partially the same. In other words, some categories can appear in set CQ1 and set CQ2 at the same time. But at least one element of set CQ1 and set CQ2 is different.

[0098] In practical applications, the distribution of actual training data is often uneven. Strictly speaking, any dataset is imbalanced, which can cause detection network models to tend to learn features from a wider variety of training samples. In the field of object detection, negative samples (i.e., sample images that do not contain the target object, such as images without contraband) often far outnumber positive samples (sample images that contain the target object, such as images containing contraband). This creates a significant imbalance between positive and negative samples, which in turn increases training costs and degrades the network model.

[0099] For example, in subway security inspections, prohibited items like knives and guns appear far less frequently than regular items like containers and umbrellas, resulting in suboptimal detection. For example, the Shanghai Metro sees an average of over 11 million passengers per weekday. Shanghai's rail transit system has 415 stations, equipped with over 6,000 security inspectors and equipment such as X-ray scanners. They conduct an average of 3.2 million inspections per day, detecting over 35,000 dangerous goods of various types annually. This translates to an average of only 1.09 prohibited items detected per million inspections. Furthermore, prohibited items can range from controlled knives to firearms, resulting in a highly unbalanced distribution of positive and negative samples. Similarly, in satellite remote sensing, individual images often have very high resolutions, such as 30,000 x 40,000 pixels, but very little effective foreground is actually detected, leading to a similar imbalance between foreground and background.

[0100] In an embodiment of the present invention, in order to make the training samples more balanced, the difference in the number of categories of the training samples corresponding to any two of the detection network models is less than a preset first category difference; in the training samples corresponding to each of the detection network models, the difference in the number of training samples of any two categories is less than the first sample difference threshold.

[0101] By adjusting the first category difference, the categories of training samples used for each detection network model can be made more balanced. By adjusting the first sample difference threshold, the number of training samples in each category can be made more balanced, thereby avoiding the impact of sample imbalance on target detection accuracy and improving target detection results.

[0102] For unbalanced training samples, the same training method can be used to obtain each detection network model, or different training methods can be used to obtain each detection network model, which is not limited here.

[0103] In one embodiment of the present invention, the following method may be used for training to obtain the two or more detection network models:

[0104] First, the number of training samples in each category in all training samples (ie, training sample set) can be counted, and a first category with the most training samples and a second category with the least training samples can be determined.

[0105] Next, the training sample interval defined by the number of training samples in the first category to the number of training samples in the second category is evenly divided into K parts to obtain K-1 sample quantity separation values, where K is a positive integer greater than 1.

[0106] Finally, based on the K-1 sample quantity separation value and the number of training samples in each category, the categories of the obtained training samples are separated to obtain K training samples of the detection network model and train them.

[0107] Assume that the training sample set includes N categories, and the number of training samples for each category is M1, M2, ..., MN. Among them, the first category with the most training samples, that is, the category with the most training samples among the N categories, has a training sample number of Mmax. The second category with the fewest training samples, that is, the category with the fewest training samples among the N categories, has a training sample number of Mmin.

[0108] For the difference [Mmin, Mmax], it is evenly divided into K parts to obtain K sub-intervals. The value range of each sub-interval is: [Mmin, Mmin+△], (Mmin+△, Mmin+2*△], ..., (Mmin+(K-1)*△, Mmax]. Among them, △ = (Mmax-Mmin) / K.

[0109] The training samples of all categories with the same number of training samples in the same subinterval are used as training samples for training a detection network model. For example, when K*2, the training samples of all categories with the number of training samples in [Mmin, (Mmin+Mmax) / 2] are used as training samples for training the detection network model Q1. The training samples of all categories with the number of training samples in ((Mmin+Mmax) / 2, Mmax] are used as training samples for training the detection network model Q2.

[0110] In practice, category adjustments can also be made based on the specific features of each category. For example, easily confused and similar categories, such as scissors and knives, can be grouped into the same network to help the network learn the differences between the two items. Assuming the number of categories with the same visual features is M1 and M2, and the detection network models are Q1 and Q2, there are two possible scenarios:

[0111] 1) The difference between M1 and M2 is too large. For example, if M1 / M2 > β, where β is a constant greater than 1 (such as set to 3), at this time, M1 can be updated to βM2 (for example, select βM2 samples from M1 samples using a random function), M2 remains unchanged, and these M1 training samples and these M2 training samples are used as training samples for training to obtain the same detection network model (such as detection network model Q1).

[0112] Meanwhile, in order to avoid significantly affecting the performance of detecting the category of these M1 training samples, according to the average value Mavg' of all training samples of the detection network model Q2, then extract Mavg' samples from these M1 training samples as training samples for training to obtain the detection network model Q2.

[0113] If M1 / M2 < 1 / β, it is also implemented in a similar way as above, which will not be elaborated here.

[0114] 2) The difference between M1 and M2 is not obvious. For example, if 1 / β < M1 / M2 < β, then these M1 training samples and these M2 training samples are used as training samples for training to obtain the same detection network model.

[0115] Step 12, fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item.

[0116] Among them, the intermediate detection results include: target item category information, target item position information, and confidence information.

[0117] In specific implementation, the detection network model can output the intermediate detection results in the form of detection frames. Each detection frame has target item category information, target item position information, and confidence information. For example, the intermediate detection results output by the detection network model Qn: {network Qn, category n, confidence, Xnmin, Ynmin, Xnmax, Ynmax}, the category of the target item is n, (Xnmin, Ynmin) is the lower left vertex coordinate of the detection frame output by the detection network model Qn, and (Xnmax, Ynmax) is the upper right vertex coordinate of the detection frame output by the detection network model Qn, as Figure 2 shown in the rectangular frame.

[0118] For the same input image, each detection network model will output an intermediate detection result. Taking the number of detection network models as M as an example, the identifiers of each detection network model are Q1, Q2,..., QM respectively. All the results output by the M detection network models are shown in Table 1:

[0119] Table 1

[0120]

[0121] In Table 1, Boxn is the identifier of the intermediate detection result, n∈{1,N}, category C1_n is the category identifier of the target item in the intermediate detection result Boxn, and PQn is the confidence value in the intermediate detection result Boxn.

[0122] In a specific implementation, a variety of methods can be used to fuse the intermediate detection results output by each detection network model to obtain the detection result of the target object, and there is no specific limitation.

[0123] In one embodiment of the present invention, when the target item category information of the same target item exists in only one of the intermediate detection results, the intermediate detection result may be used as the detection result of the target item.

[0124] For example, when the category C1_n of the target object in the intermediate detection result Boxn only has the intermediate detection result output by the detection network model Qn, the intermediate detection result Boxn can be directly used as the detection result of the target object.

[0125] When the target item category information of the same target item exists in two or more intermediate detection results, the detection result of the target item is obtained by combining the target item location information and confidence information in the two or more intermediate detection results.

[0126] In a specific implementation, when the target item category information of the same target item exists in more than two intermediate detection results, for any intermediate detection result with a non-lowest confidence among the two or more intermediate detection results, the first area is calculated respectively, and the following operations are performed on any first area: the overlapping area between the first area and each second area is calculated, and the corresponding intermediate detection result is eliminated according to the size of the overlapping area, and the intermediate detection result after the elimination operation is performed on the two or more intermediate detection results is used as the detection result of the target item in this round; the detection result of the target item in each round and the intermediate detection result with the lowest confidence among the two or more intermediate detection results are jointly used as the detection result of the target item.

[0127] Among them, the first area is the target item area corresponding to the target item position information of the intermediate detection result, the second area is the target item area corresponding to the target item position information of the remaining intermediate detection results among the two or more intermediate detection results, and the confidence level of the intermediate detection result where the first area is located is higher than the confidence level of the intermediate detection result where the second area is located.

[0128] In one embodiment of the present invention, the removing of the corresponding intermediate detection results according to the size of the overlapping area may include:

[0129] Based on the size of the overlapping area, determining the difference between the area of ​​each target object and the overlapping area in the two corresponding intermediate detection results;

[0130] When the difference between the target object area and the overlapped area is smaller than a first area threshold, performing a rejection operation on the intermediate detection results where the target object area difference between the target object area and the overlapped area is smaller than the first area threshold;

[0131] When the difference between the target object area and the overlapping area is greater than the first area threshold, a rejection operation is performed based on the difference between the target object areas in the two corresponding intermediate detection results.

[0132] For example, when the category C1_n of the target object in the intermediate detection result Boxn exists in the intermediate detection results output by M detection network models at the same time, the intermediate detection results of category C1_n in each detection network model can be extracted first.

[0133] The intermediate detection results for category C1_n are stored in temporary storage space TOn. For example, the temporary storage space for category C1_n is TO1, and the temporary storage space for category C1_2 is TO2. Ultimately, the storage space for all categories is {TO1, ..., TOn, ..., TON}. Each category's storage space TOn is processed separately. At this point, the storage content of TOn is (assuming a total of n' detection boxes) as shown in Table 2:

[0134] Table 2

[0135]

[0136] Among them, Boxn' is the intermediate detection result of category C1_n in the detection network model Qn, n∈{1,N}.

[0137] Taking the fusion of the intermediate detection results of category C1_n as an example, the specific fusion process is as follows:

[0138] a) Sort all detection boxes from high to low according to confidence;

[0139] b) The detection frame with the highest confidence is used as the frame to be inspected, and the overlapping area S between the frame to be inspected and another frame (with a lower confidence than the frame to be inspected) is calculated in sequence to obtain the detection result of the target object in this round.

[0140] Take the operation performed by the check box and one of the other boxes as an example. Figure 3 As shown, the area of ​​the frame to be inspected is the first area S1, and the area of ​​one of the other frames is the second area S2. At this time, the overlap coefficient α of the two frames can be calculated.

[0141] When S1 > γS2, where γ is a constant greater than 1, at this time S1 is much larger than S2, and the coincidence coefficient α = S / S2. If α exceeds the first preset coincidence threshold G1, it indicates that the coincidence area S is close to the second area S2, and other boxes can be considered to have been included in the detected box. Therefore, other boxes can be removed from TOn.

[0142] When S1 < S2 / γ, at this time S1 is much smaller than S2, and α = S / S1. If α exceeds the second preset coincidence threshold G2, it indicates that the coincidence area S is close to the first area S1, and the detected box can be considered to have been included in other boxes. Therefore, the detected box can be removed from TOn.

[0143] When S2 / γ ≤ S1 ≤ γS2, at this time the first area S1 is comparable to the second area S2, and α = S / (S1 + S2 - S). If α exceeds the third preset coincidence threshold G3, it indicates that the first area S1 is close to the second area S2, and the detection effects of the detected box and other boxes are comparable. At this time, only one detection box needs to be retained. For example, the detected box with a higher confidence level can be selected, and other boxes can be removed from TOn.

[0144] Among them, γ, the first preset coincidence threshold G1, the second preset coincidence threshold G2, and the third preset coincidence threshold G3 are all preset constants, and the specific values corresponding to different categories may vary.

[0145] In an embodiment of the present invention, when the detection effects of the detected box and other boxes are comparable, if the area of the target object in the removed intermediate detection result is larger than the area of the target object in the retained intermediate detection result, the position information of the target object in the removed intermediate detection result can be used to update the position information of the target object in the retained intermediate detection result.

[0146] For example, when other boxes are removed, if S2 > S1, in order to minimize the impact of eliminating the detected box, the confidence level of the detected box can be kept unchanged, and the position information of the target object in other boxes can be used to update the position information of the target object in the detected box, so that the area of the detected box is larger and the detection output covers more. If S2 ≤ S1, it indicates that the area and confidence level of the detected box are both the largest, and no update is required.

[0147] After performing the above operations on the detected box and one other box, the detection result of the target object in this round is obtained. Using the detection result of the target object in this round, the storage space Ton is updated to obtain the updated storage space TOn', which is used as the output of category C1_n.

[0148] c) After one round of traversal is completed, start traversing from the box with the second highest confidence level until the lowest box, and a total of n' - 1 rounds of traversal are required. Furthermore, the fusion outputs {TO1', ……, TOn', ……, TON'} of all categories are obtained as the detection result of the target object.

[0149] In a specific implementation, according to the target object position information in the detection result of the target object (the (Xmin, Ymin), (Xmax, Ymax) parameters of the detection box), a complete image can be extracted, such as Figure 4 As shown, images 41 and 42 are both extracted bottle bottom images, image 43 is an extracted metal bar image, and images 44 and 45 are extracted fruit knife images. The actual target object detection result may be only one target object image or multiple target object images, for example, Figure 4 As shown, when the target object is a bottle bottom, the detection result of the target object may include image 41 and image 42 at the same time.

[0150] From the above content, it can be seen that the target detection method in the embodiment of the present invention can make the training samples of the detection network model more balanced, and use more than two detection network models to perform target detection at the same time, and finally fuse the intermediate results output by each detection network model to obtain the detection result of the target object, which can make the target detection result more accurate.

[0151] The embodiment of the present invention also provides another target detection method, referring to Figure 5 , the method may include the following steps:

[0152] Step 51: After receiving the input image, use two or more preset detection network models to perform target object detection on the input image.

[0153] The categories of the training samples corresponding to any two detection network models are different.

[0154] Step 52: Fusion the intermediate detection results output by each detection network model to obtain the detection result of the target object.

[0155] The intermediate detection results include: target item category information, target item location information and confidence information.

[0156] Regarding steps 51 and 52, they can be implemented with reference to the description of steps 11 and 12, which will not be repeated here.

[0157] Step 53: Classify the detection result of the target object using a preset classification network model to obtain a classification result of the target object.

[0158] The classification result of the target item includes: target item category information and confidence information.

[0159] In practical applications, new images often appear continuously, which means that various unknown objects and stacking forms may appear. Therefore, the detection results of target objects obtained by the detection network model may still have false detections. Figure 4 As shown in , metal bottle bottoms and metal strips may be mistakenly detected as cutting tools.

[0160] To reduce false detections and improve detection accuracy, embodiments of the present invention utilize a pre-defined classification network to further determine the category of a target item. For a given target item, all positive sample images reflect the item's relevance, which in turn reflects the characteristics of the target item. Therefore, the classification network model can learn the correlations between these characteristic items and use them to determine the target item's detection results. The classification network model does not need to focus on the target item's location; it only learns the classification characteristics to accurately determine the target item's category.

[0161] In a specific implementation, a preset classification network model can be used to classify the detection results of the target object using a variety of methods to obtain the classification results of the target object.

[0162] In one embodiment of the present invention, in order to further reduce false detection, the number of the classification network models can be more than two. Figure 6 , using the preset classification network model, the detection results of the target object can be classified using the following method:

[0163] Step 61: When the detection result of the target object is received, the detection result of the target object can be classified using two or more preset classification network models, wherein the categories of the training samples corresponding to any two classification network models are different.

[0164] In a specific implementation, the number of the classification network models can be only two, or three or more, and there is no specific limitation. The only requirement is that the categories of the training samples corresponding to any two of the classification network models are different. The training samples corresponding to the two classification network models are the training samples used to train the classification network model.

[0165] Each classification network model can be obtained by training the same network model using training samples of different categories, or can be obtained by training different network models using training samples of different categories. That is, the model structures corresponding to the classification network models can be the same or different, and there is no specific limitation. For example, each classification network model can be obtained by training a convolutional neural network model using training samples of different categories.

[0166] It is understood that each classification network model can use the same classification method to classify the detection results of the target object, or it can use different classification methods to classify the detection results of the target object, which is determined by the model structure corresponding to each classification network model. If the model structure corresponding to each classification network model is the same, the method for classifying the detection results of the target object is also the same. If the model structure corresponding to each classification network model is different, the method for classifying the detection results of the target object is also different.

[0167] In a specific implementation, the categories of the training samples corresponding to any two classification network models may be completely different or partially different.

[0168] For example, consider L classification network models. Each classification network model is labeled H1, H2, ..., HL (L ≥ 2). The total number of training sample categories corresponding to the L classification network models is C2, which is typically different from the total number of training sample categories corresponding to the detection network model. This allows for the addition of some categories prone to false positives. The training sample categories corresponding to each classification network model can overlap. In other words, for any class n, n∈C2, class n can appear simultaneously in the training samples corresponding to classification network models H1, H2, ..., and HL.

[0169] In practice, the training of the classification network model and the detection network model are independent of each other, each using a different training sample set. The detection network model's training sample set consists of a large number of images, while the classification network's training sample set can consist of a large number of detection boxes. The training methods for each classification network model can be the same or different.

[0170] To improve the accuracy of the classification model's classification judgment, the difference in the number of training samples between any two categories in the training samples corresponding to each classification network model is less than a second sample difference threshold. This second sample difference threshold can be set to minimize the difference in the number of training samples between any two categories corresponding to the same classification network model.

[0171] For example, in the training sample set of the classification network model, the category with the least number of training samples is category 1, and the number of training samples of category 1 is Mcmin. At this time, if category 1 is used as the training network of the classification network model H1, the maximum number of training samples of other categories corresponding to the classification network model H1 is βMCmin.

[0172] If the number of training samples of other categories in the training sample set exceeds βMCmin, then the βMCmin training samples can be used to train the classification network model H1. If the number of training samples of other categories in the training sample set does not exceed βMCmin, then all training samples of other categories can be used to train the classification network model H1.

[0173] In one embodiment of the present invention, the training samples corresponding to the two or more classification network models may be determined by the following method:

[0174] First, training samples with the same visual features are selected from all training samples (ie, the training set corresponding to the classification network model), and the training samples with the same visual features are used as training samples corresponding to the same classification network model.

[0175] In a specific implementation, objects with the same visual features are similar in shape. For example, a metal bottle bottom, a metal strip, and a fruit knife have similar shapes. In this case, the metal bottle bottom, the metal strip, and the fruit knife can be used as training samples to train the classification network model A1. Another example is that a metal bottle and a metal gas cylinder have similar shapes. In this case, the metal bottle and the metal gas cylinder can be used as training samples to train the classification network model A2.

[0176] Next, training samples with the same item usage characteristics are selected from the remaining training samples after the selection according to the visual features, and the training samples with the same item usage characteristics are used as training samples corresponding to the same classification network model.

[0177] In a specific implementation, items with the same usage characteristics, such as wrenches, pliers, and saws, can be used as training samples for the same classification network model. In this case, the difference between the number of training samples with the same usage characteristics and the number of training samples of other categories already in the classification network model should be less than the second sample difference threshold.

[0178] Finally, from the remaining training samples after selection based on visual features and item usage characteristics, training samples with the same item material are selected, and the training samples with the same item material are used as training samples corresponding to the same classification network model.

[0179] In a specific implementation, items of the same material, such as mobile phones, laptops, and monitors, can be used as training samples for the same classification network model. In this case, the difference between the number of training samples of the same material and other categories in the classification network model should be less than the second sample difference threshold.

[0180] In one embodiment of the present invention, two categories of training samples with different visual features but a difference in the number of training samples less than a preset third sample difference threshold can also be used as training samples corresponding to the same classification network model; the third sample difference threshold is less than the second sample difference threshold.

[0181] That is to say, two categories of training samples with very different visual features but similar numbers of training samples can be used as training samples corresponding to the same classification network model.

[0182] Step 62: Fusing the intermediate classification results output by each of the classification network models to obtain the classification result of the target item.

[0183] In a specific implementation, the detection result of the target object can be output in the form of a detection box. Table 3 shows the detection result Boxn of the target object:

[0184] Table 3

[0185] Boxn Network Qn Category C1_n Confidence PQ Xmin Ymin Xmax Ymax

[0186] After the classification network model Hi, the intermediate classification results output by the classification network model Hi may be as shown in Table 4:

[0187] Table 4

[0188]

[0189] As can be seen from Table 4, the classification network model Hi determines that the target items in the detection result Boxn of the target items belong to category C2_1, category C2_2 and category C2_i.

[0190] After the classification network model Hj, the intermediate classification results output by the classification network model Hj may be as shown in Table 5:

[0191] Table 5

[0192]

[0193] As can be seen from Table 5, the classification network model Hj determines that the target item in the detection result Boxn of the target item belongs to category C2_1, category C2_2 and category C2_j.

[0194] After passing through different classification network models, different intermediate classification results are output.

[0195] In a specific implementation, a variety of methods can be used to fuse the intermediate classification results output by each classification network model to obtain the classification result of the target object.

[0196] In one embodiment of the present invention, when the target item category information of the same target item exists in only one intermediate classification result, the intermediate classification result is used as the classification result of the target item.

[0197] When the target item category information of the same target item exists in only two intermediate classification results, the intermediate classification result with a higher confidence level among the two intermediate classification results is used as the classification result of the target item;

[0198] When the target item category information of the same target item exists in more than three intermediate classification results, a preset number of intermediate classification results are selected as the classification result of the target item based on the confidence information in the three or more intermediate classification results.

[0199] For example, assuming that the classification result of the target item is OH, if category C2_i only exists in the intermediate classification result of the classification network model Hi, then category C2_i and the corresponding confidence PH3 will be retained in the classification result OH of the target item.

[0200] If category C2_1 only exists in the classification network model Hi and the classification network model Hj, and PH1<PH4, then the confidence PH4 corresponding to C2_1 is retained in the classification result OH of the target item.

[0201] If category C2_2 exists in more than three intermediate classification results, at this time, if the confidence corresponding to category C2_2 in multiple intermediate classification results is extremely high, for example, the confidence corresponding to category C2_2 is greater than GP1, then a small number (for example, three) of intermediate classification results with the highest confidence can be selected and retained in the classification result OH of the target item.

[0202] If the confidence scores for category C2_2 in all intermediate classification results are low, for example, all confidence scores for category C2_2 are less than GP2, then a small number (for example, one) of the intermediate classification results with the highest confidence scores can be selected and retained in the classification result OH for the target item. GP2 is less than GP1.

[0203] If the highest confidence corresponding to category C2_2 in all intermediate classification results is less than GP1 and the lowest confidence is greater than GP2, then multiple (for example, five) intermediate classification results with the highest confidence can be selected and retained in the classification result OH of the target item.

[0204] As can be seen from the above content, the target detection method in the embodiment of the present invention, after obtaining the detection result of the target object, further determines the category of the target object by using the classification network, thereby further improving the accuracy of target detection.

[0205] Reference Figure 7 , an embodiment of the present invention further provides another target detection method, which may include the following steps:

[0206] Step 71: After receiving the input image, use two or more preset detection network models to perform target object detection on the input image.

[0207] The categories of the training samples corresponding to any two detection network models are different.

[0208] Step 72: fuse the intermediate detection results output by each detection network model to obtain the detection result of the target object.

[0209] The classification result of the target item includes: target item category information and confidence information.

[0210] Regarding steps 71 and 72, they can be implemented with reference to the description of steps 11 and 12, which will not be repeated here.

[0211] Step 73: Classify the detection result of the target object using a preset classification network model to obtain a classification result of the target object.

[0212] The classification result of the target item includes: target item category information and confidence information.

[0213] Regarding step 73, the specific implementation can refer to the description of step 53, which will not be repeated here.

[0214] In step 74 , the classification result of the target object is used to determine whether the detection result of the target object is valid, and the valid detection result of the target object is used as the identification result of the target object.

[0215] In a specific implementation, the classification result of the target item can be directly used as the identification result of the target item, or the classification result of the target item can be used to further determine whether the detection result of the target item is valid, and the valid detection result of the target item can be used as the identification result of the target item, thereby further improving the accuracy of target detection.

[0216] Since the detection results of the target items include not only the category information of the target items but also the location information of the target items, the classification results of the target items are used to eliminate invalid detection results of the target items from the detection results of many target items, and the remaining detection results of the target items are used as the recognition results of the target items.

[0217] In a specific implementation, various methods may be used to determine whether the detection result of the target object is valid using the classification result of the target object.

[0218] In one embodiment of the present invention, the preset first training sample category mapping relationship information can be used to perform category conversion on the target item category information in the detection result of the target item, and based on the category conversion result, it is determined whether the detection result of the target item is valid to obtain the recognition result of the target item.

[0219] Among them, the first training sample category mapping relationship information is the correspondence relationship information between the training sample category of the same training sample when it serves as the training sample corresponding to the detection network model and the training sample category of the same training sample when it serves as the training sample corresponding to the classification network model.

[0220] In a specific implementation, a training sample can be used as both a training sample for training a detection network model and a training sample for training a classification network model. When used as a training sample for training a detection network model, the training sample belongs to category C1_i. When used as a training sample for training a classification network model, the training sample belongs to category C2_j. The correspondence between category C1_i and category C2_j is the category mapping relationship information for the training sample. The first training sample category mapping relationship information may include category mapping relationship information for multiple training samples.

[0221] In a specific implementation, the first training sample category mapping relationship information can be obtained in advance, and the specific acquisition method is not limited.

[0222] In one embodiment of the present invention, based on the category conversion result, the following method can be used to determine whether the detection result of the target object is valid:

[0223] First, based on the category conversion result, determine whether the target item category information in the detection result of the target item matches the target item category information in the classification result of the target item to obtain a first judgment result. Then, based on the first judgment result, combine the confidence information in the detection result of the target item and the classification result of the target item to determine whether the detection result of the target item is valid.

[0224] When the first judgment result is that the two match, if the confidence of one of the detection result of the target item or the classification result of the target item is higher than the first confidence threshold, the detection result of the target item is valid, otherwise it is invalid.

[0225] When the first judgment result is that the two do not match, if the confidence of one of the detection result of the target item or the classification result of the target item is higher than the second confidence threshold, the detection result of the target item is valid, otherwise it is invalid; the second confidence threshold is greater than the first confidence threshold.

[0226] In a specific implementation, the intermediate detection result Boxn output by any detection network model Qn is used as the detection result Boxn of the target object. The specific content of Boxn is shown in Table 3. Table 6 shows the classification results of the target object.

[0227] Table 6

[0228] Network Hi Category C2_i Confidence PH3 Network Hj Category C2_j Confidence PH6

[0229] In the embodiment of the present invention, for the convenience of description, for a training sample category C1_n corresponding to any detection network model, the category conversion result is f(C1_n).

[0230] If f(C1_n)∈{C2_i, C2_j}, that is, the result after category conversion belongs to {C2_i, C2_j}, it means that the detection result of the target item and the classification result of the target item both have the same prediction result, and the two match.

[0231] When the two match, if the confidence PQ in the detection result of the target item is higher than the first confidence threshold, it indicates that the detection result Boxn of the target item is highly credible. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid. Alternatively, if the confidence (PH3) of the corresponding category (such as category C2_i) in the classification result of the target item is higher than the second confidence threshold, it indicates that the target item belongs to category C2_i to a large extent. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid.

[0232] When the two do not match, if the confidence PQ in the detection result of the target item is higher than the third confidence threshold, it indicates that the detection result Boxn of the target item is extremely reliable. At this time, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid. Alternatively, if the confidence (PH3) of the corresponding category (such as category C2_i) in the classification result of the target item is higher than the fourth confidence threshold, it indicates that the target item belongs to category C2_i to a great extent. At this time, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid.

[0233] In a specific implementation, when the detection result of the target item is invalid, the preset second training sample category mapping relationship information can be used to perform category conversion on the target item category information in the detection result of the target item, and based on the category conversion result, a similar item recognition result of the target item is obtained.

[0234] Among them, the second training sample category mapping relationship information is the correspondence information between the training sample category in which the same training sample is located when it serves as the training sample corresponding to the detection network model, and the similar training sample category in which it is located when it serves as the training sample corresponding to the classification network model; the similar training sample category is a training sample category with the same visual features, or one or more training sample categories at the same level in the training sample category table.

[0235] For example, if the training sample category corresponding to the detection network model is scissors, the similar training sample categories corresponding to the classification network model can be fruit knife and dagger. The correspondence between scissors, fruit knife, and dagger is a second training sample category mapping relationship information. After performing category conversion on the scissors category, f(C1_n) becomes fruit knife and dagger. At this point, it is determined whether the fruit knife or dagger matches the classification result of the target object.

[0236] When the two match, if the confidence PQ in the detection result of the target item is higher than the fifth confidence threshold, it indicates that the detection result Boxn of the target item is highly credible. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid, wherein the fifth confidence threshold is higher than the first confidence threshold. Alternatively, if the confidence (PH3) of the corresponding category (such as category C2_i) in the classification result of the target item is higher than the sixth confidence threshold, it indicates that the target item belongs to category C2_i to a large extent. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid, wherein the sixth confidence threshold is higher than the second confidence threshold.

[0237] When the two do not match, if the confidence PQ in the detection result of the target item is higher than the seventh confidence threshold, it indicates that the detection result Boxn of the target item is extremely reliable. At this time, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid, wherein the seventh confidence threshold is higher than the third confidence threshold, and the seventh confidence threshold is greater than the fifth confidence threshold. Alternatively, if the confidence (PH3) of the corresponding category (such as category C2_i) in the classification result of the target item is higher than the eighth confidence threshold, it indicates that the target item belongs to category C2_i to a great extent. At this time, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid, wherein the eighth confidence threshold is higher than the fourth confidence threshold, and the eighth confidence threshold is greater than the sixth confidence threshold.

[0238] It should be noted that, in a specific implementation, the first confidence threshold to the eighth confidence threshold can be set according to actual conditions, and will not be described in detail here.

[0239] From the above content, it can be seen that the target detection method in the embodiment of the present invention, by utilizing the classification result of the target item to determine whether the detection result of the target item is valid, and using the valid detection result of the target item as the identification result of the target item, can further improve the accuracy of target detection.

[0240] In order to enable those skilled in the art to better understand and implement the present invention, the target detection system corresponding to the above method is described in detail below.

[0241] Figure 8 FIG. 1 is a schematic diagram of the structure of a target detection system according to an embodiment of the present invention. Figure 8 The target detection system 80 may include: two or more target detection units 81 and a first fusion unit 82.

[0242] The target detection unit 81 is adapted to perform target detection on the input image using a preset detection network model after receiving the input image;

[0243] The first fusion unit 82 is adapted to fuse the intermediate detection results output by each of the detection network models to obtain a detection result of the target object; the intermediate detection result includes: target object category information, target object location information and confidence information;

[0244] The categories of the training samples corresponding to any two detection network models are different.

[0245] In one embodiment of the present invention, the target detection system 80 may further include: a classification unit 83.

[0246] The classification unit 83 is adapted to classify the detection result of the target object using a preset classification network model to obtain a classification result of the target object; the classification result of the target object includes: target object category information and confidence information.

[0247] In one embodiment of the present invention, referring to Figure 8 The classification unit 83 may include: two or more target classification subunits 831 and a second fusion subunit 832.

[0248] The classification subunit 831 is adapted to classify the detection result of the target object using a preset classification network model when receiving the detection result of the target object;

[0249] The second fusion subunit 832 is adapted to fuse the intermediate classification results output by each of the classification network models to obtain a classification result of the target item;

[0250] The number of the classification network models is more than two, and the categories of the training samples corresponding to any two of the classification network models are different.

[0251] In one embodiment of the present invention, the target detection system 80 may further include: a judgment unit 84.

[0252] The judgment unit 84 is adapted to use the classification result of the target object to judge whether the detection result of the target object is valid, and use the valid detection result of the target object as the recognition result of the target object.

[0253] Regarding each unit in the target detection system 80, specific implementation can be carried out by referring to the description of the corresponding steps in the above target detection method, which will not be repeated here.

[0254] It can be seen from the above content that the target detection system 80 in the embodiment of the present invention can effectively improve the detection accuracy compared with using a single network model to perform target detection.

[0255] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A target detection method, characterized in that: include: After receiving the input image, use two or more preset detection network models to detect the target object in the input image; Fusing the intermediate detection results output by each of the detection network models to obtain a detection result of the target object; The intermediate detection results include: target item category information, target item location information and confidence information; Using a preset classification network model, the detection result of the target object is classified to obtain a classification result of the target object; the classification result of the target object includes: target object category information and confidence information; The training samples corresponding to the two or more classification network models are determined by the following method: Selecting training samples with the same visual features from all training samples, and using the training samples with the same visual features as training samples corresponding to the same classification network model; Selecting training samples with the same item usage characteristics from the remaining training samples after the selection based on the visual features, and using the training samples with the same item usage characteristics as training samples corresponding to the same classification network model; Selecting training samples with the same material from the remaining training samples after selecting according to the visual features and the usage characteristics of the items, and using the training samples with the same material as the training samples corresponding to the same classification network model; The categories of the training samples corresponding to any two detection network models are different.

2. The target detection method according to claim 1, wherein: The categories of the training samples corresponding to any two detection network models are completely different, or partially different.

3. The target detection method according to claim 1, wherein: The difference in the number of categories of training samples corresponding to any two of the detection network models is less than a preset first category difference; in the training samples corresponding to each of the detection network models, the difference in the number of training samples of any two categories is less than a first sample difference threshold.

4. The target detection method according to claim 1, wherein: The two or more detection network models are trained by the following method: Count the number of training samples in each category among all training samples, and determine the first category with the most training samples and the second category with the least training samples; The training sample interval defined by the number of training samples of the first category to the number of training samples of the second category is evenly divided into K parts to obtain K-1 sample number separation values, where K is a positive integer greater than 1; Based on the K-1 sample quantity separation value and the number of training samples in each category, the categories of the obtained training samples are separated to obtain K training samples of the detection network model and then train them.

5. The target detection method according to claim 1, wherein: The fusing of the intermediate detection results output by each of the detection network models to obtain the detection result of the target object includes: When the target item category information of the same target item exists in only one of the intermediate detection results, the intermediate detection result is used as the detection result of the target item; When the target item category information of the same target item exists in two or more intermediate detection results, the detection result of the target item is obtained by combining the target item location information and confidence information in the two or more intermediate detection results.

6. The target detection method according to claim 5, wherein: Combining the target object location information and the confidence information in the two or more intermediate detection results to obtain the detection result of the target object includes: For any intermediate detection result with a non-lowest confidence level among the two or more intermediate detection results, respectively calculate the first area, and perform the following operations on each of the first areas: calculate the overlapping area between the first area and each second area, and perform a rejection operation on the corresponding intermediate detection result based on the size of the overlapping area, and use the intermediate detection result after the rejection operation among the two or more intermediate detection results as the detection result of the target object in the current round; The detection result of the target item in each round and the intermediate detection result with the lowest confidence among the two or more intermediate detection results are collectively used as the detection result of the target item; Among them, the first area is the target item area corresponding to the target item position information of the intermediate detection result, the second area is the target item area corresponding to the target item position information of the remaining intermediate detection results among the two or more intermediate detection results, and the confidence level of the intermediate detection result where the first area is located is higher than the confidence level of the intermediate detection result where the second area is located.

7. The target detection method according to claim 6, wherein: The step of performing a removal operation on the corresponding intermediate detection results according to the size of the overlapped area includes: Based on the size of the overlapping area, determining the difference between the area of ​​each target object and the overlapping area in the two corresponding intermediate detection results; When the difference between the target object area and the overlapped area is smaller than a first area threshold, performing a rejection operation on the intermediate detection results where the target object area difference between the target object area and the overlapped area is smaller than the first area threshold; When the difference between the target object area and the overlapping area is greater than the first area threshold, a rejection operation is performed based on the difference between the target object areas in the two corresponding intermediate detection results.

8. The target detection method according to claim 7, wherein: The performing of a rejection operation based on a difference between the target object areas in the two corresponding intermediate detection results includes: Calculating the difference between the target object areas in the two corresponding intermediate detection results; When the difference between the areas of the target objects is smaller than the second area threshold, a rejection operation is performed on the intermediate detection results with lower confidence.

9. The target detection method according to claim 8, wherein: The performing of a rejection operation based on the difference between the target object areas in the two corresponding intermediate detection results further includes: The target object position information of the discarded intermediate detection results is used to update the target object position information of the retained intermediate detection results.

10. The target detection method according to claim 6, wherein: In descending order of confidence, each first area in the intermediate detection results having a non-lowest confidence is calculated, and a elimination operation is performed.

11. The target detection method according to any one of claims 1 to 10, wherein: The method of classifying the detection result of the target object by using a preset classification network model to obtain the classification result of the target object includes: When the detection result of the target object is received, the detection result of the target object is classified using two or more preset classification network models; fusing the intermediate classification results output by each of the classification network models to obtain a classification result of the target object; The number of the classification network models is more than two, and the categories of the training samples corresponding to any two of the classification network models are different.

12. The target detection method according to claim 11, wherein: The categories of the training samples corresponding to any two classification network models are completely different, or partially different.

13. The target detection method according to claim 11, wherein: In the training samples corresponding to each of the classification network models, the difference in the number of training samples between any two categories is less than a second sample difference threshold.

14. The target detection method according to claim 13, wherein: The training samples corresponding to the two or more classification network models are also determined by the following method: Using two categories of training samples with different visual features but a difference in the number of training samples less than a preset third sample difference threshold as training samples corresponding to the same classification network model; The third sample difference threshold is smaller than the second sample difference threshold.

15. The target detection method according to claim 11, wherein: The step of fusing the intermediate classification results output by each of the classification network models to obtain the classification result of the target object includes: When the target item category information of the same target item exists in only one intermediate classification result, the intermediate classification result is used as the classification result of the target item; When the target item category information of the same target item exists in only two intermediate classification results, the intermediate classification result with a higher confidence level among the two intermediate classification results is used as the classification result of the target item; When the target item category information of the same target item exists in more than three intermediate classification results, a preset number of intermediate classification results are selected as the classification result of the target item based on the confidence information in the three or more intermediate classification results.

16. The target detection method according to claim 1, wherein: The method further includes: using the classification result of the target item to determine whether the detection result of the target item is valid, and using the valid detection result of the target item as the identification result of the target item.

17. The target detection method according to claim 16, wherein: The determining whether the detection result of the target object is valid by using the classification result of the target object includes: Using the preset first training sample category mapping relationship information, performing category conversion on the target item category information in the detection result of the target item, and judging whether the detection result of the target item is valid based on the category conversion result, thereby obtaining an identification result of the target item; Among them, the first training sample category mapping relationship information is the correspondence relationship information between the category of the same training sample when it is used as the training sample corresponding to the detection network model and the category of the same training sample when it is used as the training sample corresponding to the classification network model.

18. The target detection method according to claim 17, wherein: The determining whether the detection result of the target object is valid based on the category conversion result includes: Based on the category conversion result, determining whether the target item category information in the target item detection result matches the target item category information in the target item classification result, to obtain a first determination result; Based on the first judgment result, combined with the detection result of the target object and the confidence information in the classification result of the target object, it is determined whether the detection result of the target object is valid.

19. The target detection method according to claim 18, wherein: The determining, based on the first judgment result and in combination with the detection result of the target object and the confidence information in the classification result of the target object, whether the detection result of the target object is valid includes: When the first judgment result is that the two match, if the confidence level in the detection result of the target item is higher than the first confidence threshold, or the confidence level in the classification result of the target item is higher than the second confidence threshold, then the detection result of the target item is valid; otherwise, it is invalid; When the first judgment result is that the two do not match, if the confidence level in the detection result of the target item is higher than a third confidence threshold, or the confidence level in the classification result of the target item is higher than a fourth confidence threshold, then the detection result of the target item is valid; otherwise, it is invalid; The third confidence threshold is greater than the first confidence threshold, and the fourth confidence threshold is greater than the second confidence threshold.

20. The target detection method according to claim 16, wherein: Also includes: When the detection result of the target object is invalid, performing category conversion on the target object category information in the detection result of the target object using the preset second training sample category mapping relationship information, and obtaining a similar object recognition result of the target object based on the category conversion result; The second training sample category mapping relationship information is the corresponding relationship information between the training sample category of the same training sample when it is used as the training sample corresponding to the detection network model and the similar training sample category when it is used as the training sample corresponding to the classification network model; The similar training sample categories are training sample categories with the same visual features, or one or more training sample categories at the same level in the training sample category table.

21. A target detection system, characterized in that: include: Two or more object detection units, each of which is adapted to perform object detection on an input image using a preset detection network model after receiving the input image; a first fusion unit adapted to fuse the intermediate detection results output by each of the detection network models to obtain a detection result of the target object; The intermediate detection results include: target item category information, target item location information and confidence information; a classification unit adapted to classify the detection result of the target object using a preset classification network model to obtain a classification result of the target object; the classification result of the target object includes: target object category information and confidence information; The training samples corresponding to the two or more classification network models are determined by the following method: Selecting training samples with the same visual features from all training samples, and using the training samples with the same visual features as training samples corresponding to the same classification network model; Selecting training samples with the same item usage characteristics from the remaining training samples after the selection based on the visual features, and using the training samples with the same item usage characteristics as training samples corresponding to the same classification network model; Selecting training samples with the same material from the remaining training samples after selecting according to the visual features and the usage characteristics of the items, and using the training samples with the same material as the training samples corresponding to the same classification network model; The categories of the training samples corresponding to any two detection network models are different.

22. The target detection system according to claim 21, wherein: Also includes: a classification unit, adapted to classify the detection result of the target object using a preset classification network model to obtain a classification result of the target object; The classification result of the target item includes: target item category information and confidence information.

23. The target detection system according to claim 22, wherein: The classification units include: Two or more target classification subunits, each of which is adapted to classify the detection result of the target object using a preset classification network model upon receiving the detection result of the target object; a second fusion subunit, adapted to fuse the intermediate classification results output by each of the classification network models to obtain a classification result of the target object; The categories of the training samples corresponding to any two classification network models are different.

24. The target detection system according to claim 22, wherein: Also includes: The judgment unit is adapted to judge whether the detection result of the target object is valid by using the classification result of the target object, and use the valid detection result of the target object as the recognition result of the target object.

Images