Target Recognition Methods and Systems

Abstract

A target recognition method and system are disclosed. The method includes: upon receiving an input image, performing target detection on the input image using a preset detection network model to obtain target item detection results; the target item detection results include: target item category information, target item location information, and confidence information; classifying the target item detection results using a preset classification network model to obtain target item classification results; the target item classification results include: target item category information and confidence information; using the target item classification results, determining whether the target item detection results are valid, and using valid target item detection results as the target item recognition results. Applying the above scheme can improve the accuracy of target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection, and more specifically to a target recognition method and system. Background Technology

[0002] The prototype of deep learning uses computer system architecture to simulate the human brain and is widely used in fields such as security inspection. In security inspection, deep learning is primarily used for object detection, that is, detecting the location and classification information of target objects in an input image.

[0003] Existing deep learning-based object detection solutions typically use a single convolutional neural network (CNN) or a fully connected deep neural network (DNN) for object detection.

[0004] However, the accuracy of target detection using the above method is poor. Summary of the Invention

[0005] The problem this invention aims to solve is to improve the accuracy of target detection.

[0006] To address the above problems, embodiments of the present invention provide a target recognition method, the method comprising:

[0007] After receiving the input image, the input image is subjected to target detection using a preset detection network model to obtain the detection results of the target items; the detection results of the target items include: target item category information, target item location information, and confidence information;

[0008] Using a pre-defined classification network model, the detection results of the target item are classified to obtain the classification result of the target item; the classification result of the target item includes: target item category information and confidence information;

[0009] Using the classification results of the target item, determine whether the detection result of the target item is valid, and take the valid detection result of the target item as the identification result of the target item.

[0010] Optionally, determining whether the detection result of the target item is valid using the classification result of the target item includes:

[0011] Using the preset first training sample category mapping relationship information, the category information of the target item in the detection result of the target item is converted into a category, and based on the category conversion result, it is determined whether the detection result of the target item is valid, and the recognition result of the target item is obtained.

[0012] The first training sample category mapping information is the correspondence between the category of the same training sample when it is used as a training sample for the detection network model and the category of the same training sample when it is used as a training sample for the classification network model.

[0013] Optionally, determining whether the detection result of the target item is valid based on the category conversion result includes:

[0014] Based on the category conversion result, determine whether the target item category information in the detection result of the target item matches the target item category information in the classification result of the target item, and obtain the first judgment result;

[0015] Based on the first judgment result, and combined with the detection result of the target item and the confidence information in the classification result of the target item, it is determined whether the detection result of the target item is valid.

[0016] Optionally, determining whether the detection result of the target item is valid based on the first judgment result, combined with the detection result of the target item and the confidence information in the classification result of the target item, includes:

[0017] When the first judgment result is a match between the two, if the confidence level in the detection result of the target item is higher than the first confidence level threshold, or the confidence level in the classification result of the target item is higher than the second confidence level threshold, then the detection result of the target item is valid; otherwise, it is invalid.

[0018] When the first judgment result is that the two do not match, if the confidence level of the detection result of the target item is higher than the third confidence level threshold, or the confidence level of the classification result of the target item is higher than the fourth confidence level threshold, then the detection result of the target item is valid; otherwise, it is invalid.

[0019] Wherein, the third confidence threshold is greater than the first confidence threshold, and the fourth confidence threshold is greater than the second confidence threshold.

[0020] Optionally, the method further includes:

[0021] When the detection result of the target item is invalid, the target item category information in the detection result is converted using the preset second training sample category mapping relationship information, and the similar item identification result of the target item is obtained based on the category conversion result.

[0022] The second training sample category mapping information refers to the correspondence between the training sample category in which the same training sample is located when it is used as a training sample corresponding to the detection network model and the similar training sample category in which it is located when it is used as a training sample corresponding to the classification network model. The similar training sample category is a training sample category with the same visual features, or one or more training sample categories at the same level in the training sample category table.

[0023] Optionally, after receiving the input image, the step of performing target detection on the input image using a preset detection network model to obtain the detection result of the target item includes:

[0024] After receiving the input image, the target object is detected in the input image using two or more preset detection network models.

[0025] The intermediate detection results output by each of the detection network models are fused to obtain the detection result of the target item; the intermediate detection result includes: target item category information, target item location information, and confidence information;

[0026] Wherein, the training samples corresponding to any two of the detection network models belong to different categories.

[0027] Optionally, the training samples corresponding to any two of the detection network models may be of completely different or partially different categories.

[0028] Optionally, the difference in the number of categories of training samples corresponding to any two detection network models is less than a preset first category difference; and the difference in the number of training samples of any two categories in the training samples corresponding to each detection network model is less than a first sample difference threshold.

[0029] Optionally, the two or more detection network models are trained using the following method:

[0030] Count the number of training samples in each category among all training samples, and determine the first category with the most training samples and the second category with the fewest training samples;

[0031] The training sample interval defined by the number of training samples in the first category to the number of training samples in the second category is evenly divided into K parts to obtain K-1 sample number separation values, where K is a positive integer greater than 1.

[0032] Based on K-1 sample count separators and the number of training samples for each category, the obtained training samples are separated into categories to obtain K training samples for the detection network model, and then trained.

[0033] Optionally, fusing the intermediate detection results output by each of the detection network models to obtain the detection result of the target item includes:

[0034] When the target item category information of the same target item exists only in one of the intermediate detection results, the intermediate detection result is used as the detection result of the target item.

[0035] When the target item category information of the same target item exists in two or more intermediate detection results, the detection result of the target item is obtained by combining the target item location information and confidence information in the two or more intermediate detection results.

[0036] Optionally, the step of combining the target item location information and confidence information from the two or more intermediate detection results to obtain the detection result of the target item includes:

[0037] For any intermediate detection result with a confidence level that is not the lowest among the two or more intermediate detection results, calculate the first area respectively, and perform the following operations on each of the first areas: calculate the overlapping area between the first area and each of the second areas, and perform a rejection operation on the corresponding intermediate detection result according to the size of the overlapping area, and take the intermediate detection result after the rejection operation among the two or more intermediate detection results as the detection result of the target item in this round;

[0038] The detection results of the target item in each round, and the intermediate detection result with the lowest confidence among the two or more intermediate detection results, are used together as the detection result of the target item.

[0039] Wherein, the first area is the target item area corresponding to the target item location information of the intermediate detection result, the second area is the target item area corresponding to the target item location information of the remaining intermediate detection results among the two or more intermediate detection results, and the confidence level of the intermediate detection result containing the first area is higher than the confidence level of the intermediate detection result containing the second area.

[0040] Optionally, the step of removing intermediate detection results based on the size of the overlapping area includes:

[0041] Based on the size of the overlapping area, determine the difference between the area of ​​each target item and the overlapping area in the two intermediate detection results.

[0042] When the difference between the area of ​​the target item and the overlapping area is less than the first area threshold, an elimination operation is performed on the intermediate detection results where the area of ​​the target item whose overlapping area difference is less than the first area threshold is located.

[0043] When the difference between the area of ​​the target item and the overlapping area is greater than the first area threshold, a rejection operation is performed based on the difference between the areas of the target items in the two intermediate detection results.

[0044] Optionally, the step of performing a rejection operation based on the difference between the areas of the target items in the two corresponding intermediate detection results includes:

[0045] Calculate the difference between the areas of the target item in the two corresponding intermediate detection results;

[0046] When the difference between the areas of the target items is less than the second area threshold, the intermediate detection results with lower confidence are removed.

[0047] Optionally, the step of performing the removal operation based on the difference between the areas of the target items in the two corresponding intermediate detection results further includes:

[0048] Use the target item location information from the removed intermediate detection results to update the target item location information from the retained intermediate detection results.

[0049] Optionally, the first area of ​​each intermediate detection result with a non-lowest confidence level is calculated in descending order of confidence level, and a removal operation is performed.

[0050] Optionally, the step of classifying the detection results of the target item using a preset classification network model to obtain the classification result of the target item includes:

[0051] When the detection result of the target item is received, the detection result of the target item is classified using two or more preset classification network models;

[0052] The intermediate classification results output by each of the classification network models are fused to obtain the classification result of the target item;

[0053] Wherein, the training samples corresponding to any two of the classification network models belong to different categories.

[0054] Optionally, the training samples corresponding to any two of the classification network models may be of completely different or partially different categories.

[0055] Optionally, in the training samples corresponding to each of the classification network models, the difference in the number of training samples between any two categories is less than the second sample difference threshold.

[0056] Optionally, the training samples corresponding to the two or more classification network models are determined by the following method:

[0057] From all training samples, select training samples with the same visual features, and use the training samples with the same visual features as the training samples corresponding to the same classification network model.

[0058] From the remaining training samples selected according to visual features, training samples with the same item usage characteristics are selected, and these training samples with the same item usage characteristics are used as training samples corresponding to the same classification network model.

[0059] From the remaining training samples selected based on visual features and item usage characteristics, training samples with the same item material are selected, and these training samples with the same item material are used as training samples corresponding to the same classification network model.

[0060] Optionally, the training samples corresponding to the two or more classification network models are further determined by the following method:

[0061] Training samples from two categories with different visual features but whose difference in the number of training samples is less than a preset third sample difference threshold are used as training samples for the same classification network model; the third sample difference threshold is less than the second sample difference threshold.

[0062] Optionally, fusing the intermediate classification results output by each of the classification network models to obtain the classification result of the target item includes:

[0063] When the target item category information of the same target item exists only in one intermediate classification result, the intermediate classification result shall be used as the classification result of the target item.

[0064] When the target item category information of the same target item exists only in two intermediate classification results, the intermediate classification result with higher confidence among the two intermediate classification results shall be used as the classification result of the target item.

[0065] When the target item category information of the same target item exists in more than three intermediate classification results, a preset number of intermediate classification results are selected as the classification result of the target item based on the confidence information of the three or more intermediate classification results.

[0066] This invention also provides a target recognition system, the system comprising:

[0067] The target detection unit is adapted to perform target detection on the input image using a preset detection network model after receiving the input image, and obtain the detection result of the target item; the detection result of the target item includes: target item category information, target item location information and confidence information;

[0068] The target classification unit is adapted to classify the detection results of the target item using a preset classification network model to obtain the classification result of the target item; the classification result of the target item includes: target item category information and confidence information;

[0069] The judgment unit is adapted to use the classification result of the target item to determine whether the detection result of the target item is valid, and to take the valid detection result of the target item as the identification result of the target item.

[0070] Optionally, the target detection unit includes:

[0071] Two or more target detection subunits, wherein the target detection subunits are adapted to perform target detection on the input image using a preset detection network model after receiving the input image;

[0072] The first fusion subunit is adapted to fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item; the intermediate detection result includes: target item category information, target item location information, and confidence information;

[0073] Wherein, the training samples corresponding to any two of the detection network models belong to different categories.

[0074] Optionally, the target classification unit includes:

[0075] Two or more target classification sub-units, wherein the target classification sub-units are adapted to classify the detection results of the target items using a preset classification network model when the detection results of the target items are received;

[0076] The second fusion subunit is adapted to fuse the intermediate classification results output by each of the classification network models to obtain the classification result of the target item;

[0077] Wherein, the training samples corresponding to any two of the classification network models belong to different categories.

[0078] Compared with the prior art, the technical solution of the embodiments of the present invention has the following advantages:

[0079] The solution of this invention, upon receiving an input image, first uses a detection network model to obtain the detection result of the target object, then uses a classification network model to obtain the classification result of the target object, and finally uses the classification result of the target object to determine whether the detection result of the target object is valid, thereby using the valid detection result of the target object as the recognition result of the target object. Compared with using only a single network model for target detection, the addition of subsequent classification and judgment steps can further improve the accuracy of target detection. Attached Figure Description

[0080] Figure 1 This is a flowchart of a target recognition method according to an embodiment of the present invention;

[0081] Figure 2 This is a flowchart of a method for obtaining detection results of a target item according to an embodiment of the present invention;

[0082] Figure 3 This is a schematic diagram of the detection results of a target item in an embodiment of the present invention;

[0083] Figure 4 This is a schematic diagram illustrating the calculation of the overlapping area between the frame to be inspected and other frames in an embodiment of the present invention;

[0084] Figure 5 This is a schematic diagram of the detection results of multiple target items in an embodiment of the present invention;

[0085] Figure 6 This is a flowchart of a method for obtaining the classification of target items in an embodiment of the present invention;

[0086] Figure 7 This is a schematic diagram of the structure of a target recognition system according to an embodiment of the present invention. Detailed Implementation

[0087] Existing deep learning-based object detection solutions typically use a single CNN or DNN for object detection, resulting in poor accuracy.

[0088] To address the aforementioned problems, this invention provides a target detection method. Applying the scheme described in this invention, after receiving an input image, a detection network model is first used to obtain the detection result of the target item, then a classification network model is used to obtain the classification result of the target item, and finally, the classification result of the target item is used to determine whether the detection result of the target item is valid. Thus, the valid detection result of the target item is used as the recognition result of the target item, effectively improving the accuracy of target detection.

[0089] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0090] Reference Figure 1 This invention provides a target recognition method, which may include the following steps:

[0091] Step 11: After receiving the input image, the input image is subjected to target detection using a preset detection network model to obtain the detection result of the target item.

[0092] The detection results of the target item include: target item category information, target item location information, and confidence level information.

[0093] In practice, you can use only one detection network model to detect objects in the input image, or you can use two or more detection network models to detect objects in the input image. There is no limitation here, as long as the detection result of the target object can be obtained.

[0094] Step 12: Using a preset classification network model, classify the detection results of the target item to obtain the classification result of the target item.

[0095] The classification results of the target items include: target item category information and confidence level information.

[0096] In practical applications, new images are constantly emerging, meaning that various unknown objects and stacked object patterns may appear. Therefore, the detection results of target objects obtained through detection network models may still contain false positives. Figure 5 As shown, metal bottle bottoms and metal strips may be mistaken for knives.

[0097] To reduce false positives and improve detection accuracy, embodiments of this invention utilize a pre-defined classification network to further determine the category of the target item. For a given target item, all its positive sample images reflect the item's correlation, which reveals the target item's characteristics. Therefore, the classification network model can learn the correlation features between these characteristic items and determine the detection result of the target item. The classification network model does not need to focus on the target item's location; it only needs to learn classification features to accurately determine the target item's category.

[0098] In practice, you can use only one classification network model to classify the detection results of the target item, or you can use two or more classification network models to classify the detection results of the target item. There is no limitation here, as long as the classification result of the target item can be obtained.

[0099] Step 13: Using the classification result of the target item, determine whether the detection result of the target item is valid, and take the valid detection result of the target item as the identification result of the target item.

[0100] In practice, the classification result of the target item can be directly used as the identification result of the target item, or the classification result of the target item can be used to further determine whether the detection result of the target item is valid, and the valid detection result of the target item can be used as the identification result of the target item, thereby further improving the accuracy of target detection.

[0101] Since the detection results of the target item include not only the category information of the target item, but also the location information of the target item, the classification results of the target item are used to remove invalid target item detection results from the numerous target item detection results, and the remaining target item detection results are used as the target item identification results.

[0102] In practice, various methods can be used to determine whether the detection results of the target item are valid, based on the classification results of the target item.

[0103] In one embodiment of the present invention, the target item category information in the detection result of the target item can be converted by using the preset first training sample category mapping relationship information, and the detection result of the target item can be determined based on the category conversion result to obtain the identification result of the target item.

[0104] The first training sample category mapping information is the correspondence between the training sample category when the same training sample is used as a training sample corresponding to the detection network model and the training sample category when it is used as a training sample corresponding to the classification network model.

[0105] In practice, a training sample can be used as a training sample for both a detection network model and a classification network model. When the training sample is used to train the detection network model, it belongs to category C1_i. When the training sample is used to train the classification network model, it belongs to category C2_j. The correspondence between categories C1_i and C2_j is the category mapping information of the training sample. This first training sample category mapping information may include category mapping information for multiple training samples.

[0106] In practice, the first training sample category mapping information can be obtained in advance, and the specific method of obtaining it is not limited.

[0107] In one embodiment of the present invention, based on the category conversion result, the following method can be used to determine whether the detection result of the target item is valid:

[0108] First, based on the category conversion result, it is determined whether the target item category information in the detection result of the target item matches the target item category information in the classification result of the target item, and a first judgment result is obtained. Then, based on the first judgment result, combined with the confidence information in the detection result of the target item and the classification result of the target item, it is determined whether the detection result of the target item is valid.

[0109] When the first judgment result is a match between the two, if the confidence level of either the detection result or the classification result of the target item is higher than the first confidence threshold, then the detection result of the target item is valid; otherwise, it is invalid.

[0110] When the first judgment result is that the two do not match, if the confidence level of either the detection result or the classification result of the target item is higher than the second confidence threshold, then the detection result of the target item is valid; otherwise, it is invalid. The second confidence threshold is greater than the first confidence threshold.

[0111] In practice, the intermediate detection result Boxn output by any detection network model Qn is used as the detection result Boxn of the target item. The specific content of Boxn is shown in Table 1.

[0112] Table 1

[0113] Boxn Network Qn Category C1_n Confidence level PQ Xmin Ymin Xmax Ymax

[0114] Table 2 shows the classification results of the target items.

[0115] Table 2

[0116] Network Hi Category C2_i Confidence level PH3 Network Hj Category C2_j Confidence level PH6

[0117] In the embodiments of the present invention, for ease of description, the category conversion result for any training sample category C1_n corresponding to any detection network model is f(C1_n).

[0118] If f(C1_n)∈{C2_i, C2_j}, that is, the result after category conversion belongs to {C2_i, C2_j}, it indicates that the detection result of the target item and the classification result of the target item both have the same prediction result, and the two match.

[0119] When the two match, if the confidence level PQ in the detection result of the target item is higher than the first confidence threshold, it indicates that the confidence level of the detection result Boxn of the target item is very high. In this case, the detection result Boxn of the target item can be directly determined to be valid; otherwise, the detection result Boxn of the target item is determined to be invalid. Alternatively, if the confidence level (PH3) of the corresponding category (e.g., category C2_i) in the classification result of the target item is higher than the second confidence threshold, it indicates that the target item largely belongs to category C2_i. In this case, the detection result Boxn of the target item can be directly determined to be valid; otherwise, the detection result Boxn of the target item is determined to be invalid.

[0120] When the two do not match, if the confidence level PQ in the detection result of the target item is higher than the third confidence threshold, it indicates that the confidence level of the detection result Boxn of the target item is extremely high. In this case, the detection result Boxn of the target item can be directly determined to be valid; otherwise, the detection result Boxn of the target item is determined to be invalid. Alternatively, if the confidence level (PH3) of the corresponding category (e.g., category C2_i) in the classification result of the target item is higher than the fourth confidence threshold, it indicates that the target item largely belongs to category C2_i. In this case, the detection result Boxn of the target item can be directly determined to be valid; otherwise, the detection result Boxn of the target item is determined to be invalid.

[0121] In practice, when the detection result of the target item is invalid, the category information of the target item in the detection result can be converted by using the preset second training sample category mapping relationship information, and the similar item identification result of the target item can be obtained based on the category conversion result.

[0122] The second training sample category mapping information refers to the correspondence between the training sample category in which the same training sample is located when it is used as a training sample corresponding to the detection network model and the similar training sample category in which it is located when it is used as a training sample corresponding to the classification network model. The similar training sample category is a training sample category with the same visual features, or one or more training sample categories at the same level in the training sample category table.

[0123] For example, when the training sample category corresponding to the detection network model is scissors, the similar training sample categories corresponding to the classification network model can be fruit knives and daggers. The correspondence between scissors and fruit knives and daggers is a second training sample category mapping relationship. After classifying the scissors category, f(C1_n) represents fruit knives and daggers. At this point, it is determined whether the fruit knife or dagger matches the classification result of the target item.

[0124] When the two match, if the confidence level PQ in the detection result of the target item is higher than the fifth confidence threshold, it indicates that the confidence level of the detection result Boxn of the target item is very high. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid. Here, the fifth confidence threshold is higher than the first confidence threshold. Alternatively, if the confidence level (PH3) of the corresponding category (e.g., category C2_i) in the classification result of the target item is higher than the sixth confidence threshold, it indicates that the target item largely belongs to category C2_i. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid. Here, the sixth confidence threshold is higher than the second confidence threshold.

[0125] When the two do not match, if the confidence level PQ in the detection result of the target item is higher than the seventh confidence threshold, it indicates that the confidence level of the detection result Boxn of the target item is extremely high. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid. Here, the seventh confidence threshold is higher than the third confidence threshold and is higher than the fifth confidence threshold. Alternatively, if the confidence level (PH3) of the corresponding category (e.g., category C2_i) in the classification result of the target item is higher than the eighth confidence threshold, it indicates that the target item largely belongs to category C2_i. In this case, the detection result Boxn of the target item can be directly determined to be valid. Otherwise, the detection result Boxn of the target item is determined to be invalid. Here, the eighth confidence threshold is higher than the fourth confidence threshold and is higher than the sixth confidence threshold.

[0126] It should be noted that, in actual implementation, the first confidence threshold to the eighth confidence threshold can be set according to the actual situation, which will not be elaborated here.

[0127] As can be seen from the above, the target recognition method in this embodiment of the invention can further improve the accuracy of target detection by using the classification results of the target item to determine whether the detection result of the target item is valid, and taking the valid detection result of the target item as the recognition result of the target item.

[0128] In one embodiment of the present invention, reference is made to... Figure 2 The detection results of the target item can be obtained using the following methods:

[0129] Step 21: After receiving the input image, the target object is detected in the input image using two or more preset detection network models.

[0130] Wherein, the training samples corresponding to any two of the detection network models belong to different categories.

[0131] In practice, the number of the two or more detection network models is not limited; for example, it can be two, three, or more, as long as the categories of the training samples corresponding to any two detection network models are different. The training samples corresponding to the detection network models are the training samples used to train the network models to obtain the detection network models.

[0132] Each detection network model can be obtained by training the same network model with training samples of different categories, or by training different network models with training samples of different categories. That is, the model structures corresponding to each detection network model can be the same or different, without any specific limitation. For example, each detection network model can be obtained by training a convolutional neural network model with training samples of different categories.

[0133] Understandably, different detection network models can use the same or different detection methods to detect the input image, depending on their model structure. If the model structures are the same, the detection methods will be the same; conversely, if the model structures are different, the detection methods will differ.

[0134] In practice, the categories of training samples corresponding to any two detection network models can be completely different or partially different, thus making the training samples corresponding to each detection network model not completely the same.

[0135] Taking two detection network models as an example, namely detection network model Q1 and detection network model Q2, the total number of categories in the training samples of the two detection network models is N, where N is an integer ≥ 2, and the set of N categories is C1. Specifically, the set of all categories in the training samples of detection network model Q1 is CQ1, and the set of all categories in the training samples of detection network model Q2 is CQ2, where CQ1 ∈ C1 and CQ2 ∈ C1. Sets CQ1 and CQ2 can contain the same category; that is, the elements of sets CQ1 and CQ2 can be partially the same. In other words, some categories can appear in both sets CQ1 and CQ2. However, at least one element in sets CQ1 and CQ2 must be different.

[0136] In practical applications, the distribution of actual training data is often uneven. Strictly speaking, any dataset is imbalanced, which causes detection network models to tend to learn features from a wider range of training samples. In the field of object detection, negative samples (i.e., images that do not contain the target object, such as images that do not contain contraband) are often far more numerous than positive samples (images that do contain the target object, such as images that contain contraband), resulting in a severe imbalance between positive and negative samples. This, in turn, leads to increased training costs and network model degradation.

[0137] For example, in subway security checks, prohibited items such as knives and guns are detected far less frequently than regular items like containers and umbrellas, resulting in unsatisfactory detection effectiveness. Taking the Shanghai Metro as an example, the average daily passenger flow exceeds 11 million. Shanghai Metro has 415 stations, equipped with over 6,000 security personnel and X-ray security scanners, conducting an average of 3.2 million package checks daily. A total of over 35,000 dangerous items are detected annually, averaging only 1.09 prohibited items per million packages. Moreover, prohibited items include many types, such as controlled knives and firearms, demonstrating a highly unbalanced distribution of positive and negative samples. Similarly, in satellite remote sensing, single images often have very high resolutions, such as 30,000*40,000, but the actual effective foreground is very limited, indicating an imbalance between foreground and background.

[0138] In embodiments of the present invention, in order to make the training samples more balanced, the difference in the number of categories of training samples corresponding to any two detection network models is less than a preset first category difference; and in the training samples corresponding to each detection network model, the difference in the number of training samples of any two categories is less than a first sample difference threshold.

[0139] By adjusting the first class difference, the training samples used for each detection network model can be more balanced across classes. By adjusting the first sample difference threshold, the number of training samples for each class can be more balanced. This avoids the impact of sample imbalance on the accuracy of target detection and improves the target detection effect.

[0140] For imbalanced training samples, in one embodiment of the present invention, the following method can be used for training to obtain the two or more detection network models:

[0141] First, we can count the number of training samples in each category across all training samples (i.e., the training sample set) and determine the first category with the most training samples and the second category with the fewest training samples.

[0142] Next, the training sample interval defined by the number of training samples in the first category to the number of training samples in the second category is evenly divided into K parts, resulting in K-1 sample quantity separation values, where K is a positive integer greater than 1.

[0143] Finally, based on the K-1 sample count separation values ​​and the number of training samples in each category, the obtained training samples are separated into categories to obtain K training samples for the detection network model, and then the model is trained.

[0144] Assume the training sample set includes N categories, with M1, M2, ..., MN training samples for each category. The first category, having the most training samples out of the N categories, has Mmax training samples. The second category, having the fewest training samples out of the N categories, has Mmin training samples.

[0145] To distinguish [Mmin, Mmax], divide it evenly into K parts, resulting in K sub-intervals. The value range of each sub-interval is as follows: [Mmin, Mmin+△], (Mmin+△, Mmin+2*△], ..., (Mmin+(K-1)*△, Mmax]. Where △=(Mmax-Mmin) / K.

[0146] Training samples from all categories whose training sample counts fall within the same sub-interval are used as training samples for a detection network model. For example, when K*2, training samples from all categories whose training sample counts fall within [Mmin, (Mmin+Mmax) / 2] are used as training samples for detection network model Q1. Training samples from all categories whose training sample counts fall within ((Mmin+Mmax) / 2, Mmax] are used as training samples for detection network model Q2.

[0147] In practical implementation, category adjustments can be made based on the specific features of each category. For example, easily confused or similar categories, such as scissors and knives, can be grouped into the same network to help the network learn the differences between the two items. Assuming the number of two categories with the same visual features is M1 and M2, and the detection network models are Q1 and Q2, two scenarios are possible:

[0148] 1) If the difference between M1 and M2 is too large, such as M1 / M2>β, where β is a constant greater than 1 (e.g., set to 3), then M1 can be updated to βM2 (for example, by using a random function to select βM2 samples from M1 samples), while M2 remains unchanged. These M1 training samples and M2 training samples are then used as training samples to train the same detection network model (e.g., detection network model Q1).

[0149] Meanwhile, to avoid significantly affecting the performance of detecting the category of the M1 training samples, Mavg' samples can be extracted from the M1 training samples based on the average value Mavg' of all training samples of the detection network model Q2, and used as training samples for training the detection network model Q2.

[0150] If M1 / M2 < 1 / β, the same method as described above shall be followed, and will not be repeated here.

[0151] 2) If the difference between M1 and M2 is not obvious, i.e., 1 / β < M1 / M2 < β, then the M1 training samples and the M2 training samples are used as the training samples for training the same detection network model.

[0152] Step 22: Fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item.

[0153] Among them, the intermediate detection results include: target item category information, target item location information, and confidence information.

[0154] In a specific implementation, the detection network model can output the intermediate detection results in the form of detection boxes. Each detection box has target item category information, target item location information, and confidence information. For example, the intermediate detection result output by the detection network model Qn: {network Qn, category n, confidence, Xnmin, Ynmin, Xnmax, Ynmax}, the category of the target item is n, (Xnmin, Ynmin) is the lower left vertex coordinate of the detection box output by the detection network model Qn, and (Xnmax, Ynmax) is the upper right vertex coordinate of the detection box output by the detection network model Qn, as Figure 3 shown in the rectangular box.

[0155] For the same input image, each detection network model will output an intermediate detection result. Taking the number of detection network models as M as an example, the identifiers of each detection network model are Q1, Q2,..., QM respectively. All the results output by the M detection network models are shown in Table 3:

[0156] Table 3

[0157]

[0158] In Table 3, Boxn is the identifier of the intermediate detection result, n ∈ {1, N}, category C1_n is the category identifier of the target item in the intermediate detection result Boxn, and PQn is the confidence value in the intermediate detection result Boxn.

[0159] In a specific implementation, various methods can be used to fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item, and specific limitations are not made.

[0160] In an embodiment of the present invention, when the target item category information of the same target item exists only in one of the intermediate detection results, the intermediate detection result where it is located can be used as the detection result of the target item.

[0161] For example, if the target item category C1_n in the intermediate detection result Boxn only exists as the intermediate detection result output by the detection network model Qn, then the intermediate detection result Boxn can be directly used as the detection result of the target item.

[0162] When the target item category information of the same target item exists in two or more intermediate detection results, the detection result of the target item is obtained by combining the target item location information and confidence information in the two or more intermediate detection results.

[0163] In specific implementation, when the target item category information of the same target item exists in two or more intermediate detection results, for any intermediate detection result with a non-lowest confidence level among the two or more intermediate detection results, a first area is calculated, and the following operations are performed on each of the first areas: the overlapping area between the first area and each second area is calculated, and the corresponding intermediate detection result is eliminated according to the size of the overlapping area. The intermediate detection result after the elimination operation among the two or more intermediate detection results is taken as the detection result of the target item in this round; the detection results of the target item in each round and the intermediate detection result with the lowest confidence level among the two or more intermediate detection results are jointly taken as the detection result of the target item.

[0164] Wherein, the first area is the target item area corresponding to the target item location information of the intermediate detection result, the second area is the target item area corresponding to the target item location information of the remaining intermediate detection results among the two or more intermediate detection results, and the confidence level of the intermediate detection result containing the first area is higher than the confidence level of the intermediate detection result containing the second area.

[0165] In one embodiment of the present invention, the step of removing intermediate detection results based on the size of the overlapping area may include:

[0166] Based on the size of the overlapping area, determine the difference between the area of ​​each target item and the overlapping area in the two intermediate detection results.

[0167] When the difference between the area of ​​the target item and the overlapping area is less than the first area threshold, an elimination operation is performed on the intermediate detection results where the area of ​​the target item whose overlapping area difference is less than the first area threshold is located.

[0168] When the difference between the area of ​​the target item and the overlapping area is greater than the first area threshold, a rejection operation is performed based on the difference between the areas of the target items in the two intermediate detection results.

[0169] For example, when the category C1_n of the target item in the intermediate detection result Boxn exists in the intermediate detection results output by M detection network models at the same time, the intermediate detection results regarding the category C1_n in each detection network model can be extracted first.

[0170] The intermediate detection results regarding the category C1_n are stored in the temporary storage space TOn. For example, the temporary storage space for the category C1_n is TO1, and the temporary storage space for the category C1_2 is TO2, and finally the storage spaces for all categories {TO1,..., TOn,..., TON} are obtained. Process the storage space TOn for each category respectively. At this time, the stored content of TOn (assuming there are n' detection boxes in total) is shown in Table 4 specifically:

[0171] Table 4

[0172]

[0173] Among them, Boxn’ is the intermediate detection result regarding the category C1_n in the detection network model Qn, where n ∈ {1, N}.

[0174] Taking the fusion of each intermediate detection result of the category C1_n as an example, the specific fusion process is as follows:

[0175] a) Sort all the detection boxes in descending order of confidence.

[0176] b) Take the detection box with the highest confidence as the box to be detected, and calculate the overlapping area S between the box to be detected and another box (with a lower confidence than the box to be detected) in turn, and obtain the detection result of the target item in this round.

[0177] Taking the operation performed on the box to be detected and one of the other boxes as an example, as Figure 4 shown, the area of the box to be detected is the first area S1, and the area of one of the other boxes is the second area S2. At this time, the overlapping coefficient α of the two boxes can be calculated. <@

[0178] When S1 > γS2, where γ is a constant greater than 1, at this time S1 is much larger than S2, and the overlapping coefficient α = S / S2. If α exceeds the first preset overlapping threshold G1, it indicates that the overlapping area S is close to the second area S, and the other box can be considered to be included in the box to be detected, so the other box can be removed from TOn.

[0179] When S1 < S2 / γ, at this time S1 is much smaller than S2, and α = S / S1. If α exceeds the second preset overlapping threshold G2, it indicates that the overlapping area S is close to the first area S1, and the box to be detected can be considered to be included in the other box, so the box to be detected can be removed from TOn.

[0180] When S2 / γ ≤ S1 ≤ γS2, the first area S1 and the second area S2 are roughly equivalent, and α = S / (S1 + S2 - S). If α exceeds the third preset overlap threshold G3, it indicates that the first area S1 and the second area S2 are close, and the detection effect of the bounding box and other boxes is comparable. In this case, only one bounding box needs to be retained. For example, the bounding box with high confidence can be selected, and other boxes can be removed from TOn.

[0181] Among them, γ, the first preset overlap threshold G1, the second preset overlap threshold G2 and the third preset overlap threshold G3 are all preset constants, and the specific values ​​corresponding to different categories may differ.

[0182] In one embodiment of the present invention, when the detection effects of the frame to be inspected and other frames are comparable, if the area of ​​the target item in the rejected intermediate detection results is greater than the area of ​​the target item in the retained intermediate detection results, the target item location information of the rejected intermediate detection results can be used to update the target item location information of the retained intermediate detection results.

[0183] For example, when other bounding boxes are removed, if S2 > S1, to minimize the impact of removing the bounding box, the confidence level of the bounding box can be kept unchanged, and the target item position information in the bounding box can be updated using the target item position information in other bounding boxes, making the area of ​​the bounding box larger and the detection output coverage wider. If S2 ≤ S1, it means that the area and confidence level of the bounding box are both maximum, so no update is needed.

[0184] After performing the above operation on the target box and another box, the detection result of the target item in this round is obtained. Using the detection result of the target item in this round, the storage space Ton is updated to obtain the updated storage space TOn', which is used as the output of category C1_n.

[0185] c) After one round of traversal, start traversing from the second highest confidence box until the lowest confidence box, which requires a total of n'-1 rounds, and then obtain the fused output of all categories {TO1', ..., TOn', ..., TON'}, as the detection result of the target item.

[0186] In practical implementation, based on the target item's location information (the (Xmin, Ymin) and (Xmax, Ymax) parameters of the detection box) in the detection results, a complete image can be extracted, such as... Figure 5 As shown, images 51 and 52 are the extracted bottle bottom images, image 53 is the extracted metal strip image, and images 54 and 55 are the extracted fruit knife images. The actual target object detection result may be only one target object image, or it may be multiple target object images, for example... Figure 5As shown, when the target item is the bottom of the bottle, the detection result of the target item may include both image 51 and image 52.

[0187] As can be seen from the above, the method for obtaining the target item detection result in the embodiments of the present invention can make the training samples of the detection network model more balanced, and by using two or more detection network models to perform target detection simultaneously, and finally fusing the intermediate results output by each detection network model to obtain the target item detection result, the target item detection result can be more accurate.

[0188] In another embodiment of the present invention, reference is made to Figure 6 The following method can be used to obtain the classification results of the target item:

[0189] Step 61: When the detection result of the target item is received, the detection result of the target item can be classified using two or more preset classification network models.

[0190] Wherein, the training samples corresponding to any two of the classification network models belong to different categories.

[0191] In practice, the number of classification network models can be two, three, or more; there is no specific limitation, as long as the categories of the training samples corresponding to any two classification network models are different. The training samples corresponding to the two classification network models are the training samples used to train the classification network models.

[0192] Each classification network model can be obtained by training the same network model with training samples from different categories, or by training different network models with training samples from different categories. That is, the model structures corresponding to each classification network model can be the same or different, without any specific restrictions. For example, each classification network model can be obtained by training a convolutional neural network model with training samples from different categories.

[0193] Understandably, different classification network models can use the same classification method to categorize the detection results of target items, or they can use different classification methods, depending on the model structure of each network model. If the model structures of different classification network models are the same, the way they categorize the detection results of target items will also be the same; conversely, if the model structures of different classification network models are different, the way they categorize the detection results of target items will also be different.

[0194] In practice, the categories of training samples corresponding to any two classification network models can be completely different or partially different.

[0195] Taking a classification network model with L components as an example, the labels of each classification network model are H1, H2, ..., HL (L≥2). The total number of categories in the training samples corresponding to the L classification network models is C2, which is usually different from the total number of categories in the training samples corresponding to the detection network model, and can include some categories that are prone to false positives. There can be overlap between the categories in the training samples corresponding to the classification network models. In other words, for any category n, n∈C2, category n can appear simultaneously in the training samples corresponding to classification network models H1, H2, ..., and HL.

[0196] In practice, the training of the classification network model and the detection network model are independent of each other, each with its own training sample set. The training sample set for the detection network model consists of a large number of images, while the training sample set for the classification network can consist of a large number of bounding boxes. The training methods for each classification network model can be the same or different.

[0197] To improve the accuracy of category determination by the classification model, the difference in the number of training samples for any two categories within the training samples corresponding to each classification network model should be less than a second sample difference threshold. This second sample difference threshold can be set to ensure that the difference in the number of training samples for any two categories within the same classification network model is small.

[0198] For example, in the training sample set of a classification network model, the category with the fewest training samples is category 1, and the number of training samples for category 1 is Mcmin. In this case, if category 1 is used as the training network for classification network model H1, then the number of training samples for other categories corresponding to classification network model H1 is at most βMCmin.

[0199] If the number of training samples for other categories in the training sample set exceeds βMCmin, then βMCmin training samples are used to train the classification network model H1. If the number of training samples for other categories in the training sample set does not exceed βMCmin, then all training samples for those other categories can be used to train the classification network model H1.

[0200] In one embodiment of the present invention, the training samples corresponding to the two or more classification network models can be determined by the following method:

[0201] First, from all training samples (i.e., the training set corresponding to the classification network model), training samples with the same visual features are selected, and training samples with the same visual features are used as training samples corresponding to the same classification network model.

[0202] In practical implementation, models should share similar visual characteristics, i.e., similar shapes. For example, the shapes of a metal bottle bottom, a metal strip, and a fruit knife are similar; in this case, the metal bottle bottom, the metal strip, and the fruit knife can be used as training samples for training the classification network model A1. Similarly, the shapes of a metal bottle and a metal gas canister are similar; in this case, the metal bottle and the metal gas canister can be used as training samples for training the classification network model A2.

[0203] Next, from the remaining training samples selected according to visual features, training samples with the same item usage characteristics are selected, and these training samples with the same item usage characteristics are used as training samples corresponding to the same classification network model.

[0204] In practice, items with the same usage characteristics, such as wrenches, pliers, and saws, can be used as training samples for the same classification network model. In this case, the difference between the number of training samples with the same usage characteristics and the number of training samples for other categories already in the classification network model should be less than the second sample difference threshold.

[0205] Finally, from the remaining training samples selected according to visual features and item usage characteristics, training samples with the same item material are selected, and these training samples with the same item material are used as training samples corresponding to the same classification network model.

[0206] In practice, items of the same material, such as mobile phones, laptops, and monitors, can be used as training samples for the same classification network model. In this case, the difference between the number of samples with the same material and the number of training samples for other categories in the classification network model should be less than the second sample difference threshold.

[0207] In one embodiment of the present invention, training samples of two categories with different visual features but a difference in the number of training samples less than a preset third sample difference threshold can be used as training samples corresponding to the same classification network model; the third sample difference threshold is less than the second sample difference threshold.

[0208] In other words, training samples from two categories with very different visual features but similar numbers of training samples can be used as training samples for the same classification network model.

[0209] Step 62: Fuse the intermediate classification results output by each of the classification network models to obtain the classification result of the target item.

[0210] In practice, the detection results of the target item can be output in the form of a detection box, as shown in Table 1.

[0211] After passing through the classification network model Hi, the intermediate classification results output by the classification network model Hi may be as shown in Table 5:

[0212] Table 5

[0213]

[0214] As can be seen from Table 5, the classification network model Hi determines that the target item in the detection result Boxn belongs to category C2_1, category C2_2 and category C2_i.

[0215] After passing through the classification network model Hj, the intermediate classification results output by the classification network model Hj may be as shown in Table 6:

[0216] Table 6

[0217]

[0218] As can be seen from Table 6, the target items in Boxn, determined by the classification network model Hj, belong to categories C2_1, C2_2, and C2_j.

[0219] Different classification network models output different intermediate classification results.

[0220] In practice, various methods can be used to fuse the intermediate classification results output by each classification network model to obtain the classification result of the target item.

[0221] In one embodiment of the present invention, when the target item category information of the same target item exists only in one intermediate classification result, the intermediate classification result is used as the classification result of the target item.

[0222] When the target item category information of the same target item exists only in two intermediate classification results, the intermediate classification result with higher confidence among the two intermediate classification results shall be used as the classification result of the target item.

[0223] When the target item category information of the same target item exists in more than three intermediate classification results, a preset number of intermediate classification results are selected as the classification result of the target item based on the confidence information of the three or more intermediate classification results.

[0224] For example, assuming the classification result of the target item is OH, if category C2_i only exists in the intermediate classification results of the classification network model Hi, then category C2_i and its corresponding confidence PH3 will be retained in the classification result OH of the target item.

[0225] If category C2_1 exists only in classification network models Hi and Hj, and PH1 < PH4, then the confidence level PH4 corresponding to C2_1 will be retained in the classification result OH of the target item.

[0226] If category C2_2 exists in more than three intermediate classification results, and if the confidence level corresponding to category C2_2 in multiple intermediate classification results is extremely high, for example, if the confidence level corresponding to category C2_2 is greater than GP1, then a small number (e.g., three) of the intermediate classification results with the highest confidence level can be selected and retained in the classification result OH of the target item.

[0227] If the confidence scores for all intermediate classification results for category C2_2 are low (e.g., all confidence scores for category C2_2 are less than GP2), then a small number (e.g., one) of the intermediate classification results with the highest confidence scores can be selected and retained in the target item's classification result OH. Here, GP2 is less than GP1.

[0228] If the highest confidence level for category C2_2 is less than GP1 and the lowest confidence level is greater than GP2 among all intermediate classification results, then the highest confidence levels (e.g., five) of the intermediate classification results can be selected and retained in the classification result OH of the target item.

[0229] As can be seen from the above, the method for obtaining target item classification results in this embodiment of the invention can further improve the accuracy of target detection by using a classification network model to further determine the category of the target item after obtaining the target item detection results.

[0230] To enable those skilled in the art to better understand and implement the present invention, the target recognition system corresponding to the above method is described in detail below.

[0231] Reference Figure 7 The present invention provides a target recognition system 70, which includes: a target detection unit 71, a target classification unit 72, and a judgment unit 73. Wherein:

[0232] The target detection unit 71 is adapted to perform target detection on the input image using a preset detection network model after receiving the input image, and obtain the detection result of the target item; the detection result of the target item includes: target item category information, target item location information and confidence information;

[0233] The target classification unit 72 is adapted to classify the detection results of the target item using a preset classification network model to obtain the classification result of the target item; the classification result of the target item includes: target item category information and confidence information;

[0234] The judgment unit 73 is adapted to use the classification result of the target item to determine whether the detection result of the target item is valid, and to use the valid detection result of the target item as the identification result of the target item.

[0235] In one embodiment of the present invention, the target detection unit 71 may include: two or more target detection subunits 711 and a first fusion subunit 712. Wherein:

[0236] The target detection subunit 711 is adapted to perform target detection on the input image using a preset detection network model after receiving the input image;

[0237] The first fusion subunit 712 is adapted to fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item; the intermediate detection result includes: target item category information, target item location information and confidence information;

[0238] Wherein, the training samples corresponding to any two of the detection network models belong to different categories.

[0239] In one embodiment of the present invention, the target classification unit 72 may include: two or more target classification subunits 721 and a second fusion subunit 722. Wherein:

[0240] The target classification subunit 721 is adapted to classify the detection result of the target item using a preset classification network model when the detection result of the target item is received.

[0241] The second fusion subunit 722 is adapted to fuse the intermediate classification results output by each of the classification network models to obtain the classification result of the target item;

[0242] Wherein, the training samples corresponding to any two of the classification network models belong to different categories.

[0243] As can be seen from the above, the target recognition system 70 in this embodiment of the invention can effectively improve the accuracy of target detection compared to using a single network model for target detection.

[0244] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A target recognition method, characterized in that, include: After receiving the input image, the preset detection network model is used to perform target detection on the input image to obtain the detection result of the target item; The detection results of the target item include: target item category information, target item location information, and confidence level information; Using a pre-defined classification network model, the detection results of the target item are classified to obtain the classification result of the target item; the classification result of the target item includes: target item category information and confidence information; Using the classification results of the target item, determine whether the detection result of the target item is valid, and take the valid detection result of the target item as the identification result of the target item; The step of performing target detection on the input image using preset detection network models after receiving the input image to obtain the target item detection result includes: after receiving the input image, performing target item detection on the input image using two or more preset detection network models; fusing the intermediate detection results output by each detection network model to obtain the target item detection result; wherein the training samples corresponding to any two detection network models are of different categories; the detection network model outputs the intermediate detection result in the form of detection boxes, and each detection box has target item category information, target item location information, and confidence information; The step of fusing the intermediate detection results output by each of the detection network models to obtain the detection result of the target item includes: when the target item category information of the same target item exists only in one intermediate detection result, the intermediate detection result in which it exists is taken as the detection result of the target item; when the target item category information of the same target item exists in two or more intermediate detection results, the target item location information and confidence information in the two or more intermediate detection results are combined to obtain the detection result of the target item. The step of combining the target item location information and confidence information from the two or more intermediate detection results to obtain the detection result of the target item includes: for any intermediate detection result with a non-lowest confidence among the two or more intermediate detection results, calculating a first area, and performing the following operations on each first area: calculating the overlapping area between the first area and each second area, and performing a rejection operation on the corresponding intermediate detection result according to the size of the overlapping area, and taking the intermediate detection result after the rejection operation from the two or more intermediate detection results as the detection result of the target item in this round; taking the detection results of the target item in each round and the intermediate detection result with the lowest confidence among the two or more intermediate detection results as the detection result of the target item; wherein, the first area is the target item area corresponding to the target item location information of the intermediate detection result, the second area is the target item area corresponding to the target item location information of the remaining intermediate detection results from the two or more intermediate detection results, and the confidence in the intermediate detection result containing the first area is higher than the confidence in the intermediate detection result containing the second area.

2. The target recognition method as described in claim 1, characterized in that, The step of using the classification result of the target item to determine whether the detection result of the target item is valid includes: Using the preset first training sample category mapping relationship information, the category information of the target item in the detection result of the target item is converted into a category, and based on the category conversion result, it is determined whether the detection result of the target item is valid, and the recognition result of the target item is obtained. The first training sample category mapping information is the correspondence between the category of the same training sample when it is used as a training sample for the detection network model and the category of the same training sample when it is used as a training sample for the classification network model.

3. The target recognition method as described in claim 2, characterized in that, The determination of the validity of the detection result of the target item based on the category conversion result includes: Based on the category conversion result, determine whether the target item category information in the detection result of the target item matches the target item category information in the classification result of the target item, and obtain the first judgment result; Based on the first judgment result, and combined with the detection result of the target item and the confidence information in the classification result of the target item, it is determined whether the detection result of the target item is valid.

4. The target recognition method as described in claim 3, characterized in that, The step of determining whether the detection result of the target item is valid based on the first judgment result, combined with the detection result of the target item and the confidence information in the classification result of the target item, includes: When the first judgment result is a match between the two, if the confidence level in the detection result of the target item is higher than the first confidence level threshold, or the confidence level in the classification result of the target item is higher than the second confidence level threshold, then the detection result of the target item is valid; otherwise, it is invalid. When the first judgment result is that the two do not match, if the confidence level of the detection result of the target item is higher than the third confidence level threshold, or the confidence level of the classification result of the target item is higher than the fourth confidence level threshold, then the detection result of the target item is valid; otherwise, it is invalid. Wherein, the third confidence threshold is greater than the first confidence threshold, and the fourth confidence threshold is greater than the second confidence threshold.

5. The target recognition method as described in claim 1, characterized in that, Also includes: When the detection result of the target item is invalid, the target item category information in the detection result is converted using the preset second training sample category mapping relationship information, and the similar item identification result of the target item is obtained based on the category conversion result. The second training sample category mapping information is the correspondence information between the training sample category in which the same training sample is located when it is used as a training sample corresponding to the detection network model and the similar training sample category in which it is located when it is used as a training sample corresponding to the classification network model. The similar training sample categories are training sample categories with the same visual features, or one or more training sample categories at the same level in the training sample category table.

6. The target recognition method as described in claim 1, characterized in that, The training samples corresponding to any two of the aforementioned detection network models are of completely different categories, or partially different categories.

7. The target recognition method as described in claim 1, characterized in that, The difference in the number of categories of training samples corresponding to any two of the detection network models is less than a preset first category difference; the difference in the number of training samples of any two categories in the training samples corresponding to each detection network model is less than a first sample difference threshold.

8. The target recognition method as described in claim 1, characterized in that, The two or more detection network models were trained using the following method: Count the number of training samples in each category among all training samples, and determine the first category with the most training samples and the second category with the fewest training samples; The training sample interval defined by the number of training samples in the first category to the number of training samples in the second category is evenly divided into K parts to obtain K-1 sample number separation values, where K is a positive integer greater than 1. Based on K-1 sample count separators and the number of training samples for each category, the obtained training samples are separated into categories to obtain K training samples for the detection network model, and then trained.

9. The target recognition method as described in claim 8, characterized in that, The step of removing intermediate detection results based on the size of the overlapping area includes: Based on the size of the overlapping area, determine the difference between the area of ​​each target item and the overlapping area in the two intermediate detection results. When the difference between the area of ​​the target item and the overlapping area is less than the first area threshold, an elimination operation is performed on the intermediate detection results where the area of ​​the target item whose overlapping area difference is less than the first area threshold is located. When the difference between the area of ​​the target item and the overlapping area is greater than the first area threshold, a rejection operation is performed based on the difference between the areas of the target items in the two intermediate detection results.

10. The target recognition method as described in claim 9, characterized in that, The step of performing a removal operation based on the difference between the areas of the target items in the two intermediate detection results includes: Calculate the difference between the areas of the target item in the two corresponding intermediate detection results; When the difference between the areas of the target items is less than the second area threshold, the intermediate detection results with lower confidence are removed.

11. The target recognition method as described in claim 10, characterized in that, The step of performing a removal operation based on the difference between the areas of the target items in the two intermediate detection results further includes: Use the target item location information from the removed intermediate detection results to update the target item location information from the retained intermediate detection results.

12. The target recognition method as described in claim 8, characterized in that, Calculate the first area of ​​each intermediate detection result that is not the lowest confidence level in descending order of confidence level, and perform a removal operation.

13. The target recognition method as described in claim 1, characterized in that, The step of classifying the detection results of the target item using a preset classification network model to obtain the classification result of the target item includes: When the detection result of the target item is received, the detection result of the target item is classified using two or more preset classification network models; The intermediate classification results output by each of the classification network models are fused to obtain the classification result of the target item; Wherein, the training samples corresponding to any two of the classification network models belong to different categories.

14. The target recognition method as described in claim 13, characterized in that, The training samples corresponding to any two of the classification network models are in completely different categories, or partially different categories.

15. The target recognition method as described in claim 13, characterized in that, In the training samples corresponding to each of the classification network models, the difference in the number of training samples between any two categories is less than the second sample difference threshold.

16. The target recognition method as described in claim 15, characterized in that, The training samples corresponding to the two or more classification network models are determined by the following method: from all training samples, training samples with the same visual features are selected, and training samples with the same visual features are used as training samples corresponding to the same classification network model. From the remaining training samples selected according to visual features, training samples with the same item usage characteristics are selected, and these training samples with the same item usage characteristics are used as training samples corresponding to the same classification network model. From the remaining training samples selected based on visual features and item usage characteristics, training samples with the same item material are selected, and these training samples with the same item material are used as training samples corresponding to the same classification network model.

17. The target recognition method as described in claim 15, characterized in that, The training samples corresponding to the two or more classification network models are also determined by the following method: Training samples of two categories with different visual features but a difference in the number of training samples less than a preset third sample difference threshold are used as training samples corresponding to the same classification network model. The third sample difference threshold is less than the second sample difference threshold.

18. The target recognition method as described in claim 13, characterized in that, The process of fusing the intermediate classification results output by each of the classification network models to obtain the classification result of the target item includes: When the target item category information of the same target item exists only in one intermediate classification result, the intermediate classification result shall be used as the classification result of the target item. When the target item category information of the same target item exists only in two intermediate classification results, the intermediate classification result with higher confidence among the two intermediate classification results shall be used as the classification result of the target item. When the target item category information of the same target item exists in more than three intermediate classification results, a preset number of intermediate classification results are selected as the classification result of the target item based on the confidence information of the three or more intermediate classification results.

19. A target recognition system, characterized in that, include: The target detection unit is adapted to perform target detection on the input image using a preset detection network model after receiving the input image, and obtain the detection result of the target item; The detection results of the target item include: target item category information, target item location information, and confidence level information; The target classification unit is adapted to classify the detection results of the target item using a preset classification network model to obtain the classification result of the target item; the classification result of the target item includes: target item category information and confidence information; The judgment unit is adapted to use the classification result of the target item to determine whether the detection result of the target item is valid, and to take the valid detection result of the target item as the recognition result of the target item; The target detection unit includes: two or more target detection subunits, each adapted to perform target detection on the input image using a preset detection network model after receiving the input image; a first fusion subunit, adapted to fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item; the intermediate detection result includes: target item category information, target item location information, and confidence information; wherein, the categories of the training samples corresponding to any two of the detection network models are different; the detection network model outputs the intermediate detection result in the form of detection boxes, each detection box having target item category information, target item location information, and confidence information; The first fusion subunit is adapted to take the intermediate detection result as the detection result of the target item when the target item category information of the same target item exists in only one intermediate detection result; when the target item category information of the same target item exists in two or more intermediate detection results, the detection result of the target item is obtained by combining the target item location information and confidence information in the two or more intermediate detection results. The step of combining the target item location information and confidence information from the two or more intermediate detection results to obtain the detection result of the target item includes: for any intermediate detection result with a non-lowest confidence among the two or more intermediate detection results, calculating a first area, and performing the following operations on each first area: calculating the overlapping area between the first area and each second area, and performing a rejection operation on the corresponding intermediate detection result according to the size of the overlapping area, and taking the intermediate detection result after the rejection operation from the two or more intermediate detection results as the detection result of the target item in this round; taking the detection results of the target item in each round and the intermediate detection result with the lowest confidence among the two or more intermediate detection results as the detection result of the target item; wherein, the first area is the target item area corresponding to the target item location information of the intermediate detection result, the second area is the target item area corresponding to the target item location information of the remaining intermediate detection results from the two or more intermediate detection results, and the confidence in the intermediate detection result containing the first area is higher than the confidence in the intermediate detection result containing the second area.

20. The target recognition system as described in claim 19, characterized in that, The target detection unit includes: Two or more target detection subunits, wherein the target detection subunits are adapted to perform target detection on the input image using a preset detection network model after receiving the input image; The first fusion subunit is adapted to fuse the intermediate detection results output by each of the detection network models to obtain the detection result of the target item; the intermediate detection result includes: target item category information, target item location information, and confidence information; Wherein, the training samples corresponding to any two of the detection network models belong to different categories.

21. The target recognition system as described in claim 19, characterized in that, The target classification unit includes: Two or more target classification sub-units, wherein the target classification sub-units are adapted to classify the detection results of the target items using a preset classification network model when the detection results of the target items are received; The second fusion subunit is adapted to fuse the intermediate classification results output by each of the classification network models to obtain the classification result of the target item; Wherein, the training samples corresponding to any two of the classification network models belong to different categories.

Images