Pseudo-label adding method and device, nonvolatile storage medium, and electronic device

Abstract

The application discloses a pseudo-label adding method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: obtaining multiple training images with different class labels from a preset image training set as to-be-fused images, wherein the preset image training set comprises multiple training images with added class labels, and the class label is used for indicating the class of an instance object in the training image; fusing the instance objects in the multiple to-be-fused images to generate a to-be-recognized image comprising a fused object, wherein the fused object is the fusion result of the instance objects; analyzing the to-be-recognized image by using a preset prediction model to determine multiple detection boxes of the fused object and the class label of each detection box; and adding a pseudo-label representing an unknown class to the to-be-recognized image in the case that the fused object has detection boxes with inconsistent class labels. The application solves the technical problem that the recognition result is inaccurate for an object without a pre-labeled class.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method and device for adding a pseudo label, a non-volatile storage medium, and an electronic device. Background Art

[0002] In open scenarios, semantic understanding of the real world is often necessary. For example, in the field of autonomous driving, objects on the road must be detected to understand their semantics and make decisions to plan vehicle control operations. However, in the real open world, the types of objects are endless. If a certain type of obstacle is not detected, it may lead to an accident and cause unavoidable losses.

[0003] Since the categories of targets are endless, there are two main methods for detecting targets in open scenes. The first method requires labeling training for each category, and then re-labeling training when new categories appear. This method solves the problem of target detection at the data level, but it also brings higher labor and time costs for target detection applications in real scenes. The second method is to add other sensors, such as radar or millimeter wave sensors, to understand the semantic information of the image. However, this method increases the hardware cost of the product and reduces the product's competitiveness.

[0004] Furthermore, in open scenes, when identifying objects that are not pre-labeled, the recognition result should be the object category with the highest similarity to the object, which may cause misjudgment and inaccurate recognition results.

[0005] With regard to the above-mentioned problem of inaccurate recognition results for objects that are not pre-labeled, no effective solution has been proposed so far. Summary of the Invention

[0006] Embodiments of the present invention provide a method and apparatus for adding pseudo labels, a non-volatile storage medium, and an electronic device, to at least solve the technical problem of inaccurate recognition results for objects that are not pre-labeled.

[0007] According to one aspect of an embodiment of the present invention, a method for adding pseudo labels is provided, comprising: obtaining a plurality of training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set comprises: a plurality of training images to which the category labels have been added, and the category labels are used to indicate the categories of instance objects in the training images; fusing the instance objects in the plurality of images to be fused to generate an image to be recognized, wherein the image to be recognized comprises: a fusion object, which is a fusion result of the instance objects; analyzing the image to be recognized using a preset prediction model to determine a plurality of detection frames of the fusion object and a category label of each of the detection frames, wherein the preset prediction model is trained using a preset image training set, and the preset prediction model is used to represent the association between the sample frames of the instance objects in the training images and the category labels; in the event that there are detection frames with inconsistent category labels in the fusion object, adding a pseudo label representing an unknown category to the image to be recognized.

[0008] Optionally, fusing instance objects in multiple images to be fused to generate an image to be identified includes: identifying an instance region in each of the images to be fused, wherein the instance region includes: the instance object; segmenting the multiple images to be fused to obtain a set of image blocks, wherein the image block set includes: multiple image blocks segmented based on the instance region; randomly selecting multiple image blocks in the image block set for splicing to obtain a fused region, wherein the fused region includes the fused object; pasting the fused region to the image to be identified to generate the image to be identified.

[0009] Optionally, randomly selecting multiple image blocks from the image block set for splicing to obtain the fusion area includes: determining an image block subset for each image to be fused in the image block set; and randomly selecting a preset number of image blocks in each image block subset for splicing to obtain the fusion area.

[0010] Optionally, using a preset prediction model to analyze the image to be identified to determine multiple detection frames of the fusion object, and the category label of each detection frame includes: using a preset prediction model to analyze the image to be identified to determine multiple candidate frames of the fusion object; determining the intersection-and-union ratio of the multiple candidate frames; selecting multiple candidate frames whose intersection-and-union ratio is greater than a preset threshold as multiple detection frames of the fusion object; and determining the category label of each detection frame.

[0011] Optionally, after analyzing the image to be identified using a preset prediction model to determine multiple detection frames of the fused object and the category label of each detection frame, the method further includes: detecting whether the category labels of the multiple detection frames of the fused object are consistent; and when the category labels of the detection frames are consistent, adding a pseudo label consistent with the category label to the fused object.

[0012] Optionally, when there are detection frames with inconsistent category labels in the fusion object, adding a pseudo label representing an unknown category to the image to be identified includes: when there are detection frames with inconsistent category labels in the fusion object, determining the confidence of each detection frame; retaining the detection frame with the highest confidence as the predicted frame in the image to be identified; and adding a pseudo label representing an unknown category to the predicted frame.

[0013] Optionally, when there is a detection frame with inconsistent category labels in the fusion object, after adding a pseudo label representing an unknown category to the image to be identified, the method further includes: putting the image to be identified with the pseudo label added into the preset image training set.

[0014] According to another aspect of an embodiment of the present invention, a pseudo-label adding device is also provided, comprising: an acquisition module for acquiring a plurality of training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set comprises: a plurality of training images to which the category labels have been added, and the category labels are used to indicate the categories of instance objects in the training images; a generation module for fusing the instance objects in the plurality of images to be fused to generate an image to be identified, wherein the image to be identified comprises: a fusion object, which is a fusion result of the instance objects; an analysis module for analyzing the image to be identified using a preset prediction model to determine a plurality of detection frames of the fusion object and a category label of each detection frame, wherein the preset prediction model is trained using a preset image training set, and the preset prediction model is used to represent the association between the sample frame of the instance object in the training image and the category label; an adding module for adding a pseudo-label representing an unknown category to the image to be identified when there is a detection frame with inconsistent category labels in the fusion object.

[0015] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided. The non-volatile storage medium is used to store a program. When the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned pseudo-label adding method.

[0016] According to another aspect of an embodiment of the present invention, an electronic device is provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the processor, wherein the program executes the above-mentioned method for adding pseudo labels when running.

[0017] In an embodiment of the present invention, a fusion object of an unknown category is constructed based on instance objects of known categories in a training image to which category labels have been added, and a pseudo label representing the unknown category is added to the image to be identified with the fusion object, thereby realizing the construction of sample images of unknown categories and the addition of pseudo labels. Furthermore, the training image to which category labels have been added and the image to be identified to which pseudo labels representing the unknown category have been added are used as training samples for training an image recognition model, so that the trained image recognition model can recognize objects that are not pre-labeled as unknown categories, thereby realizing the technical effect of accurately identifying objects that are not pre-labeled, thereby solving the technical problem of inaccurate recognition results for objects that are not pre-labeled. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a flowchart of a method for adding a pseudo label according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a process for detecting objects of unknown categories based on category enhancement according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of generating a pseudo label representing an unknown category according to an embodiment of the present invention;

[0022] Figure 4a is a schematic diagram of identifying a paraglider according to an embodiment of the present invention;

[0023] Figure 4b This is a diagram for identifying a paraglider according to the traditional method;

[0024] Figure 5a is a schematic diagram of identifying polar bears according to an embodiment of the present invention;

[0025] Figure 5b This is a diagram for identifying polar bears according to traditional methods;

[0026] Figure 6a is a schematic diagram of identifying an elephant according to an embodiment of the present invention;

[0027] Figure 6b It is a diagram for identifying elephants according to the traditional method;

[0028] Figure 7 is a schematic diagram of a pseudo-label adding device according to an embodiment of the present invention;

[0029] Figure 8 It is a structural block diagram of a computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] According to an embodiment of the present invention, an embodiment of a method for adding a pseudo-label is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0034] Target detection: In this article, it refers to finding all target objects in an image and obtaining their location information in the image.

[0035] Intersection-over-Union (IoU): In object detection and segmentation tasks, IoU is often used to measure the degree of match between the predicted bounding box and the true bounding box, as one of the evaluation indicators of model performance.

[0036] Closed set: In this article, it refers to a set of images whose categories in the test set have been labeled in the training set.

[0037] Open set: In this article, it refers to a set of pictures whose categories in the test set have not been labeled in the training set.

[0038] Instance object: is a specific object or individual object that is identified and located in an image or video.

[0039] Open set recognition: A model trained on a training set, when tested using a test set (the test set contains categories not in the training set), outputs a specific category if the input is known category data, and performs appropriate processing (identifies it as unknown) if the input is unknown category data.

[0040] Figure 1 is a flow chart of a method for adding a pseudo label according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0041] Step S102: obtaining a plurality of training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set includes: a plurality of training images with added category labels, where the category labels are used to indicate the categories of instance objects in the training images;

[0042] Step S104, fusing the instance objects in the multiple images to be fused to generate an image to be recognized, wherein the image to be recognized includes: a fusion object, which is a fusion result of the instance objects;

[0043] Step S106: Analyze the image to be identified using a preset prediction model to determine multiple detection frames of the fused object and a category label for each detection frame, wherein the preset prediction model is trained using a preset image training set, and the preset prediction model is used to represent the association between sample frames and category labels of instance objects in the training image;

[0044] Step S108: When there is a detection frame with inconsistent category labels in the fused object, a pseudo label representing an unknown category is added to the image to be identified.

[0045] In an embodiment of the present invention, a fusion object of an unknown category is constructed based on instance objects of known categories in a training image to which category labels have been added, and a pseudo label representing the unknown category is added to the image to be identified with the fusion object, thereby realizing the construction of sample images of unknown categories and the addition of pseudo labels. Furthermore, the training image to which category labels have been added and the image to be identified to which pseudo labels representing the unknown category have been added are used as training samples for training an image recognition model, so that the trained image recognition model can recognize objects that are not pre-labeled as unknown categories, thereby realizing the technical effect of accurately identifying objects that are not pre-labeled, thereby solving the technical problem of inaccurate recognition results for objects that are not pre-labeled.

[0046] In the above step S102, the preset image training set includes: multiple training images with added category labels. In the process of training the image recognition model based on the preset image training set, the image recognition model can frame the image features of the entity object in the training image to obtain a sample frame of the entity object. The category label is used to indicate the category of the instance object in the sample frame.

[0047] In the above step S104, the image to be identified can be determined based on two or more images to be fused. The following only describes the process of determining the image to be identified based on two images to be fused. The process of determining the image to be identified based on more than two images to be fused is similar to this and will not be repeated.

[0048] In the above step S104 , the multiple images to be fused that generate the image to be recognized have category labels representing different categories.

[0049] In step S104, the fused object may be obtained by mixing and combining features of multiple instance objects of known categories, for example, by segmenting multiple instance objects and then randomly selecting features from the segmented instance objects for splicing and combination.

[0050] In the above step S104, the image to be identified is generated based on multiple images to be fused. Although the image to be identified has the features of multiple images to be fused, it is different from each image to be fused. Therefore, the image to be fused can be used as a sample image representing an unknown category, and a corresponding pseudo label is not added to the sample image.

[0051] In the above step S106, the preset prediction model is pre-trained using a preset image training set. The preset prediction model is used to add pseudo labels to images of unlabeled categories (such as images to be identified), and then use the images with pseudo labels and the preset image training set to train the image recognition model ultimately used for image recognition.

[0052] In the above step S106, during the process of training the preset prediction model, multiple training images with added category labels in the preset image training set can be input into the preset prediction model, and then the preset prediction model can identify the sample frame of the instance object in the training image and establish an association relationship between the sample frame and the category label.

[0053] In the above step S106, when the preset prediction model analyzes the image to be identified, the preset prediction model can identify the detection frame of the fusion object in the image to be identified. Since the preset prediction model does not record the fusion object, in order to make the detection frame accurately cover the image features of the fusion object, it is necessary to establish multiple detection frames of the fusion object, and then predict the category label corresponding to each detection frame based on the association relationship between the pre-recorded sample frame and the category label.

[0054] In the above step S106, the image to be identified is analyzed based on the preset prediction model, and the analysis result can represent the probability that the fused object belongs to different categories. For example, the analysis result can be expressed as: the probability that the fused object belongs to category A is A%, and the probability that it belongs to category B is B%. Different categories can be determined based on different detection frames of the fused object.

[0055] It should be noted that in traditional image recognition and the process of adding pseudo labels, if the fused object has a detection frame with inconsistent category labels, the category label with the highest probability or the highest confidence is selected from multiple category labels and added as a pseudo label.

[0056] In the above step S108, after using the preset prediction model to analyze the image to be identified, the analysis results need to be further analyzed to determine whether the fusion object has a detection frame with inconsistent category labels. If the fusion object has a detection frame with inconsistent category labels, it means that the preset prediction model cannot classify the fusion object into a known category. In this case, the fusion object is treated as an unknown category, and a pseudo label representing the unknown category is added to the image to be identified where the fusion object is located.

[0057] Optionally, if the category labels of multiple detection frames of the fused object are consistent, a pseudo label consistent with the category label is added to the image to be identified where the fused object is located.

[0058] In the above step S108, the unknown category is a type of category, and the category type may include: known categories and unknown categories, wherein the known category may be a specific category such as category A, category B, or cat category, dog category, etc. Therefore, the category type may include: category A, category B, and unknown category, or cat category, dog category, and unknown category.

[0059] Optionally, the category type is represented by a category label, but the labels are divided into true labels added to the training images in advance, and pseudo labels added using a preset prediction model, wherein the true label (that is, the category label added to the training images in advance) can be used to indicate a known category, such as the true label A indicates that the training image A belongs to category A, and the true label B indicates that the training image B belongs to category B; the pseudo label (that is, added using a preset prediction model) can be used to indicate both known categories and unknown categories, such as the pseudo label A indicates that the image A to be identified belongs to category A, and the pseudo label unknown indicates that the image to be identified unknown belongs to an unknown category.

[0060] As an optional embodiment, instance objects in multiple images to be fused are fused to generate an image to be identified, including: identifying an instance region in each image to be fused, wherein the instance region includes: an instance object; segmenting the multiple images to be fused to obtain a set of image blocks, wherein the image block set includes: multiple image blocks segmented based on the instance region; randomly selecting multiple image blocks from the image block set for splicing to obtain a fusion region, wherein the fusion region includes the fusion object; and pasting the fusion region to the image to be identified to generate the image to be identified.

[0061] In the above embodiment of the present invention, in the process of fusing instance objects in multiple images to be fused, the instance objects in each image to be fused can be first identified, and then the instance objects in the multiple images to be fused can be divided into image blocks and randomly spliced ​​together, so as to obtain a fused object that includes features of multiple instance objects and has obvious differences from each instance object. Then, the fused object is pasted back into the original image to be identified, and the image to be identified including the fused object can be obtained, thereby realizing the generation of images to be identified belonging to unknown categories.

[0062] Optionally, the instance area can adopt a regular graphic. Through the instance area, a regular image area including all the features of the entity object can be determined in the image to be fused, and then segmentation and splicing are performed based on the regular image area, which can make the fusion process smoother; and then segmentation and splicing are performed according to the instance area of ​​the regular graphic to obtain a fusion area with the same regular shape.

[0063] Optionally, the instance regions in each to-be-fused image have the same shape, and the fusion region has the same shape as the instance region.

[0064] Optionally, the multiple image blocks in the image block set are segmentation results of instance regions in multiple images to be fused.

[0065] Optionally, before segmenting the instance regions in the multiple images to be fused, the instance regions in the multiple images to be fused may be normalized to make the sizes of the multiple instance regions consistent.

[0066] Optionally, the size of the fusion region is consistent with that of the instance region. Pasting the fusion region having the same size as the instance region back into the image to be identified can make the fusion object cover the instance object in the original image to be identified.

[0067] As an optional embodiment, randomly selecting multiple image blocks from an image block set for splicing to obtain a fusion area includes: determining an image block subset for each image to be fused in the image block set; and randomly selecting image blocks according to a preset number in each image block subset for splicing to obtain a fusion area.

[0068] In the above-mentioned embodiment of the present invention, the image blocks are obtained by segmenting the instance areas of multiple images to be fused. Therefore, when using an image block set to store image blocks, multiple image block subsets can be set in the image block set, and each image block subset respectively stores multiple image blocks segmented from the corresponding images to be fused. Furthermore, when splicing the fusion area, in order to make the features of multiple instance objects evenly distributed in the fusion area, the same number of image blocks can be selected from each image block set for splicing.

[0069] As an optional embodiment, using a preset prediction model to analyze the image to be identified, determining multiple detection frames of the fusion object, and the category label of each detection frame includes: using a preset prediction model to analyze the image to be identified, determining multiple candidate frames of the fusion object; determining the intersection-and-union ratio of the multiple candidate frames; selecting multiple candidate frames whose intersection-and-union ratio is greater than a preset threshold as multiple detection frames of the fusion object; and determining the category label of each detection frame.

[0070] In the above-mentioned embodiment of the present invention, the use of a preset prediction model to analyze the image to be identified requires the use of accurate image features of the fusion object. Therefore, in order to accurately obtain the image features of the fusion object, multiple candidate frames can be used to frame the fusion object in the image to be identified. Since the image features required by the preset prediction model must include the main body of the fusion object and minimize the background area, a detection frame that meets the analysis conditions can be selected from multiple candidate frames, and then the preset prediction model can be used to analyze the image features in the detection frame, and a category label can be added to each detection frame based on the analysis results.

[0071] It should be noted that the process of determining the detection frame based on the candidate frame can be achieved by calculating the intersection-over-union ratio, where a larger intersection-over-union ratio indicates a higher overlap rate of multiple candidate frames, and the candidate frame more meets the usage requirements, and multiple candidate frames with an intersection-over-union ratio greater than a preset threshold can be selected as detection frames.

[0072] It should be noted that the intersection-over-union (IoU) is based on the coverage area of ​​the two candidate frames. If the IoU is greater than a preset threshold, it means that the overlap rate of the two candidate frames is high, and then both candidate frames will be used as detection frames.

[0073] As an optional embodiment, after using a preset prediction model to analyze the image to be identified and determining multiple detection frames of the fused object and the category label of each detection frame, the method also includes: detecting whether the category labels of the multiple detection frames of the fused object are consistent; when the category labels of the detection frames are consistent, adding a pseudo label that is consistent with the category label to the fused object.

[0074] In the above-mentioned embodiment of the present invention, after using a preset prediction model to analyze the image to be identified, the analysis results of the preset prediction model can be further analyzed to detect whether the category labels of multiple detection frames of the fusion object are consistent. If the category labels of multiple detection frames are consistent, it means that the fusion object is determined to belong to the category indicated by the category label, and then a pseudo label consistent with the category label can be added to the fusion object, that is, a pseudo label consistent with the category label is added to the image to be identified including the fusion object.

[0075] As an optional embodiment, when there are detection frames with inconsistent category labels in the fused object, adding a pseudo label representing an unknown category to the image to be identified includes: when there are detection frames with inconsistent category labels in the fused object, determining the confidence of each detection frame; retaining the detection frame with the highest confidence as the predicted frame in the image to be identified; and adding a pseudo label representing the unknown category to the predicted frame.

[0076] In the above embodiment of the present invention, when there are detection frames with inconsistent category labels in the fusion object, confidence detection can be performed on the detection frames with inconsistent category labels, and the detection frame with the highest confidence can be selected as the prediction frame for adding pseudo labels, and then a pseudo label representing an unknown category is added to the prediction frame, thereby achieving the purpose of adding pseudo labels representing unknown categories to the image to be identified.

[0077] As an optional embodiment, when there are detection frames with inconsistent category labels in the fused object, after adding a pseudo label representing an unknown category to the image to be identified, the method also includes: placing the image to be identified with the added pseudo label into a preset image training set.

[0078] In the above embodiment of the present invention, a pseudo-label representing an unknown category is added, and the image to be identified with the pseudo-label is placed in a preset image training set. Then, a new label category representing the unknown category is added to the preset image training set, and then an image recognition model is trained based on the preset image training set. The trained image recognition model can recognize images belonging to unknown categories, and the recognition results can be more accurate.

[0079] Optionally, after adding a pseudo-label representing an unknown category and the image to be identified with the pseudo-label into a preset image training set, a preset prediction model for adding the pseudo-label is trained based on the preset image training set. In the process of adding the pseudo-label using the preset prediction model, the analysis results of the preset prediction model may simultaneously contain detection frames representing unknown categories and detection frames representing known categories, and then the label categories representing known categories in multiple detection frames can be compared to see whether they are consistent. If the label categories representing the known categories are inconsistent, a pseudo-label representing the unknown category is added.

[0080] The present application also provides a preferred embodiment, which provides a method for detecting objects of unknown categories based on category enhancement. For the problem of target detection in open scenarios, a large amount of data annotation of different categories is required, and the continuous iteration of training models leads to the disadvantage of high implementation cost of deep learning. The technical solution provided by the present application generates new instances (such as fused objects) by randomly sampling known instances of different categories. The generated instances have the characteristics of known categories, but are different from the known categories. Such instances are used as pseudo-labels, and their categories are defined as unknown categories. The detection method based on deep learning is used to learn unknown categories, so that the model itself has the ability to recognize unlabeled objects, which is more in line with real application scenarios, thereby enabling more real business scenarios.

[0081] This application takes advantage of the characteristics of human behavior. When people recognize unknown objects, they often use known objects to describe them. For example, the face of an elk (Sibu Xiang) is said to be like a horse, the horns are like a deer, the hooves are like a cow, and the tail is like a donkey. This application uses a simple method to generate unknown categories from known examples, thereby strengthening the model's high-dimensional classification boundaries on known categories and improving the model's recognition ability for unknown categories. This allows the model to recognize open-set objects without the need for additional unknown category annotations.

[0082] The technical solution provided by this application is based on deep learning neural networks in open scenarios, combined with the method of generating pseudo labels of unknown categories and the uncertainty-non-maximum suppression method, to detect those unlabeled objects. The specific steps of the solution are as follows:

[0083] Step 1: Preprocessing the training images of a training set (such as a preset image training set), wherein the preprocessing process includes: flip enhancement, rotation enhancement, mosaic enhancement, and generating pseudo labels of unknown categories.

[0084] Optionally, a pseudo label of an unknown category is generated to generate a pseudo label for an instance of an unknown category. The generation steps are as follows: first, take multiple instance objects of a known category, then divide each instance object of a known category into n image blocks, randomly select the image blocks of these instance objects and splice them together to generate a new instance image (such as an image to be identified). The category of this new instance image is assigned to an unknown category, and is used as a pseudo label to be mixed with the original real known category label (i.e., true label) for training. The essence is to make the generated category label (i.e., pseudo label) similar to the category label of the known category (i.e., true label), but not belonging to the known category.

[0085] Step 2: Input the image processed in step 1 (such as the image to be recognized) into the convolutional neural network model Faster-RCNN for training and learning.

[0086] Step 3: Perform uncertainty-non-maximum suppression post-processing on the prediction results of the convolutional neural network model Faster-RCNN.

[0087] Optionally, after learning the neural network in step 2, the model will output detection frames of n+1 categories, where n is the detection frame of known categories and 1 is the detection frame of unknown categories. However, through the learning of pseudo-labels, there are still a large number of detection frames among the n known categories outputted, which belong to unknown categories but are misclassified. Therefore, an uncertainty-non-maximum suppression algorithm is designed.

[0088] Optionally, a method for calculating uncertainty-non-maximum value includes the following steps:

[0089] 1. Use ordinary non-maximum suppression to obtain the post-processing results (consistent with the normal detection network).

[0090] 2. Take out all candidate boxes that are judged to be known classes (candidate boxes of unknown classes remain unchanged).

[0091] 3. Calculate the intersection-over-union ratio for these candidate boxes of known categories.

[0092] 4. Select candidate boxes with an IoU ratio greater than the preset threshold t1 as detection boxes, indicating that these detection boxes point to the same object (candidate boxes with an IoU ratio less than the preset threshold t1 are not moved, t1 is set to about 0.95, and the larger the value, the higher the possibility that these candidate boxes belong to the same object).

[0093] 5. Perform category judgment on the detection boxes pointing to the same object. If the predicted categories of these detection boxes are consistent, it means there is no uncertainty and they belong to known categories. If the categories of these detection boxes are inconsistent, it means there is uncertainty and they point to unknown categories, and the categories of these detection boxes are set to unknown categories.

[0094] Figure 2 FIG. 1 is a schematic diagram of a process for detecting an unknown category object based on category enhancement according to an embodiment of the present invention. Figure 2 As shown in the figure, the unknown category target detection algorithm based on the COCO dataset uses the first 20 categories in the COCO dataset as known category targets for training, and the last 60 categories as unknown target categories for inference. The specific steps of the scheme are as follows:

[0095] S21, obtaining training category (20 categories) images (such as a preset image training set).

[0096] S22, take 2 known category labels.

[0097] For example, two category labels are selected to represent instance objects of known categories, and the instance objects can come from the same training image or from different training images.

[0098] S23, taking out the image frame where the label is located and dividing it into n image blocks.

[0099] For example, the instance region of the instance object is framed and then the instance region is divided into n image blocks.

[0100] S24, randomly take some image blocks from n image blocks to splice into a new image, paste it on the original image, and use it as an unknown label.

[0101] For example, after slicing the two instance regions, 1 / 2 of the image blocks are taken from each of them and spliced ​​together to form a new instance object (ie, a fused object) of unknown category, which is then pasted onto the original image (ie, the original training image).

[0102] S25, input the processed image into the convolutional neural network Faster-RCNN for training and learning.

[0103] S26, determines multiple prediction boxes through convolutional neural network.

[0104] Optionally, the process of determining the prediction frame includes: using ordinary non-maximum suppression to obtain the post-processing results, taking out all candidate frames judged to be known categories, calculating the intersection-over-union ratio of these candidate frames of known categories, taking out the candidate frames with an intersection-over-union ratio greater than a threshold of 0.95 as detection frames, indicating that these detection frames point to the same object, and performing category judgment on the detection frames pointing to the same object. If the prediction results of these detection frames are consistent in category, it means that there is no uncertainty and they belong to known categories. If the categories of these detection frames are inconsistent, retain the prediction frame with the highest confidence and modify the category of the frame to an unknown category.

[0105] S27, output the final predicted box.

[0106] Figure 3 is a schematic diagram of generating pseudo-labels representing unknown categories according to an embodiment of the present application, as shown in Figure 3 the left side respectively represent training images of sheep and dogs, wherein the sheep and dogs are category labels of known categories, and then the sheep and dogs are segmented and spliced in features to obtain Figure 3 the image to be recognized shown on the right side, which includes features of sheep and dogs, but neither belongs to sheep nor dogs, and therefore pseudo-labels representing unknown categories can be added to the image to be recognized.

[0107] Figure 4a is a schematic diagram of identifying a paraglider according to an embodiment of the present application, as shown in Figure 4a in the case that there is no training image of a paraglider in a preset image training set for training an image recognition model, the image recognition model based on the pseudo-label training image provided in the present application can identify the paraglider as an unknown category (unknown).

[0108] Figure 4b is a schematic diagram of identifying a paraglider according to a conventional method, as shown in Figure 4b in the case that there is no training image of a paraglider in a preset image training set for training an image recognition model, the image recognition model trained based on the preset image training set may identify the paraglider as a boat or an Irish flag.

[0109] Figure 5a is a schematic diagram of identifying a polar bear according to an embodiment of the present application, as shown in Figure 5a in the case that there is no training image of a polar bear in a preset image training set for training an image recognition model, the image recognition model based on the pseudo-label training image provided in the present application can identify the polar bear as an unknown category (unknown).

[0110] Figure 5b is a schematic diagram of identifying a polar bear according to a conventional method, as shown in Figure 5b in the case that there is no training image of a polar bear in a preset image training set for training an image recognition model, the image recognition model trained based on the preset image training set may identify the polar bear as a cow, cold, or other known categories.

[0111] Figure 6a is a schematic diagram of identifying an elephant according to an embodiment of the present application, as shown in Figure 6a in the case that there is no training image of an elephant in a preset image training set for training an image recognition model, the image recognition model based on the pseudo-label training image provided in the present application can identify the elephant as an unknown category (unknown).

[0112] Figure 6b It is a diagram for identifying elephants according to the traditional method, such as Figure 6b As shown, when there are no training images of elephants in the preset image training set for training the image recognition model, the image recognition model trained based on the preset image training set may recognize elephants as multiple known categories such as horses, cows, dogs, etc.

[0113] The technical solution provided in this application provides a method for generating pseudo labels of unknown categories using known categories. Compared with some methods of using GAN to generate pseudo labels of unknown categories, slices of instance objects of multiple known categories are used, and randomly sampled and fused into instances of unknown categories to generate pseudo labels. The generated pseudo labels have both the characteristics of known categories and are different from known categories, thereby expanding the classification boundaries of the algorithm in high-order space and being able to have the ability to identify unknown categories.

[0114] The technical solution provided by this application can be applied to garbage detection scenarios. For example, in some scenarios, garbage classification is required. However, since the categories of garbage are endless, when unlabeled categories appear, the model will misjudge the unknown category of garbage as wet garbage or dry garbage. In actual applications, in this case, it is actually necessary to issue a prompt to the user to indicate the presence of garbage of unknown category. Using the technical solution provided by this application, when garbage of unknown category appears in the normal dry and wet garbage classification, it can accurately identify the unknown category of garbage when it appears, and then issue a warning to the user.

[0115] The technical solution provided by this application can also be applied in autonomous driving scenarios. Since the categories of targets are endless, when an object of an unknown category appears in front of the vehicle, traditional algorithms may miss detection or make false detections, leading to accidents. By using the technical solution provided by this application, unlabeled objects can be detected as unknown categories, thereby giving users an early warning and returning the control of autonomous driving to the user, avoiding risks caused by misidentification.

[0116] According to an embodiment of the present invention, an embodiment of a pseudo-label adding device is also provided. It should be noted that the pseudo-label adding device can be used to execute the pseudo-label adding method in the embodiment of the present invention, and the pseudo-label adding method in the embodiment of the present invention can be executed in the pseudo-label adding device.

[0117] Figure 7 Schematic diagram of a pseudo-label adding device according to an embodiment of the present invention. Figure 7As shown, the apparatus can comprise: an acquisition module 72, configured to acquire a plurality of training images with different class labels from a preset image training set as to-be-fused images, wherein the preset image training set comprises a plurality of training images with added class labels, and the class label is used to indicate the class of an instance object in the training image; a generation module 74, configured to fuse the instance objects in the plurality of to-be-fused images to generate a to-be-recognized image, wherein the to-be-recognized image comprises a fused object, and the fused object is the fusion result of the instance objects; an analysis module 76, configured to analyze the to-be-recognized image using a preset prediction model to determine a plurality of detection boxes of the fused object and a class label of each detection box, wherein the preset prediction model is trained using the preset image training set, and the preset prediction model is used to represent the association between the sample box and the class label of the instance object in the training image; and an adding module 78, configured to add a pseudo label representing an unknown class to the to-be-recognized image in a case where the fused object has a detection box with inconsistent class labels.

[0118] It should be noted that the acquisition module 72 in this embodiment can be used to perform step S102 in the embodiments of the present application, the generation module 74 in this embodiment can be used to perform step S104 in the embodiments of the present application, the analysis module 76 in this embodiment can be used to perform step S106 in the embodiments of the present application, and the adding module 78 in this embodiment can be used to perform step S108 in the embodiments of the present application. The above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiments.

[0119] In the embodiments of the present application, the fused object of the unknown class is constructed according to the instance object of the known class in the training image with added class labels, and the pseudo label representing the unknown class is added to the to-be-recognized image with the fused object, thereby realizing the construction of the sample image of the unknown class and the addition of the pseudo label. Furthermore, the training image with added class labels and the to-be-recognized image with added pseudo label representing the unknown class are used as training samples for training the image recognition model, so that the trained image recognition model can recognize the object of the class not pre-annotated as the unknown class, thereby realizing the technical effect of accurately recognizing the object of the class not pre-annotated, and further solving the technical problem that the object of the class not pre-annotated has an inaccurate recognition result.

[0120] As an optional embodiment, the generating module comprises: an identifying unit configured to identify an instance region in each image to be fused, wherein the instance region comprises an instance object; a segmenting unit configured to segment the plurality of images to be fused to obtain a set of image blocks, wherein the set of image blocks comprises a plurality of image blocks segmented based on the instance region; a splicing unit configured to randomly select a plurality of image blocks in the set of image blocks to splice to obtain a fusion region, wherein the fusion region comprises a fusion object; and a generating unit configured to paste the fusion region to the image to be recognized to generate the image to be recognized.

[0121] As an optional embodiment, the splicing unit comprises: a determining subunit configured to determine, in the set of image blocks, a sub-set of image blocks of each image to be fused; and a splicing subunit configured to randomly select image blocks in each sub-set of image blocks according to a preset number to splice to obtain the fusion region.

[0122] As an optional embodiment, the analyzing module comprises: an analyzing unit configured to analyze the image to be recognized using a preset prediction model to determine a plurality of candidate boxes of the fusion object; a first determining unit configured to determine an intersection over union of the plurality of candidate boxes; a selecting unit configured to select, as a plurality of detection boxes of the fusion object, the plurality of candidate boxes having an intersection over union greater than a preset threshold; and a second determining unit configured to determine a class label of each detection box.

[0123] As an optional embodiment, the device further comprises: a detecting sub-module configured to, after determining the plurality of detection boxes of the fusion object and the class label of each detection box by analyzing the image to be recognized using the preset prediction model, detect whether the class labels of the plurality of detection boxes of the fusion object are consistent; and an adding sub-module configured to, in the case that the class labels of the detection boxes are consistent, add a pseudo label consistent with the class label to the fusion object.

[0124] As an optional embodiment, the adding module comprises: a third determining unit configured to, in the case that the fusion object has detection boxes with inconsistent class labels, determine a confidence of each detection box; a fourth determining unit configured to retain a detection box with the highest confidence as a prediction box in the image to be recognized; and an adding unit configured to add a pseudo label representing an unknown class to the prediction box.

[0125] As an optional embodiment, the device further comprises: an updating sub-module configured to, after adding the pseudo label representing the unknown class to the image to be recognized in the case that the fusion object has detection boxes with inconsistent class labels, put the image to be recognized with the added pseudo label into a preset image training set.

[0126] The embodiment of the present invention can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0127] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0128] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the method for adding pseudo labels: obtaining multiple training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set includes: multiple training images with added category labels, and the category labels are used to indicate the categories of instance objects in the training images; fusing the instance objects in the multiple images to be fused to generate an image to be identified, wherein the image to be identified includes: a fusion object, and the fusion object is a fusion result of the instance objects; using a preset prediction model to analyze the image to be identified, and determining multiple detection frames of the fusion object, and the category label of each detection frame, wherein the preset prediction model is trained using the preset image training set, and the preset prediction model is used to represent the association between the sample frame and the category label of the instance object in the training image; in the case where there is a detection frame with inconsistent category labels in the fusion object, adding a pseudo label representing an unknown category to the image to be identified.

[0129] Optionally, Figure 8 1 is a block diagram of a computer terminal according to an embodiment of the present invention. Figure 8 As shown, the computer terminal 80 may include: one or more (only one is shown in the figure) processors 82 and a memory 84.

[0130] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for adding pseudo tags in the embodiments of the present invention. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the pseudo tag adding method described above. The memory can include high-speed random access memory and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory can further include a memory remotely located relative to the processor, and these remote memories can be connected to the terminal 80 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtaining multiple training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set includes: multiple training images with added category labels, and the category labels are used to indicate the categories of instance objects in the training images; fusing the instance objects in the multiple images to be fused to generate an image to be identified, wherein the image to be identified includes: a fusion object, and the fusion object is a fusion result of the instance objects; using a preset prediction model to analyze the image to be identified, and determining multiple detection frames of the fusion object and the category label of each detection frame, wherein the preset prediction model is trained using the preset image training set, and the preset prediction model is used to represent the association between the sample frame and the category label of the instance object in the training image; in the case where there is a detection frame with inconsistent category labels in the fusion object, adding a pseudo label representing an unknown category to the image to be identified.

[0132] Optionally, the processor may also execute the program code of the following steps: identifying an instance region in each image to be fused, wherein the instance region includes: an instance object; segmenting multiple images to be fused to obtain a set of image blocks, wherein the image block set includes: multiple image blocks segmented based on the instance region; randomly selecting multiple image blocks from the image block set for splicing to obtain a fusion region, wherein the fusion region includes a fusion object; pasting the fusion region to the image to be identified to generate the image to be identified.

[0133] Optionally, the processor may further execute program code of the following steps: determining, in the image block set, an image block subset of each image to be fused; and randomly selecting a preset number of image blocks in each image block subset for splicing to obtain a fusion area.

[0134] Optionally, the processor may also execute the program code of the following steps: using a preset prediction model to analyze the image to be identified to determine multiple candidate frames for the fusion object; determining the intersection-over-union (IoU) of the multiple candidate frames; selecting multiple candidate frames whose IoU is greater than a preset threshold as multiple detection frames for the fusion object; and determining the category label of each detection frame.

[0135] Optionally, the processor may also execute the program code of the following steps: after analyzing the image to be identified using a preset prediction model and determining multiple detection frames of the fused object and the category label of each detection frame, detect whether the category labels of the multiple detection frames of the fused object are consistent; if the category labels of the detection frames are consistent, add a pseudo label that is consistent with the category label to the fused object.

[0136] Optionally, the processor may also execute the program code of the following steps: determining the confidence of each detection frame when there are detection frames with inconsistent category labels in the fused object; retaining the detection frame with the highest confidence as the predicted frame in the image to be identified; and adding a pseudo label representing an unknown category to the predicted frame.

[0137] Optionally, the processor may also execute the program code of the following steps: when there is a detection frame with inconsistent category labels in the fused object, after adding a pseudo label representing an unknown category to the image to be identified, the image to be identified with the added pseudo label is placed into a preset image training set.

[0138] An embodiment of the present invention provides a pseudo-label adding scheme. In the embodiment of the present invention, a fusion object of an unknown category is constructed based on instance objects of known categories in a training image to which category labels have been added, and a pseudo-label representing the unknown category is added to the image to be identified with the fusion object, thereby realizing the construction of sample images of unknown categories and the addition of pseudo-labels. Furthermore, the training image to which category labels have been added and the image to be identified to which pseudo-labels representing the unknown category have been added are used as training samples for training an image recognition model, so that the trained image recognition model can identify objects that are not pre-labeled as unknown categories, thereby achieving the technical effect of accurately identifying objects that are not pre-labeled, thereby solving the technical problem of inaccurate recognition results for objects that are not pre-labeled.

[0139] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 8 It does not limit the structure of the above electronic device. For example, the computer terminal 80 may also include Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 8 Different configurations shown.

[0140] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a non-volatile medium. The non-volatile storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0141] The embodiment of the present invention further provides a non-volatile storage medium. Optionally, in this embodiment, the non-volatile storage medium can be used to store the program code executed by the pseudo-label adding method provided in the above embodiment.

[0142] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0143] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: obtaining multiple training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set includes: multiple training images with added category labels, and the category labels are used to indicate the categories of instance objects in the training images; fusing the instance objects in the multiple images to be fused to generate an image to be identified, wherein the image to be identified includes: a fusion object, which is a fusion result of the instance objects; using a preset prediction model to analyze the image to be identified to determine multiple detection frames of the fusion object and the category label of each detection frame, wherein the preset prediction model is trained using the preset image training set, and the preset prediction model is used to represent the association between the sample frame and the category label of the instance object in the training image; in the case where there is a detection frame with inconsistent category labels in the fusion object, adding a pseudo label representing an unknown category to the image to be identified.

[0144] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: identifying an instance region in each image to be fused, wherein the instance region includes: an instance object; segmenting multiple images to be fused to obtain a set of image blocks, wherein the image block set includes: multiple image blocks segmented based on the instance region; randomly selecting multiple image blocks from the image block set for splicing to obtain a fusion region, wherein the fusion region includes a fusion object; pasting the fusion region to the image to be identified to generate the image to be identified.

[0145] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: determining, in a set of image blocks, a subset of image blocks for each image to be fused; and in each subset of image blocks, randomly selecting a preset number of image blocks for splicing to obtain a fusion area.

[0146] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: using a preset prediction model to analyze the image to be identified to determine multiple candidate frames for the fusion object; determining the intersection-over-union ratio of the multiple candidate frames; selecting multiple candidate frames whose intersection-over-union ratio is greater than a preset threshold as multiple detection frames for the fusion object; and determining the category label of each detection frame.

[0147] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: after analyzing the image to be identified using a preset prediction model to determine multiple detection frames of the fused object and the category label of each detection frame, detecting whether the category labels of the multiple detection frames of the fused object are consistent; when the category labels of the detection frames are consistent, adding a pseudo label consistent with the category label to the fused object.

[0148] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining the confidence of each detection frame when there are detection frames with inconsistent category labels in the fused object; retaining the detection frame with the highest confidence as the predicted frame in the image to be identified; and adding a pseudo label representing an unknown category to the predicted frame.

[0149] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: in a case where there is a detection frame with inconsistent category labels in the fused object, after adding a pseudo label representing an unknown category to the image to be identified, the image to be identified with the added pseudo label is placed into a preset image training set.

[0150] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0151] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0153] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0154] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0155] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a non-volatile storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a non-volatile storage medium, including a number of instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned non-volatile storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0156] The above is only the preferred embodiment of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for adding a pseudo label, characterized in that: include: Acquire a plurality of training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set includes: a plurality of the training images to which the category labels have been added, and the category labels are used to indicate the categories of instance objects in the training images; Fusing the instance objects in the plurality of images to be fused to generate an image to be recognized, wherein the image to be recognized includes: a fusion object, which is a fusion result of the instance objects; Analyzing the image to be identified using a preset prediction model to determine multiple detection frames of the fused object and a category label for each of the detection frames, wherein the preset prediction model is trained using a preset image training set, and the preset prediction model is used to represent the association between the sample frames of the instance objects in the training image and the category labels; In the case that the fused object has a detection frame with inconsistent category labels, a pseudo label representing an unknown category is added to the image to be identified.

2. The method according to claim 1, characterized in that Fusing the instance objects in the plurality of images to be fused to generate an image to be recognized includes: Identifying an instance region in each of the images to be fused, wherein the instance region includes: the instance object; Segmenting the plurality of images to be fused to obtain a set of image blocks, wherein the set of image blocks includes: a plurality of image blocks segmented based on the instance regions; Randomly selecting a plurality of the image blocks from the image block set and splicing them together to obtain a fusion region, wherein the fusion region includes the fusion object; Paste the fused area to the image to be recognized to generate the image to be recognized.

3. The method according to claim 2, characterized in that A plurality of image blocks are randomly selected from the image block set for splicing to obtain the fusion region, including: Determining, from the image block set, an image block subset of each of the images to be fused; In each of the image block subsets, a preset number of image blocks are randomly selected for splicing to obtain the fusion area.

4. The method according to claim 1, wherein The image to be identified is analyzed using a preset prediction model to determine multiple detection frames of the fusion object, and the category label of each detection frame includes: The image to be identified is analyzed using a preset prediction model to determine multiple candidate frames of the fusion object: Determining an intersection-over-union ratio of a plurality of the candidate frames; Selecting multiple candidate frames whose intersection-over-union ratio is greater than a preset threshold as multiple detection frames of the fused object; Determine a category label for each of the detection boxes.

5. The method according to claim 1, wherein After analyzing the image to be identified using a preset prediction model to determine multiple detection frames of the fused object and a category label for each of the detection frames, the method further includes: Detecting whether the category labels of the multiple detection frames of the fused object are consistent; When the category labels of the detection frames are consistent, a pseudo label consistent with the category label is added to the fused object.

6. The method according to claim 1, characterized in that In a case where the fused object has a detection frame with inconsistent category labels, adding a pseudo label representing an unknown category to the image to be identified includes: In the case where the fused object has detection frames with inconsistent category labels, determining the confidence of each of the detection frames; Retaining the detection frame with the highest confidence as the prediction frame in the image to be identified; Add a pseudo label representing the unknown category to the predicted box.

7. The method according to claim 1, characterized in that In the case where the fused object has a detection frame with inconsistent category labels, after adding a pseudo label representing an unknown category to the image to be identified, the method further includes: The image to be identified with the pseudo label added is placed into the preset image training set.

8. A device for adding a pseudo label, characterized in that: include: an acquisition module, configured to acquire a plurality of training images with different category labels from a preset image training set as images to be fused, wherein the preset image training set includes: a plurality of training images to which the category labels have been added, and the category labels are used to indicate the categories of instance objects in the training images; a generating module, configured to fuse instance objects in a plurality of the images to be fused to generate an image to be recognized, wherein the image to be recognized includes: a fused object, which is a fusion result of the instance objects; an analysis module, configured to analyze the image to be identified using a preset prediction model to determine multiple detection frames of the fused object and a category label for each detection frame, wherein the preset prediction model is trained using a preset image training set, and the preset prediction model is used to represent the association between the sample frame of the instance object in the training image and the category label; The adding module is used to add a pseudo label representing an unknown category to the image to be identified when there is a detection frame with inconsistent category labels in the fusion object.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium is used to store a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the pseudo-label adding method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is configured to run a program stored in the processor, wherein the method for adding a pseudo label according to any one of claims 1 to 7 is executed when the program is run.

Images