Method and system for evaluating similarity between detected target and background based on few-shot learning

Abstract

The present invention belongs to the field of computer vision technology, and discloses a method and system for evaluating the similarity between a detection target and a background based on few-sample learning. The method traverses the entire image with the bounding box of the detection target as a grid unit; sets hyperparameters to determine whether the image area is retained or deleted; sets a threshold to determine whether the gridded image area contains multiple detection targets of the same type, and obtains two parts: a segmented image and a detection target; performs a similarity evaluation and changes the way the data set is loaded; queries the image and supports image classification by deleting the nearest neighbor algorithm in the inference model and modifying the forward propagation function; redefines the model verification function and applies it to calculate the similarity between the detection target and the segmented image. Aiming at the needs of different practical scenarios, the present invention provides a certain research basis for quantitatively analyzing the number of data sets required for target detection tasks.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a method and system for evaluating the similarity between a detection target and a background based on few-sample learning. Background Art

[0002] In modern computer vision, object detection, as a key applied research direction, is widely used in fields such as autonomous driving, security monitoring, and medical image analysis. The accuracy of object detection is crucial to the reliability and effectiveness of its applications. With the advancement of deep learning technology, many convolutional neural network (CNN)-based methods, such as Faster R-CNN, YOLO, and SSD, have been proposed and achieved remarkable results. These methods, trained on large-scale datasets, aim to improve the model's ability to detect and recognize objects.

[0003] However, in complex scenes, the similarity between the detected target and the background remains a major challenge in object detection. When the target and the background have a high degree of similarity, the model may have difficulty accurately distinguishing the target from the interfering background, resulting in false or missed detections. The performance of traditional object detection models often plummets when the target and background have similar features such as color and texture. Therefore, increasing the size of the dataset is necessary to improve model training and validation accuracy.

[0004] To address this issue, some researchers have proposed similarity discrimination methods, attempting to optimize the similarity between the target and the background. For example, some contrastive learning-based methods can improve detection performance by enhancing the distinguishability of features between the target and the interfering background. However, these methods still rely on a large amount of labeled data to ensure effective model learning and high generalization capabilities.

[0005] At the same time, few-shot learning (FSL) has become a research hotspot in recent years, aiming to improve model learning performance in situations with insufficient samples. Within the FSL framework, efficient feature utilization and optimization strategies can significantly improve model performance in novel object detection tasks. Li W combined FSL with a nearest neighbor classifier, which performs classification by measuring the distance between samples in the query set and those in the support set. Although FSL can determine the similarity between the query set and the support set, and has the potential to reduce sample requirements and improve learning efficiency, its application to object detection tasks remains limited, as it cannot determine the similarity between the target and background in the same image.

[0006] The above analysis reveals the following problems and shortcomings of existing technologies: While methods such as contrastive learning attempt to improve performance by enhancing the distinguishability of features between the target and background, these methods often still rely on large amounts of training data. Furthermore, existing technologies have limited effectiveness when the background information is too complex or highly similar to the target.

[0007] Although few-shot learning has been proven to be effective in evaluating the similarity between query and support sets, and has potential advantages in reducing sample dependency and enhancing learning efficiency, its practical application in the field of object detection is limited. The existing technology cannot distinguish subtle similarity differences between the target object and the background in a single image. Summary of the Invention

[0008] To overcome the problems existing in the related art, the disclosed embodiments of the present invention provide a method and system for evaluating the similarity between a detection target and its background based on few-shot learning, and specifically relate to a technology for evaluating the similarity between a detection target and its background based on few-shot learning. The purpose of the present invention is to propose an image similarity discrimination method based on a few-shot model, focusing on evaluating the similarity between the detection target and its background in the same image. Based on the few-shot learning model, the present invention can accurately quantify and distinguish the similarity between the target object and its background in the image.

[0009] The technical solution is as follows: a method for evaluating the similarity between a detection target and background based on few-shot learning, comprising:

[0010] S1, gridding the detection target in the original image, using the bounding box of the detection target as the grid unit, and traversing the entire image;

[0011] S2, in the process of traversing the entire image, determines the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target, by setting the hyperparameter Determine whether image areas should be retained or deleted;

[0012] S3, if the current detection target is the same as the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determine whether the gridded image area contains multiple detection targets of the same type, and delete the image area;

[0013] S4, after removing the image areas that do not meet the requirements, the original image is segmented, and the segmented images are numbered in sequence according to the method of traversing the original image row by row to obtain the segmented image and the detection target. The segmented image is the background image that meets the requirements;

[0014] S5: Select the few-shot learning model as the evaluation method to perform similarity evaluation. The detection target is used as the query set, and the segmented image obtained by gridding the detection target is used as the support set. The similarity between the two is calculated. In each calculation, the detection target is used as the reference and compared with the segmented image in turn.

[0015] S6, in the comparison between the detected object and the segmented image, the data set loading method is changed through the data loader;

[0016] S7, after the dataset is loaded, delete the nearest neighbor algorithm in the inference model and modify the forward propagation function to divide the dataset into a query set and a support set;

[0017] S8, redefines the model verification function and is used to calculate the similarity between the detected target and the segmented image.

[0018] In step S2, by setting the hyperparameters Determine whether the image region is retained or deleted, including: when formula (1) is satisfied, the image region is retained, otherwise, it is deleted:

[0019] ;

[0020] Where, is the area of ​​the image after gridding, is the bounding box area of ​​aircraft No. 1, is a hyperparameter, .

[0021] In step S3, it is determined whether the gridded image region contains multiple detection targets of the same type as those in step S2, including: when formula (2) is satisfied, the image region is considered to contain detection targets of the same type, and the image region is deleted:

[0022] ;

[0023] Where, The current detection target and the The intersection-over-union ratio of the detected targets, , is a hyperparameter, .

[0024] In step S4, the file name of a segmented image is: ;in, is the file name of the original image, To take the first detected target as the grid unit, In order to avoid overlap with similar detection targets, The image area in the existing segmented image that is closest to the upper left corner of the original image;

[0025] For overlap to occur, the image area contains the detected object but , so the image area is retained; after completing the preprocessing of the data set, the segmented image and the detected target are obtained.

[0026] In step S5, the similarity between the two is calculated, and the expression is:

[0027] ;

[0028] Where, For the The similarity between the segmented image and the detected target, The first Segmented images, The first query sets.

[0029] In step S6, the data loader changes the data set loading method, including:

[0030] The original data set loading method was random, and the data set loading method was changed to become orderly;

[0031] The original dataset loading support set number is fixed, but it has been updated to support dynamic loading of the number;

[0032] Among them, the number of updates to support dynamic loading includes:

[0033] Statistics will be prefixed with The number of file statistics is stored as a list. Each time the similarity between the detected target and the segmented image is calculated, the number of support sets that need to be loaded is read first to realize dynamic loading of the data set.

[0034] In step S7, the nearest neighbor algorithm in the inference model is deleted, including: deleting the AverageMeter class, the accuracy function, and the mean_confidence_interval function in the original inference model.

[0035] In step S7, the forward propagation function is modified to include:

[0036] (1) Remove the definition and call of the class ImgtoClass_Metric;

[0037] (2) Simplify the forward method, remove the loop processing of input2, and return directly after extracting features;

[0038] (3) Change the return value of the forward method to directly return q and s, which are the features of input1 and input2 respectively.

[0039] In step S8, the model validation function is redefined to include:

[0040] (a) Mode switching: The inference model is switched to evaluation mode model.eval(), ensuring that the inference model parameters are not updated during evaluation;

[0041] (b) Data processing: First, process the dimensions of query_images and support_images. If the dimension is 5 and the first dimension is 1, compress the dimension to adapt to the format of the inference model input; transfer the processed query_images and support_images to the GPU;

[0042] (c) Inference model output calculation: Assume that the inference model accepts two input query images input_var1 and support images input_var2, and returns two outputs q and s; flatten the outputs, starting from dimension 1, and retain the batch dimension;

[0043] (d) Similarity calculation: The Euclidean distance between the expanded query image and the support image is calculated using formula (3), and the result is converted through an exponential function to obtain a similarity measure;

[0044] (e) Result collection and return: Accumulate all calculated similarity results into the all_euclidean_distances list, and finally return all calculated results.

[0045] Another object of the present invention is to provide a system for evaluating the similarity between a detection target and a background based on few-shot learning, which implements the method for evaluating the similarity between a detection target and a background based on few-shot learning. The system comprises:

[0046] The image traversal module is used to grid the detection targets in the original image, using the bounding box of the detection target as the grid unit and traversing the entire image;

[0047] Hyperparameters The setting module is used to determine the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target during the traversal of the entire image, by setting the hyperparameters Determine whether image areas should be retained or deleted;

[0048] Threshold Setting module, used to check if the current detection target is the same as the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determine that the gridded image area contains multiple detection targets of the same type and delete the image area;

[0049] The segmented image and detection target acquisition module is used to segment the original image after eliminating the image areas that do not meet the requirements, and number the segmented images in sequence according to the method of traversing the original image line by line to obtain the segmented image and detection target. The segmented image is the background image that meets the requirements;

[0050] The similarity evaluation module is used to select the few-shot learning model as the evaluation method for similarity evaluation. The detection target is used as the query set, and the segmented image obtained by gridding the detection target is used as the support set to calculate the similarity between the two. In each calculation, the detection target is used as the benchmark and compared with the segmented image in turn.

[0051] Changing the dataset loading module, which is used to change the dataset loading method through the data loader when comparing the detected target with the segmented image;

[0052] The inference model modification module is used to delete the nearest neighbor algorithm in the inference model after the data set is loaded, and to modify the forward propagation function to divide the data set into the query set and the support set;

[0053] The model verification function definition module is used to redefine the model verification function, which is used to calculate the similarity between the detected target and the segmented image.

[0054] In combination with all the above technical solutions, the beneficial effects of the present invention are as follows:

[0055] Improved object segmentation and recognition: Similarity information can help improve object segmentation and recognition accuracy. Traditional object detection algorithms may face challenges when the object and background are highly similar. By leveraging similarity data, the algorithm can better distinguish between the object and background, thereby improving detection accuracy.

[0056] Background suppression and target enhancement: In image processing, the strategy of background suppression or target enhancement can be adjusted according to the similarity between the target and the background. For example, by reducing the visibility of background areas that are highly similar to the target, the target can be made more prominent, facilitating further analysis and processing.

[0057] Scene understanding and classification: Similarity data can be used for deeper scene understanding and classification. For example, in autonomous driving, analyzing the similarity between pedestrians and background images (such as trees and buildings) on the road can help the vehicle understand its surroundings more accurately and make safer decisions.

[0058] Reinforcement learning and machine training: During machine learning training, similarity information can be used as a reinforcement signal to help the model better distinguish between learning targets and non-target elements, thereby improving learning efficiency and model performance.

[0059] Video surveillance and security: In the field of video surveillance, target-background similarity information can help monitoring systems more accurately identify abnormal behaviors or unusual events, such as finding a person in a crowd whose clothing is very different from the surroundings, or identifying unusual moving objects in a natural environment.

[0060] Content generation and editing: In the field of content generation and editing, such as film post-production or image synthesis, similarity information can help artists and designers make finer adjustments when synthesizing images or videos to achieve more natural or expected effects.

[0061] Interactive art and display: In the field of interactive art and display, dynamically adjusting the visual presentation based on the similarity between the object and the background can enhance the audience's experience and sense of interaction.

[0062] Quantitative analysis of datasets: This paper lays a certain research foundation for the analysis of dataset requirements for target detection tasks, provides a new optimization path for target detection, and quantitatively analyzes the number of datasets required for target detection tasks based on the needs of different practical scenarios.

[0063] The present invention can be widely used in a variety of industries and fields, including but not limited to security monitoring, traffic management, unmanned driving, robot vision systems, medical imaging diagnosis, agricultural monitoring, etc. Especially in scenarios that require refined identification and processing of targets that are highly integrated with the background. The present invention improves the distinction between targets and backgrounds through a similarity discrimination method, which is a problem that has not been fully solved in the prior art. It changes the traditional way of loading data sets, realizes orderly and dynamic loading, and improves the flexibility and efficiency of data processing. The present invention demonstrates the flexibility of algorithm design and overcomes the dependence on specific algorithms by deleting the nearest neighbor algorithm in the inference model and modifying the forward propagation function. The model verification function is redefined, and the Euclidean distance is used to calculate the similarity, and further processing is performed to overcome the limitations of traditional verification methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure;

[0065] Figure 1 This is a flow chart of a method for evaluating the similarity between a detection target and background based on few-shot learning provided by an embodiment of the present invention;

[0066] Figure 2Schematic diagram of a method for evaluating similarity between a detection target and background based on few-shot learning provided by an embodiment of the present invention;

[0067] Figure 3 This is an analysis flow chart of the similarity experiment of the present invention;

[0068] Figure 4 It is an original image and a rendering of the present invention;

[0069] Figure 5 This is the second effect diagram of the original image of the present invention;

[0070] Figure 6 This is the third effect diagram of the original image of the present invention;

[0071] Figure 7 This is a schematic diagram of the similarity experiment process of the present invention;

[0072] Figure 8 A similarity analysis diagram of the original image 1 of the present invention;

[0073] Figure 9 Similarity analysis diagram for the original image 2 of the present invention;

[0074] Figure 10 Similarity analysis diagram of the original image 3 of the present invention;

[0075] Figure 11 Similarity analysis diagram for the original image 4 of the present invention;

[0076] Figure 12 Similarity analysis diagram for the original image 5 of the present invention;

[0077] Figure 13 Similarity analysis diagram of the original image 6 of the present invention;

[0078] Figure 14 This is a similarity analysis diagram of the original image 7 of the present invention. DETAILED DESCRIPTION

[0079] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0080] The invention can evaluate the similarity between the detection target and the background through grid processing, few-sample learning, dynamic data loading and model optimization. The innovation of this invention lies in: the invention proposes a specific grid processing method and region elimination strategy, including setting hyperparameters and This paper proposes the application of the few-shot learning model in similarity assessment for target detection, as well as the specific implementation of feature vector extraction and similarity calculation. It proposes a dynamic loading mechanism for support and query sets, as well as changes to the corresponding dataset loading methods. It also proposes specific adjustments and optimizations to the inference model, including modification details for the forward propagation function and model validation function.

[0081] Example 1, as Figure 1 As shown, the method for evaluating the similarity between a detection target and a background based on few-shot learning provided by an embodiment of the present invention includes:

[0082] S1, gridding the detection target in the original image, using the bounding box of the detection target as the grid unit, and traversing the entire image;

[0083] For example, the names of the image regions obtained after traversal are regular, with the first row and first column numbered 1, the first row and second column numbered 2, and so on. There is an existing dataset for target detection tasks, and an image is used as an example for analysis. The original image is gridded. The detection targets in the image are airplanes, and the number is 3. The entire image is traversed using the bounding box of airplane No. 1 as the grid unit, as shown in the following example: Figure 2 As shown in part (a).

[0084] S2, in the process of traversing the entire image, determines the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target, by setting the hyperparameter Determine whether image areas should be retained or deleted;

[0085] For example, during the traversal process, the area of ​​the gridded image region is , the image region at the image boundary is sometimes much smaller than the grid cell (the bounding box area of ​​aircraft No. 1 ), so the hyperparameters are set , , when the innovative formula (1) proposed by the present invention is satisfied, the image area is retained, otherwise it is deleted:

[0086] ;

[0087] Where, is the area of ​​the image after gridding, is the bounding box area of ​​aircraft No. 1, is a hyperparameter, .

[0088] The function of the formula (1) technique is to filter out image regions that are too small after gridding.

[0089] If the current detection target is The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determine whether the gridded image area contains multiple detection targets of the same type, and delete the image area;

[0090] For example, since there are multiple detection targets in the example image, when When the image is gridded for grid cells, some of the gridded image areas may contain other detection targets of the same category. When the innovative formula (2) proposed by the present invention is satisfied, the image area can be considered to contain detection targets of the same category and the image area is deleted:

[0091] ;

[0092] Where, The current detection target and The intersection-over-union ratio of the detected targets, , is a hyperparameter, . Figure 2 Examples in .

[0093] It can be understood that the function of the formula (2) is as follows: take the first detection target as the grid unit, divide the grid into multiple image regions, and calculate the intersection and union ratio of the first detection target and the second and third detection targets in each image region ( 、 ),if The minimum value is still greater than the set threshold , it means that the image area contains similar detection targets and should not be considered as background, so the image area needs to be eliminated.

[0094] S4, after removing the image areas that do not meet the requirements, the original image is segmented, and the segmented images are numbered in sequence according to the method of traversing the original image row by row to obtain the segmented image and the detection target. The segmented image is the background image that meets the requirements;

[0095] For example, the file name of a segmented image is: ;in, is the file name of the original image, To take the first detected target as the grid unit, In order to avoid overlap with similar detection targets, The image area in the existing segmented image that is closest to the upper left corner of the original image;

[0096] For overlap to occur, the image area contains the detected object but , so the image area is retained; after completing the preprocessing of the data set, the segmented image and the detected target are obtained.

[0097] It can be understood that the present invention can visualize the image area after gridding each image file, and number them according to the method of traversing the original image line by line, which is convenient for subsequent visualization experimental analysis. In addition, two states are defined: normal and overlap. Normal means that the image area does not contain the detection target, and overlap means that the image area contains the detection target and , increase the interpretability of experimental results, optimize experimental results and hyperparameter settings.

[0098] S5: Select the few-shot learning model as the evaluation method to perform similarity evaluation. The detection target is used as the query set, and the segmented image obtained by gridding the detection target is used as the support set. The similarity between the two is calculated. In each calculation, the detection target is used as the reference and compared with the segmented image in turn.

[0099] The Small Sample Learning Model is used as an evaluation method to perform similarity evaluation. The detection target in step S4 is used as the query set, and the segmented image obtained after gridding according to the detection target is used as the support set. The similarity between the two is calculated using formula (3). Each calculation requires the detection target to be used as the benchmark and then compared with the segmented image in turn.

[0100] ;

[0101] Where, For the The similarity between the segmented image and the detected target, The first Segmented images, The first query sets.

[0102] As you can understand, this paper uses a few-shot learning model as its evaluation method, leveraging its ability to extract feature vectors from images to assess image similarity and, based on this, determine the category of the query set. This model has good generalization capabilities and can accurately identify categories with limited sample data, making it an efficient and reliable image recognition solution.

[0103] S6, in the comparison between the detected object and the segmented image, the data set loading method is changed through the data loader;

[0104] The original dataset loading method was random, so we changed it to be more ordered. Input 1 is the query set (detected targets), and input 2 is the support set (segmented images). First, the number of segmented images corresponding to each detected target is stored in a list. Then, when loading the dataset, the number of support sets can be dynamically selected, and the support set data is read in an ordered manner. After loading the support set data, the corresponding detected target images are loaded.

[0105] The original dataset loading support set number is fixed, now it is updated to support dynamic loading number. Statistics will be prefixed with The number of file statistics is stored as a list. Each time the similarity between the detected target and the segmented image is calculated, the number of support sets that need to be loaded is read first to realize dynamic loading of the data set.

[0106] S7, after the dataset is loaded, delete the nearest neighbor algorithm in the inference model and modify the forward propagation function to divide the dataset into a query set and a support set;

[0107] Evaluate the similarity between the query set image (detected target) and the support set image (segmented image). For example, since the present invention is used to evaluate the similarity between the detected target and the background, and the original inference model (deep nearest neighbor neural network) uses a nearest neighbor algorithm for more accurate image classification, the nearest neighbor algorithm in the inference model is deleted and the forward propagation function is modified.

[0108] The original inference model (deep nearest neighbor neural network) is named FourLayer_64F, and the feature extraction part of the model (self.features) consists of the following layers:

[0109] 1. The first convolutional layer: from 3 input channels (e.g. RGB images) to 64 output channels, using a 3x3 convolution kernel, a stride of 1, and padding of 1 to maintain the size of the output feature map.

[0110] 2. The first normalization layer: uses the normalization type specified by norm_layer.

[0111] 3. First activation layer: LeakyReLU with negative slope of 0.2.

[0112] 4. The first pooling layer: 2x2 maximum pooling with a stride of 2, which halves the spatial size of the feature map.

[0113] 5. The next two convolutional layers and pooling layer are the same as the first one, further extracting features and reducing the spatial size of the feature map.

[0114] Among them, the deletion of the nearest neighbor algorithm in the inference model includes: deleting the AverageMeter class, accuracy function, and mean_confidence_interval function in the original inference model;

[0115] Modifying the forward propagation function includes:

[0116] (1) Removed the definition and call of the class ImgtoClass_Metric.

[0117] The ImgtoClass_Metric class is a PyTorch neural network module that defines a model for calculating the cosine similarity between two sets of image features. This model can be used to calculate the similarity between a query image (input1) and a set of support images (input2), and is typically used in few-shot learning or nearest neighbor search tasks.

[0118] The following are the technical characteristics of the ImgtoClass_Metric class:

[0119] 1. Feature normalization: Before calculating the cosine similarity, the feature vector is L2 normalized.

[0120] 2. Nearest neighbor search: For each query sample, only the k feature vectors that are most similar to the support samples are considered.

[0121] 3. GPU acceleration: If available, calculations will be performed on the GPU to speed up processing.

[0122] 4. Dynamic similarity calculation: Similarity calculation is performed dynamically based on input features rather than being statically defined.

[0123] (2) The forward method is simplified, the loop processing of input2 is removed, and only its features are extracted and returned directly.

[0124] The original code extracts features from input2, resizes its dimensions, flattens it, and uses the flattened features to calculate similarity with the query image's features. This process is used to evaluate the similarity between the query image and the support set images in tasks like few-shot learning or nearest neighbor search. In the modern code, I only perform feature extraction.

[0125] It is worth noting that input1 is the query set (detection target) and input2 is the support set (segmented image).

[0126] (3) The return value of the forward method is changed to directly return q and s, which are the characteristics of input1 and input2 respectively.

[0127] For example, the pseudo code of the forward propagation function is shown in Table 1.

[0128] Feature extraction: input1 and input2 represent the query set and support set respectively. self.features is a feature extraction network that processes the two input images and extracts their feature representations.

[0129] Extract query image features: q = self.features(input1): This line of code extracts the features of the input1 image and stores it in the variable q.

[0130] Extract support set image features: s = self.features(input2): This line of code extracts the features of the input2 image and stores it in the variable s.

[0131] Return features: The method returns two feature tensors q and s, which can be used for subsequent similarity calculation, classification or other tasks.

[0132] Table 1 Pseudocode of forward propagation function

[0133]

[0134] S8, redefine the model verification function and apply it to calculate the similarity between the detected target and the segmented image;

[0135] The similarity is measured using Euclidean distance (Formula (3));

[0136] For example, the redefinition of the model validation function includes:

[0137] (a) Mode switching: The inference model is switched to evaluation mode (model.eval()), ensuring that the inference model parameters are not updated during evaluation.

[0138] Data processing: First, process the dimensions of query_images and support_images.

[0139] Check query_images dimension: query_images.dim() == 5: First check if query_images is a five-dimensional tensor. Five-dimensional tensors are usually represented as. query_images.size(0) == 1: Then check if the first dimension is 1.

[0140] Squeeze dimension: query_images = query_images.squeeze(0): If the first dimension of query_images is 1, use the squeeze method to remove it and compress the tensor from five dimensions to four dimensions.

[0141] Send to GPU: input_var1 = query_images.cuda(): Send the processed query_images tensor to the GPU for GPU-accelerated calculations. Perform the same dimension check and processing on support_images.

[0142] If the dimension is 5 and the first dimension is 1, the dimension is compressed to fit the format of the inference model input. The processed query_images and support_images are transferred to the GPU to improve computational efficiency.

[0143] (c) Inference model output calculation: Assume that the inference model accepts two inputs input_var1 (query image) and input_var2 (support image) and returns two outputs q and s. Flatten the outputs, starting from dimension 1, to preserve the batch dimension so that the Euclidean distance can be calculated.

[0144] (d) Similarity calculation: The Euclidean distance between the expanded query image and the support images is calculated, and the result is transformed by an exponential function to obtain the similarity measure.

[0145] (e) Result collection and return: Accumulate all calculated similarity results into the all_euclidean_distances list, and finally return all calculated results.

[0146] The pseudo code of the model verification function is shown in Table 2.

[0147] Table 2 Pseudo code of model verification function

[0148]

[0149] In order to make the similarity between the detected target and the segmented image (background) more clearly distinguishable, the 14th row in Table 2 calculates the similarity between the two feature vectors through the Euclidean distance and performs further processing as shown in the following formula.

[0150] ;

[0151] Where, For the The similarity between the segmented image and the detected target, The first Segmented images, The first query sets.

[0152] It can be seen from the above embodiments that the present invention utilizes the existing target detection task dataset, performs gridding processing on the original image, and sets hyperparameters α and β to eliminate image areas that do not meet the requirements.

[0153] The present invention uses a few-shot learning model as an evaluation method to extract feature vectors and evaluate image similarity.

[0154] The present invention takes the detection target as the query set and the segmented image as the support set, calculates the similarity between the two, and changes the dataset loading method to achieve orderly and dynamic loading.

[0155] The present invention deletes the nearest neighbor algorithm in the inference model and modifies the forward propagation function.

[0156] The present invention redefines the model verification function to use Euclidean distance to calculate the similarity between the detection target and the background, and further processes the similarity to enhance the discrimination.

[0157] In Example 2, an embodiment of the present invention provides a system for evaluating the similarity between a detection target and a background based on few-shot learning, comprising:

[0158] The image traversal module is used to grid the detection targets in the original image, using the bounding box of the detection target as the grid unit and traversing the entire image;

[0159] Hyperparameters The setting module is used to determine the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target during the traversal of the entire image, by setting the hyperparameters Determine whether image areas should be retained or deleted;

[0160] Threshold Setting module, used to check if the current detection target is the same as the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determine that the gridded image area contains multiple detection targets of the same type and delete the image area;

[0161] The segmented image and detection target acquisition module is used to segment the original image after eliminating the image areas that do not meet the requirements, and number the segmented images in sequence according to the method of traversing the original image line by line to obtain the segmented image and detection target. The segmented image is the background image that meets the requirements;

[0162] The similarity evaluation module is used to select the few-shot learning model as the evaluation method for similarity evaluation. The detection target is used as the query set, and the segmented image obtained by gridding the detection target is used as the support set to calculate the similarity between the two. In each calculation, the detection target is used as the benchmark and compared with the segmented image in turn.

[0163] Changing the dataset loading module, which is used to change the dataset loading method through the data loader when comparing the detected target with the segmented image;

[0164] The inference model modification module is used to delete the nearest neighbor algorithm in the inference model after the data set is loaded, and to modify the forward propagation function to divide the data set into the query set and the support set;

[0165] The model verification function definition module is used to redefine the model verification function, which is used to calculate the similarity between the detected target and the segmented image.

[0166] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0167] To further illustrate the effects of the embodiments of the present invention, the following experiments were conducted.

[0168] right Figure 2 The image in (a) is used for experimental analysis. Only the detection target 1 is used as the grid unit for grid division and similarity analysis. α=0.5, β=0.1 are set. Since there are many segmented images, the support set selects some segmented images. In addition, the detection target 1 is analyzed for similarity with other detection targets (2, 3) and self-similarity analysis is performed on itself to provide a reference for the similarity analysis between the detection target and the background. The analysis process of the entire similarity experiment is as follows: Figure 3 The experimental results are shown in Table 3.

[0169] Table 3 Similarity analysis results (aircraft)

[0170]

[0171] As can be seen from Table 3, the method proposed in this invention can accurately evaluate the similarity between the detection target and the background. Figure 3 In (a), part of the image is filled with black background, which makes it different from the Figure 3 The similarity of (f) is also relatively low. Figure 3 Middle (b), Figure 3 In (c), Figure 3 The similarity of (f) is high, and Figure 3 (c) contains similar texture features, Figure 3 Middle (d), Figure 3 (e) in the middle is other detection targets, Figure 3 (f) shows a very high similarity, and finally Figure 3 Self-similarity analysis is performed in (f), and the above analysis results all show the same numerical rules.

[0172] Figure 4-Figure 6 To detect the three original images of horses, we set the hyperparameter α=β=0.5. Figure 4 The detection target is to divide the image into grid units and obtain Figure 7 Support set, Figure 7 The query set is Figure 4 The detection target, Figure 7 Other detection targets are Figure 5 、 Figure 6 The detection target, the similarity experiment process principle is as follows Figure 7 The experimental results are shown in Table 4.

[0173] Table 4 Similarity analysis results (horse)

[0174]

[0175] Figure 7 Middle (a), Figure 7 (b) shows the same detection target in different images. Figure 7 Middle (c), Figure 7 (d) is the sky background that is irrelevant to the detection target. Figure 7 Middle (e), Figure 7 Middle (g) is a land background similar to the detection target. In general, the experimental results are consistent with the similarity between image texture, color and other features.

[0176] In addition, the existing technology is mainly used for image classification, while the present invention can perform similarity analysis on different images and give quantitative analysis results. Figure 8 The similarity analysis of the original images is performed, and the experimental results are shown in Table 5 and Table 6.

[0177] Table 5 Image similarity analysis (irrelevant)

[0178]

[0179] Table 6 Image similarity analysis (correlation)

[0180]

[0181] From Table 5, we can see that except Figure 14 and Figure 11As shown in Table 6, images with the same detection target all have higher similarity indices (the smaller the value, the more similar they are).

[0182] The above description is only a preferred specific implementation method of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for evaluating the similarity between a detection target and a background based on few-shot learning, characterized in that: The method includes: S1, gridding the detection target in the original image, using the bounding box of the detection target as the grid unit, and traversing the entire image; S2, in the process of traversing the entire image, determines the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target, and determines whether the image region is retained or deleted by setting the hyperparameter α; S3, if the minimum value of the intersection-over-union ratio between the current detection target and the i-th detection target is greater than the set threshold β, it is determined that the gridded image area contains multiple detection targets of the same type, and the image area is deleted; S4, after eliminating the image areas that do not meet the requirements, the original image is segmented, and the segmented images are numbered in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target, and the segmented image is the background image that meets the requirements; S5, select the few-shot learning model as the evaluation method to perform similarity evaluation, take the detection target as the query set, and the segmented image obtained after the detection target is gridded as the support set, and calculate the similarity between the two; each time the detection target is used as the benchmark and compared with the segmented image in turn; S6, in comparing the detected target with the segmented image, changing the loading method of the dataset through the data loader; S7, after the data set is loaded, delete the nearest neighbor algorithm in the inference model, and modify the forward propagation function to divide the data set into a query set and a support set; S8, redefines the model verification function and is used to calculate the similarity between the detected target and the segmented image.

2. The method for evaluating the similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S2, whether the image region is retained or deleted is determined by setting a hyperparameter α, including: when formula (1) is satisfied, the image region is retained, otherwise, it is deleted: In the formula, is the area of ​​the image after gridding, S 1 is the bounding box area of ​​the detection target No. 1, α is a hyperparameter, α∈[0,1].

3. The method for evaluating the similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S3, it is determined that the gridded image region contains multiple detection targets of the same type as those in step S2, including: when formula (2) is satisfied, the image region is considered to contain detection targets of the same type, and the image region is deleted: min(IoU i )>β(2) Where, IoU i is the intersection-and-union ratio of the current detection target and the i-th detection target, i∈[2,n], n is the total number of detection targets, β is a hyperparameter, β∈[0,1].

4. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S4, the file name of a segmented image is: 000001_index_1_normal_0; wherein 000001 is the file name of the original image, index_1 is the first detected target as a grid unit, normal is no overlap with the same type of detected targets, and 0 is the image area in the existing segmented image that is closest to the upper left corner of the original image; Overlap means overlap occurs, the image area contains the detected target but min(IoU i )<β, so the image area is retained; after completing the preprocessing of the data set, the segmented image and the detected target are obtained.

5. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S5, the similarity between the two is calculated, and the expression is: In the formula, S i is the similarity between the i-th segmented image and the detected target, a i is the i-th segmentation image of the support set, and b is the i-th query set of the support set.

6. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S6, the data loading method of the data set is changed by the data loader, including: The original data set loading method was random, and the data set loading method was changed to become orderly; The original number of datasets supported for loading is fixed, but it has been updated to support dynamic loading. Among them, the number of updates to support dynamic loading includes: The number of files with prefixes 000001_index_1, 000001_index_2…000002_index_1… is counted and stored as a list. Each time the similarity between the detected target and the segmented image is calculated, the number of support sets to be loaded is read first to realize dynamic loading of the data set.

7. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S7, the nearest neighbor algorithm in the inference model is deleted, including: deleting the AverageMeter class, the accuracy function, and the mean_confidence_interval function in the original inference model.

8. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S7, the forward propagation function is modified, including: (1) Remove the definition and call of the class ImgtoClass_Metric; (2) Simplify the forward method, remove the loop processing of input2, and return directly after extracting features; (3) Change the return value of the forward method to directly return q and s, which are the features of input1 and input2, respectively.

9. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S8, the model validation function is redefined, including: (a) Mode switching: The inference model is switched to the evaluation mode model.eval(), ensuring that the inference model parameters are not updated during the evaluation; (b) Data processing: First, process the dimensions of query_images and support_images. If the dimension is 5 and the first dimension is 1, compress the dimension to adapt to the format of the inference model input; convert the processed query_images and support_images to the GPU; (c) Inference model output calculation: Assume that the inference model accepts two input query images input_var1 and support images input_var2, and returns two outputs q and s; flatten the outputs, starting from dimension 1, and retain the batch dimension; (d) Similarity calculation: The Euclidean distance between the expanded query image and the support image is calculated using formula (3), and the result is transformed by an exponential function to obtain a similarity measure; (e) Result collection and return: Accumulate all calculated similarity results into the all_euclidean_distances list, and finally return all calculated results.

10. A detection target and background similarity assessment system based on few-shot learning, characterized in that: To implement the method for evaluating the similarity between a detection target and a background based on few-sample learning as described in any one of claims 1 to 9, the system comprises: The image traversal module is used to grid the detection target in the original image, and traverse the entire image using the bounding box of the detection target as the grid unit; A hyperparameter α setting module is used to determine the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target during the traversal of the entire image, and to determine whether the image region is retained or deleted by setting the hyperparameter α; A threshold β setting module is used to determine that the gridded image area contains multiple detection targets of the same type, and delete the image area if the minimum value of the intersection-and-union ratio between the current detection target and the i-th detection target is greater than the set threshold β; The segmented image and detection target acquisition module is used to segment the original image after eliminating the image area that does not meet the requirements, and number the segmented images in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target. The segmented image is a background image that meets the requirements; The similarity evaluation module is used to select the few-shot learning model as the evaluation method for similarity evaluation, taking the detection target as the query set and the segmented image obtained after the detection target is gridded as the support set, and calculating the similarity between the two; each time the detection target is calculated, it is necessary to take the detection target as the benchmark and compare it with the segmented image in turn; Changing the dataset loading module, used to change the dataset loading method through the data loader when comparing the detected target with the segmented image; The inference model modification module is used to delete the nearest neighbor algorithm in the inference model after the data set is loaded, and to modify the forward propagation function to divide the data set into a query set and a support set; The model verification function definition module is used to redefine the model verification function, which is used to calculate the similarity between the detected target and the segmented image.

Images