Detection target and background similarity evaluation method and system based on few-sample learning

Abstract

The invention belongs to the technical field of computer vision, and discloses a detection target and background similarity evaluation method and system based on few-sample learning. The method comprises the following steps of: traversing a whole image by taking a bounding box of a detection target as a grid unit; setting a hyper-parameter # imgabs0 # to determine whether the image area is reserved or deleted; a set threshold # imgabs1 # judges that the gridded image area contains a plurality of similar detection targets of the same kind, and two parts of a segmented image and the detection targets are obtained; performing similarity evaluation, and changing a loading mode of the data set; image query is carried out by deleting a neighbor algorithm in an inference model and modifying a forward propagation function, and image classification is supported; and redefining a model verification function, and applying the model verification function to calculate the similarity between the detection target and the segmented image. Aiming at the requirements of different practical scenes, a certain research basis is provided for quantitatively analyzing the number of data sets required by a target detection task.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular to a method and system for evaluating the similarity between a detection target and a background based on few-sample learning. Background Art

[0002] In the field of modern computer vision, object detection, as an important applied research direction, is widely used in many fields such as unmanned driving, security monitoring, and medical image analysis. The accuracy of object detection is crucial to the reliability and effectiveness of its application. Currently, with the development of deep learning technology, many methods based on convolutional neural networks (CNNs), such as FasterR-CNN, YOLO, and SSD, have been proposed and achieved remarkable results. These methods are trained through large-scale data sets and aim to improve the model's ability to detect and recognize targets.

[0003] However, in complex scenes, the similarity between the detected target and the background is still a major challenge in target detection. When there is a high degree of similarity between the detected target and the background, the model may find it difficult to accurately distinguish the target from the interfering background, resulting in false detection or missed detection. Especially when the target and the background have similar features such as color and texture, the performance of traditional target detection models often drops sharply, so it is necessary to increase the size of the dataset to improve the training effect and verification accuracy of the model.

[0004] To address this problem, some scholars have proposed similarity discrimination methods, attempting to use the similarity between the target and the background as an optimization target. For example, some contrastive learning-based methods can improve detection performance by enhancing the feature distinction between the target and the interfering background. However, these methods still need to rely on a large amount of labeled data during execution to ensure effective learning and high generalization capabilities of the model.

[0005] At the same time, few-shot learning has become a research hotspot in recent years, dedicated to improving the learning effect of the model when the samples are insufficient. Under the framework of few-shot learning, the performance of the model in new target detection tasks can be significantly improved through efficient feature utilization and optimization strategies. Li W combined the nearest neighbor classifier in few-shot learning to classify by measuring the distance between the samples in the query set and the samples in the support set. Although few-shot learning can determine the similarity between the query set and the support set, and has the potential to reduce sample requirements and improve learning efficiency, its application in target detection tasks is still limited, and it cannot determine the similarity between the target and the background in the same image.

[0006] Through the above analysis, the problems and defects of the existing technology are as follows: although contrastive learning and other methods attempt to improve performance by enhancing the feature distinction between the target and the background, these methods often still rely on a large amount of training data. In addition, if the background information is too complex or has a high similarity with the target, the discrimination effect of the existing technology is limited.

[0007] Although few-shot learning has been proven to be effective in evaluating the similarity between query sets and support sets, and has potential advantages in reducing sample dependency and enhancing learning efficiency, its practical application in the field of object detection is limited because the existing technology cannot distinguish subtle similarity differences between the target object and the background in a single image. Summary of the invention

[0008] In order to overcome the problems existing in the related art, the disclosed embodiments of the present invention provide a method and system for evaluating the similarity between a detection target and a background based on few-shot learning, and specifically relate to a technology for evaluating the similarity between a detection target and a background based on few-shot learning. The purpose of the present invention is to propose an image similarity discrimination method based on a few-shot model, focusing on evaluating the similarity between a detection target and a background in the same image. The present invention can accurately quantify and distinguish the similarity between a target object and its background in an image based on a few-shot learning model.

[0009] The technical solution is as follows: A detection target and background similarity evaluation method based on few-sample learning, comprising: S1, gridding the detection target in the original image, using the bounding box of the detection target as the grid unit, and traversing the entire image; S2, in the process of traversing the entire image, determines the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target, by setting the hyperparameter Determine whether image areas should be retained or deleted; S3, if the current detection target is the same as the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determining whether the gridded image region contains multiple detection targets of the same type, and deleting the image region; S4, after eliminating the image areas that do not meet the requirements, the original image is segmented, and the segmented images are numbered in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target, and the segmented image is the background image that meets the requirements; S5, select the few-shot learning model as the evaluation method to perform similarity evaluation, take the detection target as the query set, and the segmented image obtained after the detection target is gridded as the support set, and calculate the similarity between the two; each time the detection target is calculated, it is necessary to take the detection target as the benchmark and compare it with the segmented image in turn; S6, in comparing the detected target with the segmented image, changing the loading method of the dataset through the data loader; S7, after the data set is loaded, delete the nearest neighbor algorithm in the inference model, and modify the forward propagation function to divide the data set into a query set and a support set; S8, redefines the model verification function and is used to calculate the similarity between the detected target and the segmented image.

[0010] In step S2, by setting the hyperparameters Determine whether the image region is retained or deleted, including: when formula (1) is satisfied, the image region is retained, otherwise, it is deleted: ; In the formula, is the area of ​​the image after gridding, is the bounding box area of ​​aircraft No. 1, is a hyperparameter, .

[0011] In step S3, it is determined whether the gridded image region contains multiple detection targets of the same type as those in step S2, including: when formula (2) is satisfied, the image region is considered to contain detection targets of the same type, and the image region is deleted: ; In the formula, The current detection target and The intersection-over-union ratio of the detected targets, , is a hyperparameter, .

[0012] In step S4, the file name of a segmented image is: ;in, is the file name of the original image, To take the first detected target as the grid unit, In order to avoid overlap with similar detection targets, The image area in the existing segmented image that is closest to the upper left corner of the original image; For overlap to occur, the image area contains the detected object but , so the image area is retained; after completing the preprocessing of the data set, the segmented image and the detected target are obtained.

[0013] In step S5, the similarity between the two is calculated, and the expression is: ; In the formula, For the The similarity between the segmented image and the detected target, For the support set Segmented images, For the support set A query set.

[0014] In step S6, the data loading method of the data set is changed by the data loader, including: The original data set loading method was random, and the data set loading method was changed to become orderly; The original number of datasets supported for loading is fixed, but it has been updated to support dynamic loading. Among them, the number of updates to support dynamic loading includes: Statistics will be prefixed with The number of file statistics is stored as a list. Each time the similarity between the detected target and the segmented image is calculated, the number of support sets that need to be loaded is read first to realize dynamic loading of the data set.

[0015] In step S7, the nearest neighbor algorithm in the inference model is deleted, including: deleting the AverageMeter class, the accuracy function, and the mean_confidence_interval function in the original inference model.

[0016] In step S7, the forward propagation function is modified, including: (1) Remove the definition and call of the class ImgtoClass_Metric; (2) Simplify the forward method, remove the loop processing of input2, and return directly after extracting features; (3) Change the return value of the forward method to directly return q and s, which are the features of input1 and input2, respectively.

[0017] In step S8, the model validation function is redefined, including: (a) Mode switching: The inference model is switched to the evaluation mode model.eval(), ensuring that the inference model parameters are not updated during evaluation; (b) Data processing: First, process the dimensions of query_images and support_images. If the dimension is 5 and the first dimension is 1, compress the dimension to adapt to the format of the inference model input; convert the processed query_images and support_images to the GPU; (c) Inference model output calculation: Assume that the inference model accepts two input query images input_var1 and support images input_var2, and returns two outputs q and s; flatten the outputs, starting from dimension 1, and retain the batch dimension; (d) Similarity calculation: The Euclidean distance between the expanded query image and the support image is calculated using formula (3), and the result is transformed by an exponential function to obtain a similarity measure; (e) Result collection and return: Accumulate all calculated similarity results into the all_euclidean_distances list, and finally return all calculated results.

[0018] Another object of the present invention is to provide a detection target and background similarity assessment system based on few-sample learning, the system implements the detection target and background similarity assessment method based on few-sample learning, the system comprises: The image traversal module is used to grid the detection target in the original image, and traverse the entire image using the bounding box of the detection target as the grid unit; Hyperparameters The setting module is used to determine the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target during the traversal of the entire image, by setting the hyperparameters Determine whether image areas should be retained or deleted; Threshold Setting module, used to check if the current detection target is consistent with the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determining that the gridded image region contains multiple detection targets of the same type, and deleting the image region; The segmented image and detection target acquisition module is used to segment the original image after eliminating the image area that does not meet the requirements, and number the segmented images in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target. The segmented image is a background image that meets the requirements; The similarity evaluation module is used to select the few-shot learning model as the evaluation method for similarity evaluation, taking the detection target as the query set and the segmented image obtained after the detection target is gridded as the support set, and calculating the similarity between the two; each time the detection target is calculated, it is necessary to take the detection target as the benchmark and compare it with the segmented image in turn; Changing the dataset loading module, used to change the dataset loading method through the data loader when comparing the detected target with the segmented image; The inference model modification module is used to delete the nearest neighbor algorithm in the inference model after the data set is loaded, and to modify the forward propagation function to divide the data set into a query set and a support set; The model verification function definition module is used to redefine the model verification function, which is used to calculate the similarity between the detected target and the segmented image.

[0019] In combination with all the above technical solutions, the beneficial effects of the present invention are as follows: Improved target segmentation and recognition: Similarity information can help improve the segmentation and recognition accuracy of targets. Traditional target detection algorithms may face challenges when the target has a high similarity with the background. Using similarity data, the algorithm can better distinguish between the target and the background, thereby improving the accuracy of detection.

[0020] Background suppression and target enhancement: In image processing, the strategy of background suppression or target enhancement can be adjusted according to the similarity between the target and the background. For example, by reducing the visibility of background areas that are highly similar to the target, the target can be made more prominent for further analysis and processing.

[0021] Scene understanding and classification: Similarity data can be used for deeper scene understanding and classification. For example, in autonomous driving, analyzing the similarity between pedestrians on the road and the background (such as trees and buildings) can help the vehicle understand its surroundings more accurately and make safer decisions.

[0022] Reinforcement learning and machine training: During machine learning training, similarity information can be used as a reinforcement signal to help the model better distinguish between learning targets and non-target elements, thereby improving learning efficiency and model performance.

[0023] Video surveillance and security: In the field of video surveillance, target-background similarity information can help monitoring systems more accurately identify abnormal behaviors or unusual events, such as finding people in a crowd whose clothing is different from the environment, or identifying abnormal moving objects in a natural environment.

[0024] Content generation and editing: In the field of content generation and editing, such as film post-production or image synthesis, similarity information can help artists and designers make finer adjustments when synthesizing images or videos to achieve more natural or expected results.

[0025] Interactive Art and Display: In the field of interactive art and display, dynamically adjusting the visual presentation based on the similarity between the target and the background can enhance the audience's experience and sense of interaction.

[0026] Quantitative analysis of data sets: This paper lays a certain research foundation for the analysis of data set requirements for target detection tasks, provides a new optimization path for target detection, and quantitatively analyzes the number of data sets required for target detection tasks according to the needs of different practical scenarios.

[0027] The present invention can be widely used in a variety of industries and fields, including but not limited to security monitoring, traffic management, unmanned driving, robot vision systems, medical imaging diagnosis, agricultural monitoring, etc. Especially in scenarios where refined identification and processing of targets that are highly integrated with the background are required. The present invention improves the distinction between the target and the background through a similarity discrimination method, which is a problem that has not been fully solved in the prior art. The traditional data set loading method is changed, orderly and dynamic loading is achieved, and the flexibility and efficiency of data processing are improved. The present invention demonstrates the flexibility of algorithm design and overcomes the dependence on a specific algorithm by deleting the nearest neighbor algorithm in the inference model and modifying the forward propagation function. The model verification function is redefined, the Euclidean distance is used to calculate the similarity, and further processing is performed to overcome the limitations of traditional verification methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure; Figure 1 is a flow chart of a method for evaluating similarity between a detection target and a background based on few-sample learning provided by an embodiment of the present invention; Figure 2 is a schematic diagram of a method for evaluating similarity between a detection target and a background based on few-sample learning provided in an embodiment of the present invention; Figure 3 It is an analysis flow chart of the similarity experiment of the present invention; Figure 4 It is an original image-effect diagram of the present invention; Figure 5 This is the second effect diagram of the original image of the present invention; Figure 6 The three effect diagrams of the original image of the present invention; Figure 7 This is a schematic diagram of the similarity experiment process of the present invention; Figure 8 A similarity analysis diagram of the original image 1 of the present invention; Fig. 9 A similarity analysis diagram of the original image 2 of the present invention; Fig.10 A similarity analysis diagram of the original image three of the present invention; Fig.11 A similarity analysis diagram of the original image 4 of the present invention; Fig.12 A similarity analysis diagram of the original image 5 of the present invention; Fig.13 A similarity analysis diagram of the original image six of the present invention; Fig.14 This is a similarity analysis diagram of the original image seven of the present invention. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

[0030] The invention can evaluate the similarity between the detected target and the background through grid processing, few-sample learning, dynamic data loading and model optimization. The innovation of the invention lies in that the invention proposes a specific grid processing method and region elimination strategy, including setting hyperparameters and The present invention proposes the application of few-sample learning model in target detection similarity evaluation, as well as the specific implementation of feature vector extraction and similarity calculation. By proposing a dynamic loading mechanism for support set and query set, and changing the corresponding data set loading method. By proposing specific adjustments and optimizations to the inference model, including the modification details of the forward propagation function and the model verification function.

[0031] Embodiment 1, as Figure 1 As shown, the method for evaluating the similarity between a detection target and a background based on few-sample learning provided by an embodiment of the present invention includes: S1, gridding the detection target in the original image, using the bounding box of the detection target as the grid unit, and traversing the entire image; For example, the names of the image regions obtained after traversal are regular, with the first row and first column numbered 1, the first row and second column numbered 2, and so on. There is an existing dataset for target detection tasks, and an image is used for analysis. The original image is gridded. The detection targets in the image are airplanes, and the number is 3. The entire image is traversed using the bounding box of airplane No. 1 as the grid unit, such as Figure 2 As shown in part (a).

[0032] S2, in the process of traversing the entire image, determines the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target, by setting the hyperparameter Determine whether image areas should be retained or deleted; For example, during the traversal process, the area of ​​the gridded image region is , the image region at the image boundary is sometimes much smaller than the grid cell (the bounding box area of ​​aircraft No. 1 ), so the hyperparameters are set , , when the innovative formula (1) proposed by the present invention is satisfied, the image area is retained, otherwise, it is deleted: ; In the formula, is the area of ​​the image after gridding, is the bounding box area of ​​aircraft No. 1, is a hyperparameter, .

[0033] The technical function of formula (1) is to filter out image areas that are too small after gridding.

[0034] If the current detection target is The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determining whether the gridded image region contains multiple detection targets of the same type, and deleting the image region; For example, since there are multiple detection targets in the example image, when When the image is gridded for grid cells, other detection targets of the same category may exist in some of the gridded image regions. When the innovative formula (2) proposed by the present invention is satisfied, the image region can be considered to contain detection targets of the same category, and the image region is deleted: ; In the formula, The current detection target and The intersection-over-union ratio of the detected targets, , is a hyperparameter, . Figure 2 Example in .

[0035] It can be understood that the technical function of formula (2) is: take the first detected target as the grid unit, divide the grid into multiple image regions, and calculate the intersection and union ratio of the first detected target with the second and third detected targets in each image region ( , ),if The smallest value is still greater than the set threshold , it means that the image area contains similar detection targets and should not be considered as the background, so the image area needs to be eliminated.

[0036] S4, after eliminating the image areas that do not meet the requirements, the original image is segmented, and the segmented images are numbered in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target, and the segmented image is the background image that meets the requirements; For example, the file name of a segmented image is: ;in, is the file name of the original image, To take the first detected target as the grid unit, In order to avoid overlap with similar detection targets, The image area in the existing segmented image that is closest to the upper left corner of the original image; For overlap to occur, the image area contains the detected object but , so the image area is retained; after completing the preprocessing of the data set, the segmented image and the detected target are obtained.

[0037] It can be understood that the present invention can visualize the image area after gridding each image file, and number them according to the method of traversing the original image line by line, which is convenient for subsequent visualization experimental analysis. In addition, two states, normal and overlap, are defined. Normal means that the image area does not contain the detection target, and overlap means that the image area contains the detection target and , increase the interpretability of experimental results, optimize experimental results and hyperparameter settings.

[0038] S5, select the few-shot learning model as the evaluation method to perform similarity evaluation, take the detection target as the query set, and the segmented image obtained after the detection target is gridded as the support set, and calculate the similarity between the two; each time the detection target is calculated, it is necessary to take the detection target as the benchmark and compare it with the segmented image in turn; The Small Sample Learning Model is used as an evaluation method to perform similarity evaluation. The detection target in step S4 is used as the query set, and the segmented image obtained after gridding according to the detection target is used as the support set. The similarity between the two is calculated using formula (3). Each time the detection target is calculated, it is necessary to use it as the reference and then compare it with the segmented image in turn. ; In the formula, For the The similarity between the segmented image and the detected target, For the support set Segmented images, For the support set A query set.

[0039] It can be understood that the present invention selects a few-sample learning model as an evaluation method, and uses its ability to extract feature vectors from images to evaluate the similarity of images, and judges the category of the query set accordingly. The model has good generalization ability and can achieve accurate category recognition on limited sample data. It is an efficient and reliable image recognition solution.

[0040] S6, in comparing the detected target with the segmented image, changing the loading method of the dataset through the data loader; The original data set loading method is random, so we changed the data set loading method to make it orderly. Input1 is the query set (detection target), and input2 is the support set (segmented image). First, the number of segmented images corresponding to each detection target is stored in a list, and then when loading the data set, the number of support sets can be dynamically selected, and the support set data is also orderly when reading. After loading the support set data, load the corresponding detection target image.

[0041] The original dataset loading supports a fixed number of sets, but now it is updated to support dynamic loading. Statistics will be prefixed with The number of file statistics is stored as a list. Each time the similarity between the detected target and the segmented image is calculated, the number of support sets that need to be loaded is read first to realize dynamic loading of the data set.

[0042] S7, after the data set is loaded, delete the nearest neighbor algorithm in the inference model, and modify the forward propagation function to divide the data set into a query set and a support set; Perform similarity evaluation between the query set image (detection target) and the support set image (segmented image). Exemplarily, since the present invention is used to evaluate the similarity between the detection target and the background, and the original reasoning model (deep nearest neighbor neural network) applies the nearest neighbor algorithm for more accurate image classification, the nearest neighbor algorithm in the reasoning model is deleted, and the forward propagation function is modified; The original inference model (deep nearest neighbor neural network) is named FourLayer_64F, and the feature extraction part of the model (self.features) consists of the following layers: 1. The first convolutional layer: from 3 input channels (e.g. RGB images) to 64 output channels, using a 3x3 convolution kernel, a stride of 1, and padding of 1 to maintain the size of the output feature map.

[0043] 2. The first normalization layer: uses the normalization type specified by norm_layer.

[0044] 3. First activation layer: LeakyReLU with negative slope of 0.2.

[0045] 4. The first pooling layer: 2x2 maximum pooling with a stride of 2, which halves the spatial size of the feature map.

[0046] 5. The next two convolutional layers and pooling layer are the same as the first one, further extracting features and reducing the spatial size of the feature map.

[0047] Among them, the nearest neighbor algorithm in the inference model is deleted, including: deleting the AverageMeter class, accuracy function, and mean_confidence_interval function in the original inference model; Modifying the forward propagation function includes: (1) Removed the definition and call of the class ImgtoClass_Metric.

[0048] The ImgtoClass_Metric class is a PyTorch neural network module that defines a model for calculating the cosine similarity between two sets of image features. This model can be used to calculate the similarity between a query image (input1) and a set of support images (input2), which is usually used in few-shot learning or nearest neighbor search tasks.

[0049] The following are the technical characteristics of the ImgtoClass_Metric class: 1. Feature normalization: Before calculating the cosine similarity, the feature vector is L2 normalized.

[0050] 2. Nearest neighbor search: For each query sample, only the k feature vectors that are most similar to those in the support samples are considered.

[0051] 3. GPU acceleration: If available, calculations will be performed on the GPU to speed up processing.

[0052] 4. Dynamic similarity calculation: Similarity calculation is performed dynamically based on input features rather than being statically defined.

[0053] (2) The forward method is simplified by removing the loop processing of input2 and returning it directly after extracting its features.

[0054] The original code extracts features from input2, resizes the dimensions, flattens it, and uses the flattened features to calculate the similarity with the query image features. This process is to evaluate the similarity between the query image and the support set images in few-shot learning or nearest neighbor search tasks. In the modern code, I only perform feature extraction.

[0055] It is worth noting that input1 is the query set (detection target) and input2 is the support set (segmented image).

[0056] (3) The return value of the forward method is changed to directly return q and s, which are the characteristics of input1 and input2 respectively.

[0057] For example, the following is the pseudo code of the forward propagation function, as shown in Table 1.

[0058] Feature extraction: input1 and input2 represent the query set and support set respectively. self.features is a feature extraction network that processes the two input images and extracts their feature representations.

[0059] Extract query image features: q = self.features(input1): This line of code extracts the features of input1 image and stores it in the variable q.

[0060] Extract support set image features: s = self.features(input2): This line of code extracts the features of the input2 image and stores it in the variable s.

[0061] Return features: The method returns two feature tensors q and s, which can be used for subsequent similarity calculation, classification or other tasks.

[0062] Table 1 Pseudo code of forward propagation function

[0063] S8, redefine the model verification function and apply it to calculate the similarity between the detected target and the segmented image; The similarity is measured using Euclidean distance (Formula (3)); Exemplarily, the model validation function is redefined to include: (a) Mode switching: The inference model is switched to evaluation mode (model.eval()), ensuring that the inference model parameters are not updated during evaluation.

[0064] Data processing: First, process the dimensions of query_images and support_images. Check query_images dimension: query_images.dim() == 5: First check if query_images is a five-dimensional tensor. Five-dimensional tensors are usually represented as. query_images.size(0) == 1: Then check if the first dimension is 1.

[0065] Squeeze dimensions: query_images = query_images.squeeze(0): If the first dimension of query_images is 1, use the squeeze method to remove it and compress the tensor from five dimensions to four dimensions.

[0066] Send to GPU: input_var1 = query_images.cuda(): Send the processed query_images tensor to the GPU for GPU-accelerated calculations. Perform the same dimensionality check and processing on support_images.

[0067] If the dimension is 5 and the first dimension is 1, the dimension is compressed to fit the format of the inference model input. The processed query_images and support_images are transferred to the GPU for improved computational efficiency.

[0068] (c) Inference model output calculation: Assume that the inference model accepts two inputs input_var1 (query image) and input_var2 (support image), and returns two outputs q and s. The outputs are flattened, starting from dimension 1, retaining the batch dimension, so that the Euclidean distance can be calculated.

[0069] (d) Similarity calculation: The Euclidean distance between the expanded query image and the support images is calculated, and the result is transformed by an exponential function to obtain the similarity measure.

[0070] (e) Result collection and return: Accumulate all calculated similarity results into the all_euclidean_distances list, and finally return all calculated results.

[0071] The pseudo code of the model verification function is shown in Table 2.

[0072] Table 2 Pseudo code of model verification function

[0073] In order to make the similarity between the detected target and the segmented image (background) more distinguishable, the 14th row in Table 2 calculates the similarity between the two feature vectors by Euclidean distance and performs further processing as shown in the following formula.

[0074] ; In the formula, For the The similarity between the segmented image and the detected target, For the support set Segmented images, For the support set A query set.

[0075] It can be seen from the above embodiments that the present invention utilizes the existing target detection task data set, performs gridding processing on the original image, and sets hyperparameters α and β to eliminate image areas that do not meet the requirements.

[0076] The present invention uses a few-sample learning model as an evaluation method to extract feature vectors and evaluate image similarity.

[0077] The present invention takes the detection target as the query set and the segmented image as the support set, and calculates the similarity between the two, while changing the data set loading method to achieve orderly and dynamic loading.

[0078] The present invention deletes the nearest neighbor algorithm in the inference model and modifies the forward propagation function.

[0079] The present invention redefines the model verification function to use the Euclidean distance to calculate the similarity between the detection target and the background, and further processes the similarity to enhance the discrimination.

[0080] Embodiment 2, the embodiment of the present invention provides a detection target and background similarity assessment system based on few-sample learning, comprising: The image traversal module is used to grid the detection target in the original image, and traverse the entire image using the bounding box of the detection target as the grid unit; Hyperparameters The setting module is used to determine the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target during the traversal of the entire image, by setting the hyperparameters Determine whether image areas should be retained or deleted; Threshold Setting module, used to check if the current detection target is consistent with the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determining that the gridded image region contains multiple detection targets of the same type, and deleting the image region; The segmented image and detection target acquisition module is used to segment the original image after eliminating the image area that does not meet the requirements, and number the segmented images in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target. The segmented image is a background image that meets the requirements; The similarity evaluation module is used to select the few-shot learning model as the evaluation method for similarity evaluation, taking the detection target as the query set and the segmented image obtained after the detection target is gridded as the support set, and calculating the similarity between the two; each time the detection target is calculated, it is necessary to take the detection target as the benchmark and compare it with the segmented image in turn; Changing the dataset loading module, used to change the dataset loading method through the data loader when comparing the detected target with the segmented image; The inference model modification module is used to delete the nearest neighbor algorithm in the inference model after the data set is loaded, and to modify the forward propagation function to divide the data set into a query set and a support set; The model verification function definition module is used to redefine the model verification function, which is used to calculate the similarity between the detected target and the segmented image.

[0081] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0082] To further illustrate the effects of the embodiments of the present invention, the following experiment was conducted.

[0083] right Figure 2 The image in (a) is used for experimental analysis. Only the detection target 1 is used as the grid unit for grid division and similarity analysis. α=0.5, β=0.1 are set. Since there are many segmented images, the support set selects some segmented images. In addition, the detection target 1 is analyzed for similarity with other detection targets (2, 3) and self-similarity analysis is performed on itself to provide a reference for the similarity analysis between the detection target and the background. The analysis process of the entire similarity experiment is as follows: Figure 3 The experimental results are shown in Table 3.

[0084] Table 3 Similarity analysis results (aircraft)

[0085] It can be seen from Table 3 that the method proposed in the present invention can accurately evaluate the similarity between the detection target and the background. Figure 3 (a) is partially filled with black background, which causes it to be Figure 3 The similarity of (f) is also relatively low. Figure 3 Middle (b), Figure 3 In (c), Figure 3 The similarity of (f) is high, and Figure 3 (c) contains similar texture features. Figure 3 Middle (d), Figure 3 (e) in the middle is other detection targets, Figure 3 (f) shows a very high similarity, and finally Figure 3 In (f), self-similarity analysis is performed, and the above analysis results all show the same numerical rules.

[0086] Figure 4-Figure 6 To detect the three original images of horses, we set the hyperparameter α=β=0.5. Figure 4The detection target is the grid unit to divide the image into grids, and we get Figure 7 Support set, Figure 7 The query set is Figure 4 The detection target, Figure 7 Other detection targets are Figure 5 , Figure 6 The detection target, the similarity experiment process principle is as follows Figure 7 The experimental results are shown in Table 4.

[0087] Table 4 Similarity analysis results (horse)

[0088] Figure 7 Middle (a), Figure 7 (b) shows the same detection target in different images. Figure 7 Middle (c), Figure 7 (d) is the sky background which is irrelevant to the detection target. Figure 7 Middle (e), Figure 7 (g) in the middle is a land background similar to the detection target. In general, the experimental results are consistent with the similarities between image texture, color and other features.

[0089] In addition, the existing technology is mainly used for image classification, while the present invention can perform similarity analysis on different images and give quantitative analysis results. Figure 8 The similarity analysis is performed on the original images, and the experimental results are shown in Tables 5 and 6.

[0090] Table 5 Image similarity analysis (irrelevant)

[0091] Table 6 Image similarity analysis (correlation)

[0092] From Table 5, we can see that Fig.14 and Fig.11 Except for , the other results all reflect the differences of images. From Table 6, we can see that images with the same detection target have higher similarity indexes (the smaller the value, the more similar it is).

[0093] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.

Claims

1. A detection target and background similarity assessment method based on few-shot learning, characterized in that: The method includes: S1, gridding the detection target in the original image, using the bounding box of the detection target as the grid unit, and traversing the entire image; S2, in the process of traversing the entire image, determines the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target, by setting the hyperparameter Determine whether image areas should be retained or deleted; S3, if the current detection target is the same as the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determining whether the gridded image region contains multiple detection targets of the same type, and deleting the image region; S4, after eliminating the image areas that do not meet the requirements, the original image is segmented, and the segmented images are numbered in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target, and the segmented image is the background image that meets the requirements; S5, select the few-shot learning model as the evaluation method to perform similarity evaluation, take the detection target as the query set, and the segmented image obtained after the detection target is gridded as the support set, and calculate the similarity between the two; each time the detection target is used as the benchmark and compared with the segmented image in turn; S6, in comparing the detected target with the segmented image, changing the loading method of the dataset through the data loader; S7, after the data set is loaded, delete the nearest neighbor algorithm in the inference model, and modify the forward propagation function to divide the data set into a query set and a support set; S8, redefines the model verification function and is used to calculate the similarity between the detected target and the segmented image.

2. The method for evaluating the similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S2, by setting the hyperparameters Determine whether the image region is retained or deleted, including: when formula (1) is satisfied, the image region is retained, otherwise, it is deleted: ； In the formula, is the area of ​​the image after gridding, is the bounding box area of ​​aircraft No. 1, is a hyperparameter, .

3. The method for evaluating the similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S3, it is determined whether the gridded image region contains multiple detection targets of the same type as those in step S2, including: when formula (2) is satisfied, the image region is considered to contain detection targets of the same type, and the image region is deleted: ； In the formula, The current detection target and The intersection-over-union ratio of the detected targets, , n is the total number of detected targets, is a hyperparameter, .

4. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S4, the file name of a segmented image is: ;in, is the file name of the original image, To take the first detected target as the grid unit, In order to avoid overlap with similar detection targets, The image area in the existing segmented image that is closest to the upper left corner of the original image; For overlap to occur, the image area contains the detected object but , so the image area is retained; after completing the preprocessing of the data set, the segmented image and the detected target are obtained.

5. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S5, the similarity between the two is calculated, and the expression is: ； In the formula, For the The similarity between the segmented image and the detected target, For the support set Segmented images, For the support set A query set.

6. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S6, the data loading method of the data set is changed by the data loader, including: The original data set loading method was random, and the data set loading method was changed to become orderly; The original number of datasets supported for loading is fixed, but it has been updated to support dynamic loading. Among them, the number of updates to support dynamic loading includes: Statistics will be prefixed with The number of file statistics is stored as a list. Each time the similarity between the detected target and the segmented image is calculated, the number of support sets that need to be loaded is read first to realize dynamic loading of the data set.

7. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S7, the nearest neighbor algorithm in the inference model is deleted, including: deleting the AverageMeter class, the accuracy function, and the mean_confidence_interval function in the original inference model.

8. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S7, the forward propagation function is modified, including: (1) Remove the definition and call of the class ImgtoClass_Metric; (2) Simplify the forward method, remove the loop processing of input2, and return directly after extracting features; (3) Change the return value of the forward method to directly return q and s, which are the features of input1 and input2, respectively.

9. The method for evaluating similarity between a detection target and a background based on few-sample learning according to claim 1, characterized in that: In step S8, the model validation function is redefined, including: (a) Mode switching: The inference model is switched to the evaluation mode model.eval(), ensuring that the inference model parameters are not updated during evaluation; (b) Data processing: First, process the dimensions of query_images and support_images. If the dimension is 5 and the first dimension is 1, compress the dimension to adapt to the format of the inference model input; convert the processed query_images and support_images to the GPU; (c) Inference model output calculation: Assume that the inference model accepts two input query images input_var1 and support images input_var2, and returns two outputs q and s; flatten the outputs, starting from dimension 1, and retain the batch dimension; (d) Similarity calculation: The Euclidean distance between the expanded query image and the support image is calculated using formula (3), and the result is transformed by an exponential function to obtain a similarity measure; (e) Result collection and return: Accumulate all calculated similarity results into the all_euclidean_distances list, and finally return all calculated results.

10. A detection target and background similarity assessment system based on few-shot learning, characterized in that: To implement the method for evaluating the similarity between a detection target and a background based on few-sample learning as described in any one of claims 1 to 9, the system comprises: The image traversal module is used to grid the detection target in the original image, and traverse the entire image using the bounding box of the detection target as the grid unit; Hyperparameters The setting module is used to determine the ratio of the area of ​​the gridded image region to the area of ​​the bounding box of the detection target during the traversal of the entire image, by setting the hyperparameters Determine whether image areas should be retained or deleted; Threshold Setting module, used to check if the current detection target is consistent with the The minimum value of the intersection-over-union ratio of the detected targets is greater than the set threshold , determining that the gridded image region contains multiple detection targets of the same type, and deleting the image region; The segmented image and detection target acquisition module is used to segment the original image after eliminating the image area that does not meet the requirements, and number the segmented images in sequence according to the method of traversing the original image line by line to obtain the segmented image and the detection target. The segmented image is a background image that meets the requirements; The similarity evaluation module is used to select the few-shot learning model as the evaluation method for similarity evaluation, taking the detection target as the query set and the segmented image obtained after the detection target is gridded as the support set, and calculating the similarity between the two; each time the detection target is calculated, it is necessary to take the detection target as the benchmark and compare it with the segmented image in turn; Changing the dataset loading module, used to change the dataset loading method through the data loader when comparing the detected target with the segmented image; The inference model modification module is used to delete the nearest neighbor algorithm in the inference model after the data set is loaded, and to modify the forward propagation function to divide the data set into a query set and a support set; The model verification function definition module is used to redefine the model verification function, which is used to calculate the similarity between the detected target and the segmented image.

Images