Small target detection pre-training model optimization method and system

Abstract

The invention discloses a small target detection pre-training model optimization method and system. The method comprises the following steps: performing first preprocessing on a first target data set to obtain a second target data set; obtaining a first view, a second view and a third view for the second target data set; presetting a first detection model, and training the first detection model according to the first view, the second view and the third view of the second target data set; and completing model optimization according to the trained first detection model. The migration performance limitation in the intensive small target detection field in the prior art is effectively broken through, and a new way is opened up for the development of related technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of small target detection pre-training model optimization, and in particular to a small target detection pre-training model optimization method and system. Background Art

[0002] In the field of computer vision, model training often adopts the mode of pre-training and fine-tuning. Typically, downstream tasks are initialized using pre-trained weights obtained on large datasets such as ImageNet. These methods mainly learn general visual representations suitable for various downstream tasks through image-level pre-tasks (such as instance recognition). However, recent studies have shown that in downstream tasks with dense predictions such as object detection, this image-level pre-task-based pre-training method may not be the best choice. The reason is that such pre-trained models may over-focus on the overall features of the image and ignore other detection attributes besides image classification. At the same time, when dealing with small object images, the migration effect of this method is often unsatisfactory.

[0003] In order to deal with the above problems, the present invention proposes a new self-supervised contrastive learning algorithm, which is based on a target-level feature enhancement network. This algorithm specifically designs a self-supervised pre-training model for intensive small target detection tasks, which we named TCL, namely target level contrastive learning. The TCL algorithm strengthens the model's detection ability for small targets by deeply mining target-level features. It no longer focuses only on the overall features of the image, but turns its focus to more refined target-level features. In this way, the model can more accurately capture the features of small targets, thereby improving the detection accuracy of small targets. At the same time, the TCL algorithm also adopts a self-supervised contrastive learning method to better understand and represent data by contrasting positive and negative examples, thereby improving the generalization ability of the model and its adaptability to various downstream tasks. In addition, in order to ensure the structural consistency between the pre-trained model and the target detection task, the TCL algorithm also cleverly integrates key modules in the target detection architecture, such as FPN and R-CNN heads, so that the model can be optimized for target detection tasks while maintaining versatility. In general, the TCL algorithm effectively solves the migration performance problem in dense small target detection by integrating target-level feature extraction, self-supervised contrastive learning, and key modules of target detection architecture, providing new ideas and methods for the development of computer vision. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a small target detection pre-training model optimization method and system, which can solve the problems mentioned in the background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a method for optimizing a small target detection pre-training model, comprising:

[0009] Performing a first preprocessing on the first target data set to obtain a second target data set;

[0010] Acquire a first view, a second view, and a third view for the second target data set;

[0011] Preset a first detection model, and train the first detection model according to the first view, the second view, and the third view of the second target data set;

[0012] The model optimization is completed based on the trained first detection model.

[0013] As a preferred solution of the small target detection pre-training model optimization method of the present invention, wherein: the first detection model includes:

[0014] The first detection model includes a first network and a second network;

[0015] The first view is used to train the first network;

[0016] The second view and the third view are used to train the second network.

[0017] As a preferred solution of the small target detection pre-training model optimization method of the present invention, the first detection model also includes:

[0018] The first detection model is an arbitrary model whose input is the first view, the second view and the third view, and whose output is to minimize the sample difference;

[0019] The sample difference is the difference between the output samples of the first network and the output samples of the second network.

[0020] As a preferred solution of the small target detection pre-training model optimization method of the present invention, wherein: the obtaining of the first view, the second view and the third view for the second target data set includes:

[0021] Acquire a first view for the second target data set;

[0022] Performing a first adjustment process on the first view to obtain a second view;

[0023] Perform a second adjustment process on the second view to obtain a third view.

[0024] As a preferred solution of the small target detection pre-training model optimization method of the present invention, wherein: performing a first preprocessing on the first target data set to obtain the second target data set includes:

[0025] Performing a random search on the first target data set to obtain the first target data set with a first candidate box;

[0026] Randomly select a first target data set with a first candidate box to obtain the randomly selected first target data set with the first candidate box;

[0027] Performing a first image segmentation on the randomly selected first target data set with the first candidate frame to obtain a second target data set.

[0028] As a preferred solution of the small target detection pre-training model optimization method of the present invention, wherein: the first detection model also includes a first loss function and a second loss function;

[0029] The first loss function is a loss function of the first network;

[0030] The second loss function is a loss function of the second network.

[0031] As a preferred solution of the small target detection pre-training model optimization method described in the present invention, wherein: the first candidate box includes a first constraint.

[0032] In a second aspect, the present invention provides a small target detection pre-training model optimization system, comprising:

[0033] A data set acquisition module, used for performing a first preprocessing on the first target data set to acquire a second target data set;

[0034] A view acquisition module, configured to acquire a first view, a second view, and a third view for the second target data set;

[0035] A model training module, used for presetting a first detection model, and training the first detection model according to the first view, the second view and the third view of the second target data set;

[0036] The optimization module is used to optimize the model according to the trained first detection model.

[0037] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method when executing the computer program.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method described above when executed by a processor.

[0039] Compared with the prior art, the present invention has the following beneficial effects: the present invention proposes a method and system for optimizing a pre-trained model for small target detection, performs a first pre-processing on a first target data set, obtains a second target data set; obtains a first view, a second view, and a third view for the second target data set; presets a first detection model, trains the first detection model according to the first view, the second view, and the third view of the second target data set; and optimizes the model according to the trained first detection model. This effectively breaks through the migration performance limitations of the prior art in the field of intensive small target detection, and opens up a new path for the development of related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0041] Figure 1 A method flow chart of a small target detection pre-training model optimization method and system provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of an overall model of a small target detection pre-training model optimization method and system provided by an embodiment of the present invention;

[0043] Figure 3 A flowchart of a mask generation method and system for optimizing a small target detection pre-training model provided by an embodiment of the present invention;

[0044] Figure 4A data enhancement effect diagram of a small target detection pre-training model optimization method and system provided by an embodiment of the present invention;

[0045] Figure 5 A self-supervised contrastive learning algorithm for a small target detection pre-training model optimization method and system provided in one embodiment of the present invention;

[0046] Figure 6 A structural diagram of an improved upsampling of a small target detection pre-training model optimization method and system provided by an embodiment of the present invention;

[0047] Figure 7 A scale-aware allocation flow chart of a small target detection pre-training model optimization method and system provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0049] Example 1

[0050] Reference Figure 1-Figure 7 , which is the first embodiment of the present invention, provides a method and system for optimizing a small target detection pre-training model, including:

[0051] Before describing the embodiments of the present application in detail, some related concepts are first explained for the sake of clarity.

[0052] TCL: This paper proposes a new self-supervised contrastive learning algorithm called TCL (Target Level Contrastive Learning), which is specifically optimized for dense small targets.

[0053] ImageNet dataset: ImageNet dataset is a large visual database for visual object recognition research. It is currently the world's largest image recognition database, established by computer scientists at Stanford University in the United States to simulate the human recognition system. The dataset has more than 14 million annotated images and contains more than 20,000 categories.

[0054] Candidate Box: Candidate Boxes or Proposal Boxes in target detection refer to pre-defined or generated rectangular boxes that may contain target objects in target detection tasks.

[0055] Data enhancement: Data enhancement is also called data amplification, which means making limited data produce the equivalent value of more data without substantially increasing the data.

[0056] Mask R-CNN: Mask R-CNN (Mask Region-based Convolutional Neural Network) is an object detection and image segmentation algorithm based on deep learning. Mask R-CNN is built on the basis of Faster R-CNN. It also consists of two stages: the first stage is the Region Proposal Network (RPN), which is used to generate candidate object regions; the second stage is to classify these candidate regions, regress bounding boxes, and add an additional parallel branch to predict the mask of each object.

[0057] FPN: FPN, the full name of which is Feature Pyramid Network, is a deep neural network structure used for object detection and semantic segmentation. The core idea of ​​FPN is to use cross-layer connections and top-level feature pyramids to construct feature pyramids, aiming to preserve the semantic information of high-level features and the spatial information of low-level features.

[0058] Loss function: The loss function is a function used to evaluate the difference between the predicted results of a machine learning model and the actual results. In machine learning and deep learning, the model parameters are optimized by minimizing the loss function to achieve better prediction results.

[0059] Feature encoding: Feature encoding refers to the process of converting raw data or features into a format that can be effectively used by machine learning algorithms in machine learning and data preprocessing. The purpose of feature encoding is to adapt the data to the input requirements of the model and improve the training efficiency and prediction performance of the model.

[0060] SS: Selective Search (SS) is an image segmentation technique that performs region segmentation and merging directly on the original image.

[0061] SAM: Segment Anything Model (SAM) is a powerful segmentation model that has emerged recently and is widely used in the fields of object recognition and image segmentation.

[0062] Robustness: Robustness is the ability of a system, model or algorithm to maintain stability and effectiveness in the face of various uncertainties, interferences, changes or attacks.

[0063] Felzenszwalb algorithm: Felzenszwalb algorithm is an image segmentation algorithm based on graph theory.

[0064] Huttenlocher algorithm: Huttenlocher algorithm is an image segmentation method based on graph theory.

[0065] Region Proposal: Region Proposal is a technique used to generate candidate target regions in target detection.

[0066] There are some problems in the existing related technologies. For example, in the dense small target detection task, the model often has difficulty in accurately locating and identifying small targets. This is mainly because the feature information of small targets is relatively weak in the image and is easily disturbed by background noise. In addition, traditional target detection models often rely on large-scale annotated datasets in the pre-training stage, which not only increases the difficulty of data preparation, but also limits the generalization ability of the model.

[0067] This application provides a method that can effectively solve the above-mentioned problems. Next, we will combine multiple embodiments to explain in detail how to implement the small target detection pre-training model optimization method;

[0068] Figure 1 A method flow chart of a small target detection pre-training model optimization method and system is shown, including:

[0069] S101, performing a first preprocessing on a first target data set to obtain a second target data set;

[0070] In the embodiment of the present application, performing a first preprocessing on the first target data set to obtain the second target data set includes:

[0071] Performing a random search on the first target data set to obtain the first target data set with a first candidate box;

[0072] Randomly select a first target data set with a first candidate box to obtain the randomly selected first target data set with the first candidate box;

[0073] A first image segmentation is performed on the randomly selected first target data set with the first candidate box to obtain a second target data set.

[0074] In an optional implementation, the target dataset may be a dataset of different types, such as a natural image dataset, for example, a dataset for identifying a certain animal, a dataset for identifying a certain power grid equipment, or a dataset for identifying a certain traffic sign, etc. In addition, the target dataset may also be a synthetic image dataset, for example, an image dataset generated by computer graphics technology.

[0075] In an optional implementation, the target dataset may also be a medical image dataset, such as an X-ray image dataset, an MRI image dataset, or a CT image dataset for diagnosing a specific disease. By using these medical image datasets, the model can be trained to identify abnormal areas in the image, such as tumors, fractures, or other pathological changes, thereby assisting doctors in making more accurate diagnoses.

[0076] In an optional implementation, the target dataset may also be a remote sensing image dataset, for example, a satellite image dataset used to monitor crop growth, urban expansion, forest fires, etc. By analyzing these remote sensing image datasets, the model can identify specific geographic features and changes to provide support for environmental monitoring and resource management.

[0077] In the embodiments of the present application, small target detection is targeted, so the type of data set is not restricted, and relevant technicians can select the corresponding data set for operation according to actual needs.

[0078] In an embodiment of the present application, a random search is performed on the first target data set to obtain a first target data set with a first candidate box. The first candidate box in this position can be designed according to the above-mentioned candidate box constraint conditions, or can be designed without being designed according to the above-mentioned constraint conditions, as long as the first candidate box in the second target data set finally obtained satisfies the first constraint conditions. This is for the convenience of understanding and for the purpose of designing the first candidate box at the beginning.

[0079] In the embodiment of the present application, a first target data set with a first candidate box is randomly selected to obtain the first target data set with the first candidate box after the random selection;

[0080] In an optional embodiment, a random selection algorithm is used to select K suggestions from the generated suggestion box.

[0081] A first image segmentation is performed on the randomly selected first target data set with the first candidate box to obtain a second target data set.

[0082] In an optional embodiment, the first image segmentation may be performed using an all-things segmentation algorithm;

[0083] Specifically, the size of the original input image is adjusted to a uniform 224×224 to ensure that all images are processed with the same standard. The obtained 224×224 image is designed to use the Felzenszwalb and Huttenlocher algorithm to perform fine segmentation on the resized image. In this segmentation process, three key parameters are involved: scale, sigma, and min_size. The parameter scale controls the granularity of the segmentation, that is, it determines the degree of detail of the segmented area; the parameter sigma represents the width of the Gaussian kernel, which affects the smoothness of the segmentation; and the parameter min_size specifies the minimum size of the segmented area to prevent information loss caused by over-segmentation. The system will output an image with a fourth channel region label. In order to further optimize the segmentation effect, the algorithm will evaluate the similarity of each segmented area and merge the areas with high similarity. This step can reduce redundancy and improve the accuracy of target detection.

[0084] Furthermore, the parameter scale is set to 1.0 to adapt to the diverse object sizes in the ImageNet dataset, ensuring that the algorithm can select objects of various sizes. The parameter sigma is set to 0.8, a moderate value that aims to find a balance between the smoothness of the segmentation and the preservation of details, avoiding the segmentation results being too rough or fragmented. Finally, the parameter min_size is set to 30, a relatively small value that helps to better capture and learn the small object information in the dataset, thereby improving the ability to grasp the details of the image.

[0085] Furthermore, the Selective Search (SS) algorithm can effectively improve the accuracy of image segmentation and the sensitivity of target detection, but its accuracy is often insufficient. These candidate boxes have low target coverage and may contain a lot of irrelevant background information. If these rough candidate boxes are directly used in the pre-training model and used as target-level features for comparative learning, a large number of invalid feature comparisons may be triggered, which will adversely affect the convergence of the model.

[0086] In the embodiment of the present application, the first candidate box includes a first constraint.

[0087] In an optional embodiment, the first constraint is a constraint on the first candidate box, and may include a size constraint, that is, limiting the size range of the candidate box to ensure that the candidate box can cover most areas of the target object without being too large or too small;

[0088] In an optional embodiment, a position constraint may also be included, that is, the position of the candidate box should match the actual position of the target object to improve the accuracy of detection;

[0089] In an optional embodiment, shape constraints may also be included, that is, the shape of the candidate box should be as close as possible to the actual shape of the target object to reduce background interference.

[0090] It should be noted that these constraints can improve the quality of the candidate boxes, thereby improving the performance of target detection.

[0091] In the embodiment of the present application, the first constraint is selected as the size and area constraint, and the specific first constraint can be as follows:

[0092] The aspect ratio of the first candidate box is controlled to be in the range of 1 / 3 to 3. This constraint ensures that the generated target box shape is neither too flat nor too thin, thereby maintaining the rationality of the target shape.

[0093] The ratio of the area of ​​the first candidate box to the pixel area of ​​the original image is limited to between 0.3 and 0.8. This ratio restriction prevents the target box from being too large or too small, ensuring that it can properly cover the target in the image.

[0094] In the embodiment of the present application, the minimum width and height of the first candidate frame are set to 32 pixels, and the maximum width shall not exceed 224 pixels. These restrictions are intended to ensure that the size of the target frame remains within a reasonable range, while giving priority to frames with a pixel area between 32 and 112.

[0095] Furthermore, the introduction of the Everything Segmentation algorithm is not intended to utilize its excellent image segmentation capabilities to generate label information, but rather to enhance the pre-training process through the object-level recommendations it provides.

[0096] In the embodiment of the present application, among the first candidate frame suggestion frame set b=(x, y, w, h) generated by the SS algorithm, the suggestion frames that meet the requirements of the following formula will be retained.

[0097] 1 / 3≤w / h≤3

[0098]

[0099] Among them, W and H are the width and height of the input original image respectively.

[0100] It should be noted that performing a first preprocessing on the first target data set and obtaining a second target data set significantly enhances the detection accuracy of the model for small targets. Through the preprocessing step, the area containing the target object is effectively screened out, the interference of background noise is reduced, and the model is more focused on learning target features in subsequent training. In addition, the combination of random search and random selection increases the diversity of data and improves the generalization ability of the model. Random search helps to discover more potential target areas, while random selection avoids overfitting of the model to specific areas, ensuring that the model can maintain a high detection accuracy in different scenarios. The all-things segmentation algorithm is used for image segmentation, and the quality of the candidate box is further optimized through fine segmentation and similarity evaluation. This segmentation method can capture the detailed features of the target object, effectively merge similar areas, and reduce redundant information, thereby improving the model's ability to recognize small targets.

[0101] S102, acquiring a first view, a second view, and a third view for a second target data set;

[0102] In the embodiment of the present application, acquiring the first view, the second view, and the third view for the second target data set includes:

[0103] Acquire a first view for a second target data set;

[0104] Performing a first adjustment process on the first view to obtain a second view;

[0105] A second adjustment process is performed on the second view to obtain a third view.

[0106] In an optional embodiment, the first adjustment process and the second adjustment process can be implemented by cropping, resizing, sampling, rotating, etc. the first view. These operations help to enhance the adaptability of the model to changes in the viewing angle of the target, thereby improving the robustness of the model in practical applications.

[0107] In an optional embodiment, the first adjustment process may include randomly cropping and resizing the first view to simulate different viewing angles and size changes of the target in the real world. In this way, the model can learn the feature representation of the target at different observation angles, which is crucial to improving the application effect of the model in the real world. For example, for vehicle detection in a traffic monitoring scenario, the size and angle of the vehicle in the image will change with the position and angle of the monitoring camera. By simulating these changes, the model can better adapt to the actual monitoring environment, thereby improving the accuracy and reliability of detection.

[0108] In an optional embodiment, the second adjustment process may include downsampling the obtained second view to simulate the observation effect of the target at different distances. The downsampling operation can reduce the resolution of the image, thereby simulating the blur effect of the target when observed at a longer distance. This is critical for training the model's ability to recognize distant targets, especially in monitoring systems where the target may become blurred due to the long distance. In this way, the model can learn how to extract key features from blurred images, thereby improving detection performance in practical applications.

[0109] It should be noted that obtaining three views significantly increases the model's perspective diversity and adaptability when handling target detection tasks. By simulating the changes of the target at different perspectives and distances, the model can learn a more comprehensive and robust feature representation. This multi-perspective data enhancement method helps the model maintain high detection accuracy and stability in practical applications, regardless of the angle or distance at which the target appears.

[0110] S103, presetting a first detection model, and training the first detection model according to the first view, the second view, and the third view of the second target data set;

[0111] In the embodiment of the present application, the first detection model includes:

[0112] The first detection model includes a first network and a second network;

[0113] The first view is used to train the first network;

[0114] The second view and the third view are used to train the second network.

[0115] In an embodiment of the present application, the first network is an online network, and the second network is a target network, which can be specifically designed accordingly through a feature encoder. The feature encoder includes an online encoder and a target encoder, wherein the target encoder realizes real-time momentum update through the online encoder.

[0116] In the embodiment of the present application, the first detection model also includes:

[0117] The first detection model is an arbitrary model whose input is the first view, the second view and the third view, and whose output is to minimize the sample difference;

[0118] The sample difference is the difference between the output samples of the first network and the second network.

[0119] In an optional embodiment, the first detection model can be optimized by a contrastive learning framework. Contrastive learning is an unsupervised learning method that learns the representation of data by comparing the similarities between different samples. In the target detection task, contrastive learning can help the model distinguish between the target and the background and improve the detection accuracy.

[0120] In an optional embodiment, the first detection model can also be established in other ways, such as machine learning algorithms such as neural networks, decision trees, support vector machines, etc. Through these algorithms, the model can learn complex patterns and relationships from data, thereby accurately detecting and classifying the target.

[0121] In the embodiment of the present application, the first detection model also includes a first loss function and a second loss function;

[0122] The first loss function is the loss function of the first network;

[0123] The second loss function is the loss function of the second network.

[0124] In the embodiment of the present application, a BYOL strategy is adopted to focus on the learning of positive samples. To achieve this goal, a variety of data enhancement techniques are applied to the original image to generate different enhanced versions of the same image, which are called queries and positive samples. This is intended to make these enhanced images as close as possible in the feature space, thereby promoting the model to learn the essential features of the image more deeply, rather than just memorizing the surface morphology of the data.

[0125] In the embodiment of the present application, during the construction of the image view, all first candidate boxes contained in different RGB view images are adjusted accordingly according to the cropping and scaling ratio of the image. Subsequently, random and independent data enhancement is performed on each view. In this link, BYOL's data enhancement process is used, including random horizontal inversion, random color distortion, and Gaussian blur.

[0126] It is worth noting that random cropping is not used in this technical solution, because this technology has been used when generating V2 and V3 views, so it is no longer used at this stage. After the view construction is completed, the same first candidate box in different views after data enhancement will present different scales and position coordinates. Such a design helps the pre-trained model learn target-level representations with translation invariance and scale invariance. Target-level proposal box noise: In order to further enhance the robustness of the model, this application adopts a random interference strategy to further perform target-level data enhancement on the first candidate box generated in the previous section.

[0127] In an embodiment of the present application, the specific first detection model includes a target-level feature enhancement network, a multi-scale target-level feature allocation, and a mask-level contrastive learning loss function.

[0128] Among them, the target level feature enhancement network, in the TCL pre-trained model, in order to enhance the depth and breadth of feature extraction, this application introduces the FPN (Feature Pyramid Network) structure. When building this structure, a core step is to upsample the deep feature map so that it can be effectively integrated with the shallow features.

[0129] In an embodiment of the present application, in order to enhance the stability of the detection model to the target position and scale, improve the recognition ability of targets of different sizes, and promote the convergence speed of the training process, the present application proposes a novel scale allocation strategy. This strategy aims to reasonably distribute the generated target candidate boxes to the feature maps of different scales generated by FPN. In this way, the model's learning ability of target position and scale changes can be more effectively integrated into the pre-trained network. Under the action of the feature pyramid structure, the feature map is finely divided into four scales: P2, P3, P4 and P5. These four levels of feature maps correspond to the feature information of large, medium, sub-medium and small targets, respectively, providing the model with rich multi-scale information, so as to better adapt to and identify targets of various sizes, such as Figure 2 As shown;

[0130] This application refers to the BYOL loss function design, assuming that the i-th suggestion box b in a set of object suggestion boxes in an image in the dataset i , view V1 generates b i The corresponding target-level embedding feature representation v i As a query, views V2 and V3 are generated into b i The corresponding target level v i ′、v i ″ and query are considered as positive sample pairs.

[0131] For the i-th suggestion box, its forward contrast loss function (i.e., the first loss function) is defined as shown in the following formula.

[0132]

[0133] in, <v i ,v′ i > represents the vector dot product, ||v i ||2·||v′ i ||2 is the L2 norm product of the vector, which is used to normalize the vector. The total number of first candidate boxes in a picture is K, and the total forward loss function for the picture is set to L, which is defined as the sum of the losses of all first candidate boxes in the forward calculation divided by the total number, as shown in the following formula.

[0134]

[0135] At the same time, the symmetric loss function is calculated, and the generated target-level features in the V2 and V3 views are set to and And as a query, the target level proposal of the V1 view is fed into the embedded feature representation of the target network And as a positive sample, its symmetric loss function (i.e., the second loss function) is shown in the following formula.

[0136]

[0137] Similarly, for all K object proposals in an image, the symmetric loss function is the average of the total loss, as shown in the following formula.

[0138]

[0139] Finally, the bidirectional losses are accumulated to form the loss function L all In each training iteration, the main goal is to minimize the sum of the original loss and the symmetric loss. This unique training method ensures that the pre-trained model achieves a learning balance between the two role transformations, thereby improving the stability and consistency of feature representation. The calculation method can be found in the following formula.

[0140]

[0141] It should be noted that presetting the first detection model and training the first detection model according to the first view, the second view, and the third view of the second target data set can significantly improve the model's detection accuracy for small targets. Through multi-view training, the model can learn the characteristics of the target at different viewing angles and distances, which helps the model to maintain high detection accuracy and stability in practical applications, regardless of the angle or distance at which the target appears. In addition, through the optimization of the contrast learning framework, the model can better distinguish between the target and the background, further improving the accuracy of detection.

[0142] S104, completing model optimization according to the trained first detection model.

[0143] In an optional embodiment, the model optimization may include fine-tuning the parameters of the first detection model to adapt to the needs of a specific application scenario. For example, in a traffic monitoring scenario, the optimization may be performed for specific features of a vehicle, such as the shape, color, or license plate of the vehicle, so as to improve the detection performance of the model in the scenario.

[0144] In another optional embodiment, model optimization may also include introducing additional data sets for further training. These data sets may contain more diverse scenes and targets, which helps the model learn more generalized feature representations, thereby maintaining a high detection accuracy when facing new or unknown scenes.

[0145] In summary, the present invention proposes a method for optimizing a pre-trained model for small target detection, performing a first preprocessing on a first target data set to obtain a second target data set; obtaining a first view, a second view, and a third view for the second target data set; presetting a first detection model, training the first detection model according to the first view, the second view, and the third view of the second target data set; and completing model optimization according to the trained first detection model. This method effectively breaks through the migration performance limitations of the prior art in the field of intensive small target detection, and opens up a new path for the development of related technologies.

[0146] Example 2

[0147] In a preferred embodiment, the TCL pre-training model proposed in this application significantly improves the adaptability of the upstream model to intensive small target detection tasks through two strategies. On the one hand, the model ensures the consistency of the network architecture between pre-training and target detection, so that during the migration and fine-tuning process, the network layer weights of the downstream tasks can obtain high-quality initialization settings, laying a solid foundation for subsequent training. On the other hand, in order to more accurately learn the feature representation of the target level, as well as the scale and translation invariance of the target, TCL innovatively introduced the selective search algorithm (SelectiveSearch) and the segmentation algorithm (Segment Anything). These two algorithms work together on the pre-training data to generate candidate boxes at the target level, thereby designing a unique pre-task for TCL self-supervised learning. Through these improvements, TCL has successfully built a pre-training model optimized for intensive small target detection tasks, significantly improving the detection performance.

[0148] First, in the data processing stage, TCL adopted a unique strategy to carry out fine processing of the ImageNet dataset. By combining the Selective Search algorithm with the advanced SegmentAnything algorithm, the first candidate box is generated and accurately aligned, and data enhancement technology is implemented at the same time, and finally a dataset suitable for TCL pre-training is constructed. The innovation of this data pre-processing method is that it transforms TCL's pre-task from traditional overall image processing to introducing target-level candidate box suggestions, so that TCL has detection capabilities in addition to classification.

[0149] Second, in terms of network design, TCL incorporates modules specific to downstream tasks, namely the head structure of FPN and Mask R-CNN. Its network architecture draws on BYOL's dual network design and focuses on the use of positive samples in the contrastive learning process. The feature encoder includes an online encoder and a target encoder, where the target encoder implements real-time momentum updates through the online encoder. This design enables the query sample to be feature encoded through the online encoder, while other positive samples are processed through the target encoder.

[0150] Third, the present invention introduces a multi-scale perception strategy, which is inspired by the combination of Mask R-CNN and FPN. Specifically, the first candidate boxes of different sizes generated in the first step are assigned to feature maps of different levels generated by FPN according to their sizes. By comparative learning of the same target proposals at different scales, TCL is able to learn multi-scale visual representations and the invariance of position and scale of the target hierarchy, which is crucial for migration to target detection tasks.

[0151] Fourth, the present invention has also made innovations in the design of the loss function. For each first candidate box, the feature encoding generated by the V1 view is used as a query, and the feature encoding generated by the V2 and V3 views is used as a positive sample. The design of the supervisory signal aims to minimize the difference between the query and the corresponding positive sample, thereby constructing its predecessor task. This loss function design helps to improve the accuracy and generalization ability of the model. The above is only an embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

[0152] The Selective Search algorithm (SS) is an image segmentation technique that performs region segmentation and merging directly on the original image. The algorithm ultimately generates a series of candidate boxes that may contain the target for subsequent use by the target detection algorithm. On the other hand, the Segment Anything Model (SAM) is a powerful segmentation model that has emerged recently and is widely used in the fields of target recognition and image segmentation. SAM stands out with its excellent accuracy and fast segmentation speed. Unlike other segmentation techniques, SAM allows users to provide prompts such as text, anchor boxes, points and lines to guide segmentation when processing images, and can even perform indiscriminate segmentation of the entire image without any prompts. Due to deep learning training on large-scale data sets, SAM shows higher accuracy and robustness in determining the first candidate box, providing solid support for generating the first candidate box. In terms of working principle, the operation of SAM in the image processing process is similar to that of the Selective Search algorithm.

[0153] like Figure 4 , this paper proposes a method for preparing a dataset based on target level, which is aligned with the downstream detection task at the dataset level. By performing contrastive learning under this target-level data representation through TLC (Target-Level Contrastive Learning), the model can better learn the target scale invariance and translation invariance required in the downstream detection task during pre-training. This method can improve the model's ability to detect targets of different scales and positions, thereby enhancing its performance in complex scenes.

[0154] like Figure 5 In response to the difficulties faced by current general pre-trained models when dealing with dense downstream small target detection tasks, this application proposes a novel solution: a self-supervised contrastive learning algorithm based on a target-level feature enhancement network. This algorithm effectively solves the above problems from two dimensions. On the one hand, this application integrates the task architecture of target detection with the pre-training architecture, thereby optimizing the self-supervised model's ability to initialize the parameters required for the target detection layer. On the other hand, by introducing target-level suggestions, this application helps the pre-trained model learn and master the feature representation of the target level. These improvements significantly improve the fit between the pre-trained model and downstream detection tasks, and achieve smoother task migration.

[0155] like Figure 6 In the TCL pre-training model, in order to enhance the depth and breadth of feature extraction, this application introduces the FPN (Feature Pyramid Network) structure. When building this structure, a core step is to upsample the deep feature map so that it can be effectively fused with the shallow features.

[0156] like Figure 7 In order to enhance the stability of the detection model to the target position and scale, improve the recognition ability of targets of different sizes, and promote the convergence speed of the training process, the present invention proposes a novel scale allocation strategy. This strategy aims to reasonably allocate the generated first candidate boxes to the feature maps of different scales generated by FPN. In this way, the model's learning ability of target position and scale changes can be more effectively integrated into the pre-trained network.

[0157] In each training iteration, the main goal is to minimize the sum of the original loss and the symmetric loss. This unique training method ensures that the pre-trained model achieves a learning balance between the two role transformations, thereby improving the stability and consistency of feature representation.

[0158] Example 3

[0159] This embodiment also provides a small target detection pre-training model optimization system, including:

[0160] A data set acquisition module, used for performing a first preprocessing on the first target data set to acquire a second target data set;

[0161] A view acquisition module, used to acquire a first view, a second view and a third view for a second target data set;

[0162] A model training module, used for presetting a first detection model, and training the first detection model according to a first view, a second view, and a third view of a second target data set;

[0163] The optimization module is used to optimize the model according to the trained first detection model.

[0164] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0165] This embodiment also provides a computer device, which may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a small target detection pre-training model optimization method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0166] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0167] Performing a first preprocessing on the first target data set to obtain a second target data set;

[0168] Acquire a first view, a second view, and a third view for a second target data set;

[0169] Preset a first detection model, and train the first detection model according to the first view, the second view, and the third view of the second target data set;

[0170] The model optimization is completed based on the trained first detection model.

[0171] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0172] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0173] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0177] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A small target detection pre-training model optimization method, characterized in that: include: Performing a first preprocessing on the first target data set to obtain a second target data set; Acquire a first view, a second view, and a third view for the second target data set; Preset a first detection model, and train the first detection model according to the first view, the second view, and the third view of the second target data set; The model optimization is completed based on the trained first detection model.

2. The small target detection pre-training model optimization method according to claim 1, characterized in that: The first detection model comprises: The first detection model includes a first network and a second network; The first view is used to train the first network; The second view and the third view are used to train the second network.

3. The small target detection pre-training model optimization method according to claim 2, characterized in that: The first detection model also includes: The first detection model is an arbitrary model whose input is the first view, the second view and the third view, and whose output is to minimize the sample difference; The sample difference is the difference between the output samples of the first network and the output samples of the second network.

4. The small target detection pre-training model optimization method according to claim 3, characterized in that: The acquiring of the first view, the second view and the third view for the second target data set comprises: Acquire a first view for the second target data set; Performing a first adjustment process on the first view to obtain a second view; Perform a second adjustment process on the second view to obtain a third view.

5. The small target detection pre-training model optimization method according to claim 4, characterized in that: The performing a first preprocessing on the first target data set to obtain the second target data set includes: Performing a random search on the first target data set to obtain the first target data set with a first candidate box; Randomly select a first target data set with a first candidate box to obtain the randomly selected first target data set with the first candidate box; Performing a first image segmentation on the randomly selected first target data set with the first candidate frame to obtain a second target data set.

6. The small target detection pre-training model optimization method according to claim 5, characterized in that: The first detection model also includes a first loss function and a second loss function; The first loss function is a loss function of the first network; The second loss function is a loss function of the second network.

7. The small target detection pre-training model optimization method according to claim 6, characterized in that: The first candidate box includes a first constraint.

8. A small target detection pre-training model optimization system, characterized in that: include: A data set acquisition module, used for performing a first preprocessing on the first target data set to acquire a second target data set; A view acquisition module, configured to acquire a first view, a second view, and a third view for the second target data set; A model training module, used for presetting a first detection model, and training the first detection model according to the first view, the second view and the third view of the second target data set; The optimization module is used to optimize the model according to the trained first detection model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images