Cross-scene target identification data generation method based on large segmentation model

Abstract

The invention relates to a cross-scene target recognition training data generation method based on a segmentation large model, and the method comprises the following steps: collecting an original data sample, marking a small batch, training a target recognition model, carrying out the target detection and recognition of all data, feeding a target image region into the segmentation large model based on a recognition result, obtaining a target mask, and carrying out the recognition of the target image region. The method comprises the following steps: carrying out random scale, rotation, noise and brightness change operations on a target, fusing the changed target instance with a background, carrying out data batch augmentation based on the target instance and real background image data, and automatically producing label information based on fused target position and category priori. In conclusion, the method has the advantages that the data processing efficiency is improved, the diversity of target forms, backgrounds and target relative backgrounds in the database is improved, and the breadth of sample feature distribution in the database is improved, so that the generalization ability of a deep learning training model is improved.

Description

Technical Field

[0001] The present invention belongs to the field of airborne photoelectric detection and relates to a method for generating cross-scene target recognition data based on a large segmentation model. Background Art

[0002] When performing deep learning-based target recognition, airborne reconnaissance optoelectronic systems need to use trained target recognition models, and labeled high-quality training data sets are the basis for obtaining good target recognition models. However, due to factors such as weather and cost, the amount of data collected for many high-value targets is very limited, such as single background, single posture, single scale, etc. These factors make the model training process prone to overfitting, which poses a challenge to the training and application of high-quality target recognition models. In order to solve these problems, using large models with good generalization performance to perform highly automated data augmentation is an effective method, which can enhance the generalization performance of the model by enriching the target background, scale, posture, etc.

[0003] However, there are many problems in the specific use process. For example, in the early days, when data augmentation was performed on limited training samples, at the image level, only simple processing such as rotation, brightness change, and noise addition was often performed on the image, and the effect was not good. The richness of the amplified data was not high, and the data distribution was still limited to a small range of space. At the same time, at the target level, although methods such as cropping and pasting can be used for processing, there is a problem of over-reliance on manual work for unlabeled data. There are also problems with labeled data. Labeled data usually needs to be extracted according to a rectangular frame, which not only has low accuracy, but also contains a lot of redundant information. The expansion method based on cropping and pasting of the target area block will lead to a large degree of separation between the target area and the background in the synthetic image. The feature difference is very obvious, which is not conducive to model training. Therefore, there is an urgent need for a cross-scene target recognition data generation method based on a large segmentation model to solve the above problems. Summary of the invention

[0004] In view of this, the present invention proposes a cross-scene target recognition data generation method based on a segmentation large model, which belongs to the field of airborne photoelectric detection and solves the existing technical problems of relying on manual labor, low precision, containing redundant information, obvious feature differences and inconvenient model training.

[0005] In order to achieve the above technical objectives, the specific technical solutions adopted by the present invention are:

[0006] A method for generating cross-scene target recognition training data based on a segmentation large model comprises the following steps:

[0007] S1. Collection of original data samples;

[0008] S2, small batch labeling and target recognition model training;

[0009] S3, perform target detection and recognition on all data;

[0010] S4, based on the recognition result, the target image area is sent to the segmentation model;

[0011] S5, obtain target mask, target position, and target instance;

[0012] S6, performing random scale, rotation, noise, and brightness changes on the target;

[0013] S7, fusing the changed target instance with the background, and performing batch data augmentation based on the target instance and the real background image data;

[0014] S8. Automatically generate label information based on the fusion of target location and category priors to construct target recognition training and verification datasets.

[0015] Furthermore, in step S2, a small batch of data containing the target of interest is obtained, and after manually annotating the rectangular frame, it is used as the recognition base data, and the base data is used as training data to train the small target identifier C using YOLOv8.

[0016] Furthermore, in step S3, a small target identifier is trained to perform target detection and recognition on all data samples, and target area and category information is automatically obtained.

[0017] Furthermore, in step S4, the target area and category information in the image are used to screen the target type in the data sample, and the pre-trained segmentation model is loaded to achieve segmentation and separation of the target instance and the local background.

[0018] Furthermore, in step S5, the pre-trained segmentation model is used, the center point of the rectangular box in the source data is marked as a prompt, combined with the target area map, and input into the segmentation model to achieve segmentation and separation of the target instance and the local background, and obtain the target mask, target position, and target instance.

[0019] Furthermore, in step S6, a target transformation range is set, and data amplification of random scale, rotation, noise addition, and brightness change is performed on the acquired target instances to obtain an amplified target instance set.

[0020] Furthermore, in step S7, the background data set to be amplified is obtained, and the amplified target instance set is randomly adopted and randomly pasted on the background data set to be amplified according to the experience of the set target quantity and area range.

[0021] Furthermore, in step S8, automatic label generation of augmented data is achieved based on known target pasting position and target category information.

[0022] Furthermore, in step S8, the original data samples are mixed with the generated data samples to form a sample database.

[0023] By adopting the above technical solution, the present invention can also bring the following beneficial effects:

[0024] 1. The present invention mentions a method for generating cross-scene target recognition training data based on a large segmentation model. It provides a semi-automatic labeling method for unlabeled sample data, that is, by labeling small batches of data and training small target identifiers, thereby realizing labeling processing of unlabeled sample data, improving data processing efficiency, and having the advantage of high processing effect.

[0025] 2. The present invention mentions a method for generating cross-scene target recognition training data based on a segmentation large model. By separating the target instance from the background and randomly transforming it, the diversity of target morphology, background, and target relative to the background in the database is greatly improved, and the breadth of sample feature distribution in the database is enhanced, thereby facilitating the use of the database.

[0026] 3. The present invention mentions a method for generating cross-scene target recognition training data based on a large segmentation model. By randomly setting the target shape and quantity, the constructed database can be expanded to apply to single-target, multi-target detection, recognition and segmentation tasks, and labels can be automatically generated, which is very convenient to use and effectively improves the generalization ability of the model.

[0027] 4. The present invention mentions a method for generating cross-scene target recognition training data based on a large segmentation model. By utilizing the good target segmentation ability of the large segmentation model in different scenes and cooperating with a basic recognizer trained with small batch base data, a highly automated cross-scene data amplification can be performed on labeled and unlabeled images to generate a target recognition training data set with large data distribution differences and high diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 The present invention provides a flow chart of a method for generating cross-scene target recognition data based on a segmentation large model;

[0030] Figure 2This embodiment mentions an aerial sample data image of a method for generating cross-scene target recognition data based on a segmentation large model;

[0031] Figure 3 This embodiment mentions a method for generating cross-scene target recognition data based on a segmentation large model to extract an image of the target area;

[0032] Figure 4 This embodiment mentions a method for generating cross-scene target recognition data based on a segmentation large model, which extracts target instance images based on a segmentation large model;

[0033] Figure 5 This embodiment mentions a method for generating cross-scene target recognition data based on a segmentation large model, which fuses a target instance with a background example image. DETAILED DESCRIPTION

[0034] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0035] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0036] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present invention, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0037] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0038] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0039] Example 1

[0040] In one embodiment of the present invention, see Figure 1 The present invention provides a method for generating cross-scene target recognition data based on a segmentation large model, comprising the following steps:

[0041] 1. Obtain a small batch of data containing the target of interest, manually mark the rectangular frame, and use it as the recognition base data;

[0042] 2. Use the base data as training data and use YOLOv8 to train a small object recognizer C;

[0043] 3. Perform target detection and recognition on all data samples to obtain target area and category information;

[0044] 4. Load the pre-trained segmentation model;

[0045] 5. Set and filter the target types of interest in the data sample;

[0046] 6. Collect statistics on image data annotation;

[0047] 7. Take the center point of the rectangular box in the source data as the prompt, combine it with the target area map, input it into the segmentation model, obtain the target mask shape and position, and superimpose it with the original image to obtain the target instance;

[0048] 8. Perform random scaling, rotation, noise addition, brightness change and other data amplification on the acquired target instances to obtain an amplified target instance set;

[0049] 9. Obtain the background data set to be amplified, and randomly paste the amplified target instance set on the background data set to be amplified in batches according to the set target quantity, target area and other priors. After random transformation, each target instance is pasted on a random position of the background data to measure the amplification multiple as a proportional factor;

[0050] 10. Determine the target annotation information according to the target category and the specific location of the paste, and merge it with the image data as training data to obtain the final target recognition data set.

[0051] The specific target identification operation process is as follows:

[0052] 1. Get the original data sample set = {I 1 …I z},like Figure 2 , through random sampling, the samples are sampled in small batches according to 100:1, set′={I 1 …I z};

[0053] 2. Use Labelme and other software to label small batches of samples (I z , Label z ));

[0054] 3. Based on the labeled data, use YOLOv8 to train the target recognition model of the data set

[0055] 4. Make predictions for all original samples Get the target position (upper left point ((x l ,y l )), lower right point ((x r ,y r )) and category ((L cls )); extract the image area block where the target is located, such as Figure 3 Filter the set target type images;

[0056] 5. If the target in some samples is not If recognition is performed, these images are processed separately;

[0057] 6. Load the pre-trained segmentation model SAM, based on the point prompt method, input the target area image block (I patch ) and the target center position coordinates ((x c ,y c )), Get the prediction results of the large model;

[0058] 7. In the image, the horizontal distance h x , longitudinal axis distance h y is the step length, automatically pick an anchor point ((x n ,y n )), and use the anchor point as the prompt input to obtain the m masks with the highest output confidence to generate potential target instances;

[0059] 8. By screening the prediction results of the large model, such as confidence ranking, the target mask shape M corresponding to the target image block is obtained patch , target location and other information, such as Figure 4 , and obtain the target instance image O by superimposing it with the original image patch :

[0060] O patch =M patch *I patch

[0061] 9. Perform scale transformation, random scale, rotation, noise addition, brightness change and other data amplification methods on the acquired target instances; among them, the target scale reduction ratio S is positively proportional to the noise intensity σ, such as Figure 5 Left half of the picture;

[0062] 10. Obtain a dataset of background images without targets to be augmented This dataset is close to actual application scenarios, such as air background, ground background, sea background, etc.

[0063] 11. The target instance set after expansion, Paste in batches on the background data set to be amplified, with the pasting position and area random but not exceeding the set area boundary, such as Figure 5 , each target instance Randomly paste it on the random position of k background data, use k as the scale factor to measure the amplification multiple, and finally obtain J*k synthetic image data;

[0064] 12. According to the target category and the specific location information of the paste, the target annotation information of the corresponding image can be automatically determined and merged with the image data as the final target recognition data set.

[0065] To sum up, the present invention is a method for generating cross-scene target recognition data based on a large segmentation model. For unlabeled sample data, a semi-automatic labeling method is provided, that is, by labeling small batches of data and training small target identifiers, the labeling processing of unlabeled sample data is realized, and the data processing efficiency is improved. In the augmentation process, the target instance is separated from the background and randomly transformed, which greatly improves the diversity of target morphology, background, and target relative background in the database, and improves the breadth of sample feature distribution in the database, thereby improving the generalization ability of the deep learning training model. In addition, in the augmentation process, the target morphology and quantity are randomly set, and the constructed database can be expanded and applied to single-target, multi-target detection, recognition and segmentation tasks, and labels can be automatically generated.

[0066] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for generating cross-scene target recognition training data based on a segmentation large model, characterized in that: The following steps are involved: S1. Collection of original data samples; S2, small batch labeling and target recognition model training; S3, perform target detection and recognition on all data; S4, based on the recognition result, the target image area is sent to the segmentation model; S5, obtain target mask, target position, and target instance; S6, performing random scale, rotation, noise, and brightness changes on the target; S7, fusing the changed target instance with the background, and performing batch data augmentation based on the target instance and the real background image data; S8. Automatically generate label information based on the fusion of target location and category priors to construct target recognition training and verification datasets.

2. The method for generating cross-scene target recognition training data based on a segmentation large model as claimed in claim 1, characterized in that: In the step S2, a small batch of data containing the target of interest is obtained, and after manually marking the rectangular frame, it is used as the recognition base data, and the base data is used as the training data to train the small target identifier C using YOLOv8.

3. The method for generating cross-scene target recognition training data based on a segmentation large model as claimed in claim 1, characterized in that: In step S3, a small target identifier is trained to perform target detection and recognition on all data samples, and target area and category information is automatically obtained.

4. The method for generating cross-scene target recognition training data based on a segmentation large model according to claim 1, characterized in that: In step S4, the target area and category information in the image are used to screen the target type in the data sample, and the pre-trained segmentation model is loaded to achieve segmentation and separation of the target instance and the local background.

5. The method for generating cross-scene target recognition training data based on a segmentation large model as claimed in claim 1, characterized in that: In step S5, the pre-trained segmentation model is used, the center point of the rectangular box in the source data is marked as a prompt, combined with the target area map, and input into the segmentation model to achieve segmentation and separation of the target instance and the local background, and obtain the target mask, target position, and target instance.

6. The method for generating cross-scene target recognition training data based on a segmentation large model according to claim 1, characterized in that: In step S6, a target transformation range is set, and data amplification of random scale, rotation, noise addition, and brightness change is performed on the acquired target instances to obtain an amplified target instance set.

7. The method for generating cross-scene target recognition training data based on a segmentation large model according to claim 1, characterized in that: In step S7, the background data set to be amplified is obtained, and the amplified target instance set is randomly selected and randomly pasted on the background data set to be amplified according to the experience of the set target quantity and area range.

8. The method for generating cross-scene target recognition training data based on a segmentation large model as claimed in claim 1, characterized in that: In the step S8, automatic label generation of augmented data is achieved based on known target pasting position and target category information.

9. The method for generating cross-scene target recognition training data based on a segmentation large model as claimed in claim 1, characterized in that: In step S8, the original data samples are mixed with the generated data samples to form a sample database.

Images