Target detection method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a target detection method and device, electronic equipment and a readable storage medium. The method comprises the steps that a training image and a background image corresponding to a source scene are acquired; constructing an annotated image training sample pair according to the annotated data and the background image; constructing an unlabeled image training sample pair according to the unlabeled data and the background image; training a to-be-trained target detection model by using the annotated image training sample pair and the unannotated image training sample pair to obtain a pre-training model; obtaining a target background image of the target scene; migrating the target area to the target background image; according to the embodiment of the invention, the training efficiency of the target detection model for the target scene is improved, and the deployment efficiency of the target detection model is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a target detection method, a target detection device, an electronic device, and a computer-readable storage medium. Background Art

[0002] The task of target detection is to find all the targets (objects) of interest in an image and determine the category and location of the targets. It is one of the core issues in the field of computer vision. With the rapid development of artificial intelligence technology and hardware computing power, the speed and accuracy of target detection models have also been greatly improved, and it has become the core technology of many automation systems. The target detection model helps automation equipment accurately identify and locate objects in images or videos so that it can perform corresponding tasks, which not only improves production efficiency, but also enhances the intelligence level of the system.

[0003] At present, the data required for training target detection models in the field of automation is usually collected in a single scenario (environment) (such as a laboratory scenario or a scenario of an automated production line). However, the performance of the target detection model trained with data from a single scenario will degrade after the environment changes. When the target detection model needs to be used in different scenarios (such as production line changes or large differences in the environments of different application scenarios), it is necessary to collect data as training images again in the target scene and annotate them, and then use them to fine-tune the target detection model to ensure model performance. However, this method is time-consuming and labor-intensive, resulting in low training efficiency of the target detection model, which in turn leads to low deployment efficiency of the target detection model. Summary of the invention

[0004] The embodiments of the present invention provide a target detection method, device, electronic device and computer-readable storage medium to solve the problem that the training efficiency of the target detection model is low, which in turn leads to the low deployment efficiency of the target detection model.

[0005] An embodiment of the present invention discloses a target detection method, the method comprising:

[0006] Acquire a training image and a background image corresponding to the source scene; wherein the training image includes labeled data and unlabeled data, the training image is an image including a target area, and the background image does not include an image of the target area;

[0007] Constructing a labeled image training sample pair according to the labeled data and the background image;

[0008] Constructing an unlabeled image training sample pair according to the unlabeled data and the background image;

[0009] Using the labeled image training sample pair and the unlabeled image training sample pair to train the target detection model to be trained to obtain a pre-trained model;

[0010] Acquire a target background image of a target scene, and migrate the target area to the target background image;

[0011] The pre-trained model is adjusted by adopting the target background image of the migrated target area, and the adjusted pre-trained model is used as a trained target detection model to detect the target in the target scene according to the adjusted target detection model.

[0012] The embodiment of the present invention also discloses a target detection model training device, the device comprising:

[0013] A first acquisition module is used to acquire a training image and a background image corresponding to a source scene; wherein the training image includes annotated data and unannotated data, the training image includes a target area, and the background image does not include the target area;

[0014] A first construction module, configured to construct a labeled image training sample pair according to the labeled data and the background image;

[0015] A second construction module is used to construct an unlabeled image training sample pair according to the unlabeled data and the background image;

[0016] A pre-training module, used to train the target detection model to be trained by using the labeled image training sample pair and the unlabeled image training sample pair to obtain a pre-training model;

[0017] A second acquisition module is used to acquire a target background image of a target scene and migrate the target area to the target background image;

[0018] An adjustment module is used to adjust the pre-trained model by adopting the target background image of the migrated target area, and use the adjusted pre-trained model as a trained target detection model to detect the target in the target scene according to the adjusted target detection model.

[0019] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0020] The memory is used to store computer programs;

[0021] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0022] The embodiment of the present invention further discloses a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the embodiment of the present invention.

[0023] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0024] The embodiments of the present invention include the following advantages:

[0025] In an embodiment of the present invention, a training image and a background image corresponding to a source scene are obtained, and a labeled image training sample pair is constructed according to the labeled data and the background image, and an unlabeled image training sample pair is constructed according to the unlabeled data and the background image, and then the labeled image training sample pair and the unlabeled image training sample can be used to compare and train the target detection model to be trained to obtain a pre-trained model, and then, a target background image of the target scene is obtained, the target area is migrated to the target background image, and the pre-trained model is adjusted using the target background image to obtain a trained target detection model. In an embodiment of the present invention, after the pre-trained model is obtained based on the training image (including the target area) and the background image corresponding to the source scene, since the pre-trained model is a model obtained by preliminary training, the target background image of the target scene is further obtained and combined with the target area to generate the target background image to train the pre-trained model, and there is no need to train the model from scratch, thereby improving the training efficiency of the target detection model for the target scene, and thereby improving the deployment efficiency of the target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a target detection method provided in an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of data scenario migration provided in an embodiment of the present invention;

[0028] Figure 3 is a diagram of a labeled image comparison learning process provided in an embodiment of the present invention;

[0029] Figure 4 is a diagram of an unlabeled image comparison learning process provided in an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of an object feature position search provided in an embodiment of the present invention;

[0031] Figure 6 It is a general framework diagram of target detection model training and deployment provided in an embodiment of the present invention;

[0032] Figure 7 is a semi-supervised contrastive learning model training flow chart provided in an embodiment of the present invention;

[0033] Figure 8 is a structural block diagram of a target detection device provided in an embodiment of the present invention;

[0034] Fig. 9 It is a schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] At present, most target detection models in the field of automation use an end-to-end supervised learning training method, that is, the parameters of the target detection model are adjusted through a large amount of labeled data, so that the target detection model can accurately find the optimal mapping relationship between the input data and the label data. However, this end-to-end supervised learning training method relies on a large amount of manually labeled data. It is time-consuming and laborious to achieve high-quality labeled data, which not only increases the R&D cost, but also is not conducive to the rapid development of target detection models.

[0037] In addition, in order to ensure the universality of the target detection model, it is usually necessary to collect a large amount of data from different scenarios, which is difficult and time-consuming. In order to avoid the deployment cycle of the target detection model being too long, the data required for target detection model training in the automation field is usually collected in a single scenario (such as a laboratory scenario or a scenario of an automated production line). However, the performance of the model trained with data in a single environment will degrade after the environment changes. When the model has different usage requirements in different scenarios (such as production line changes, large differences in the environments of different application scenarios), it is often necessary to collect data again in the target scenario and annotate it for model fine-tuning to ensure the performance of the target detection model. This method is not conducive to the rapid deployment of the target detection model.

[0038] In view of the above problems, the embodiment of the present invention proposes a semi-supervised pre-training method for a target detection model based on contrastive learning, wherein the training samples (training images) of unlabeled data are extracted by an open source visual large model and a background difference algorithm, and a sample construction strategy of the same instance (e.g., the same category) as a positive sample pair is adopted. The labeled data adopts a sample construction strategy of the same category target as a positive sample pair, and the labeled data and the unlabeled data are simultaneously sent to the neural network of the target detection model to extract features; by setting anchor points on the image and searching for features generated by the sample according to the principle of the closest distance, contrastive learning training is performed; and by modifying the loss function, different loss weights are set for the labeled data and the unlabeled data to prevent the target detection model from overfitting, and finally reduce the dependence on a large amount of labeled data during the training of the target detection model, improve the training efficiency of the target detection model in the field of automation, and then improve the deployment efficiency of the target detection model. Further, the embodiment of the present invention also proposes a method for rapid training and deployment of a model in different scenarios that combines data migration and model fine-tuning, by migrating data to the target scene to generate pseudo data, and then fine-tuning the model, the adaptability of the target detection model to the target scene is improved, and there is no need to collect and label data again for different scenes, so as to improve the performance of the target detection model in the target scene.

[0039] Reference Figure 1 , shows a step flow chart of a target detection method provided in an embodiment of the present invention, which may specifically include the following steps:

[0040] Step 101, obtaining a training image and a background image corresponding to a source scene; wherein the training image includes labeled data and unlabeled data, the training image is an image including a target area, and the background image does not include an image of the target area;

[0041] Step 102: construct a labeled image training sample pair according to the labeled data and the background image;

[0042] Step 103: construct an unlabeled image training sample pair according to the unlabeled data and the background image;

[0043] In the field of automation, the task of target detection is to find all targets (objects) of interest in an image and determine the category and location of the target. For example, in the industrial field, the target can be various mechanical parts, such as bolts, nuts, bearings, gears, etc. The image is used to detect the parts on a production line for quality inspection to ensure the integrity and correctness of the parts and the quality of the products produced using these parts.

[0044] In a specific implementation, an image can be divided into a foreground area and a background area, which are used to distinguish between an area containing a target (foreground area) and an area not containing a target (background area) in the image. The background area is usually the part of the image excluding the target bounding box.

[0045] In an embodiment of the present invention, when the parameters of the target detection model are initialized, a training image is read from the data set. Specifically, a batch of training images can be sampled from the data set for each training. A batch of training images can include labeled data (labeled images) and unlabeled data (unlabeled images). The ratio of the two is random. In addition, the training image is an image including the target area (foreground area / labeled area), and the background image does not include the image of the target area. Next, each training image is traversed, and then based on whether it is labeled data or unlabeled data, a labeled image training sample pair and an unlabeled image training sample pair are constructed for comparative learning. Among them, the target area of ​​the labeled data can include a target bounding box (object bounding box) and a category. The target bounding box and the category are used to train the target detection model so that it can accurately identify and locate the target in the image (such as the category and position of the target). Specifically, the target bounding box is usually a rectangular box, which is used to frame the target area including the target in the image, so as to locate the position of the target in the image, such as the area including nuts in the image; the category refers to the type of the target. For example, when detecting bolts, nuts, cars, traffic signs and other targets in the image, these targets all belong to different categories.

[0046] Step 104: Use the labeled image training sample pair and the unlabeled image training sample pair to train the target detection model to be trained to obtain a pre-trained model;

[0047] Contrastive learning is a self-supervised learning method that learns data representation by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs. Contrastive learning can learn discriminative feature representations and improve the generalization ability and robustness of target detection models.

[0048] In an embodiment of the present invention, a semi-supervised contrastive learning is performed on the target detection model to be trained using a labeled image training sample pair and an unlabeled image training sample pair to obtain a pre-trained model. The pre-trained model can extract features from images, and make the features of targets of the same category closer in the feature space, and the features of targets of different categories and targets and backgrounds farther apart, thereby achieving the effect of target feature clustering; in addition, since objects are randomly placed at different positions in the image during the training process, the calculation of the contrast loss takes into account the influence of the target position, so the pre-trained model has a preliminary target positioning function.

[0049] Step 105: Acquire a target background image of a target scene, and migrate the target area to the target background image;

[0050] Step 106: Use the target background image of the target area to adjust the pre-trained model, and use the adjusted pre-trained model as a trained target detection model to detect the target in the target scene according to the adjusted target detection model.

[0051] In an embodiment of the present invention, a pre-trained model can be used for feature extraction. To realize the function of target detection, a detection head needs to be connected and the model needs to be fine-tuned. The data used for model fine-tuning is annotated data. In order to make the final target detection model meet the use requirements of the new scene (target scene), the original data is subjected to scene migration. Specifically, the scene migration can be to obtain a target background image of the target scene, wherein the target background image does not include the target area of ​​the target, and then the target area can be migrated to the target background image to obtain an image containing the target area belonging to the target scene. Therefore, the pre-trained model is adjusted based on the image, and a target detection model for the target scene can be quickly obtained.

[0052] In the embodiment of the present invention, the main function of the pre-trained model is feature extraction. To realize the function of target detection, it is necessary to connect the detection head and perform model fine-tuning. The data used for model fine-tuning is the labeled data. In order to make the final target detection model meet the requirements of the target scene, the original data is migrated to the scene, referring to Figure 2 , is a schematic diagram of a data scenario migration provided in an embodiment of the present invention, and the specific process includes:

[0053] (1) Connect the detection head. Connect the detection head network model after the pre-trained model and initialize it; (2) Data scene migration. Cut the marked object area, perform data enhancement processing, and randomly paste it into the background image of the target scene. Data enhancement processing can include: random cropping, rotation and translation, color change, random noise, random object deletion, etc. (3) Model fine-tuning. Freeze the pre-trained model parameters and only adjust the parameters of the detection head network model. The adjusted pre-trained model is used as the trained target detection model to detect targets in the target scene.

[0054] In one embodiment of the present invention, the method may further include:

[0055] The image to be detected is input into the trained target detection model to obtain the category and position of the target in the image to be detected.

[0056] After obtaining the target detection model for the target scene, the target detection model can be deployed in the target scene, and then the category and position of the target in the target scene can be detected based on the target detection model.

[0057] In an embodiment of the present invention, a training image and a background image corresponding to a source scene are obtained, and a labeled image training sample pair is constructed according to the labeled data and the background image, and an unlabeled image training sample pair is constructed according to the unlabeled data and the background image, and then the labeled image training sample pair and the unlabeled image training sample can be used to compare and train the target detection model to be trained to obtain a pre-trained model, and then, a target background image of the target scene is obtained, the target area is migrated to the target background image, and the pre-trained model is adjusted using the target background image to obtain a trained target detection model. In an embodiment of the present invention, after the pre-trained model is obtained based on the training image (including the target area) and the background image corresponding to the source scene, since the pre-trained model is a model obtained by preliminary training, the target background image of the target scene is further obtained and combined with the target area to generate the target background image to train the pre-trained model, and there is no need to train the model from scratch, thereby improving the training efficiency of the target detection model for the target scene, and thereby improving the deployment efficiency of the target detection model.

[0058] In an embodiment of the present invention, the target area includes a target bounding box and a category, and constructing a labeled image training sample pair according to the labeled data and the background image includes:

[0059] intercepting a target area in the annotated data;

[0060] Randomly obtain a first background image from the background images;

[0061] Copying the first background image to obtain a second background image;

[0062] A labeled image training sample pair is constructed according to the target area, the first background image, and the second background image.

[0063] Reference Figure 3 , is a diagram of a labeled image contrast learning process provided in an embodiment of the present invention. The process of constructing a labeled image training sample pair may include: if the training data is labeled data (target bounding box and category), first, read the labeled data (labeled image), intercept each foreground area labeled in the labeled image (i.e., the target area containing the target), and randomly read a background image, recorded as background image 1 (first background image), then copy a background image 1, recorded as background image 2 (second background image), and then construct a labeled image training sample pair based on the target area, the first background image and the second background image for model training of the target detection model.

[0064] The labeled image training sample pairs include labeled positive sample pairs and labeled negative sample pairs; constructing the labeled image training sample pairs according to the target area, the first background image and the second background image includes:

[0065] Acquire a specific target area from the target areas;

[0066] Performing data enhancement processing on the specific target area to obtain an enhanced specific target area;

[0067] Randomly pasting the enhanced specific target area to the first background image to obtain an enhanced first background image;

[0068] Randomly acquire a positive target region from the target region that has the same category as the specific target region;

[0069] Performing data enhancement processing on the positive target region to obtain an enhanced positive target region;

[0070] Randomly pasting the enhanced positive target area to the second background image to obtain a positive second background image;

[0071] Constructing a labeled positive sample pair according to the enhanced first background image and the positive second background image;

[0072] Randomly acquire a negative target region from the target region that is different from the category of the specific target region;

[0073] Randomly pasting the negative target area to the second background image to obtain a negative second background image;

[0074] A labeled negative sample pair is constructed according to the enhanced first background image and the negative second background image.

[0075] The data enhancement processing may at least include rotation and translation, random cropping, color change, blurring, random noise addition, and object randomization.

[0076] Reference Figure 3, after obtaining background image 1 (first background image) and background image 2 (second background image), traverse each target area, for a specific specific target area, the captured specific target area is subjected to data enhancement processing (rotation and translation, random cropping, color change, blur and other data enhancement processing), and then randomly pasted to any position in background image 1 to obtain an enhanced first background image; then, with a probability of 50% for each specific target area, a labeled positive sample pair or a labeled negative sample pair is constructed, specifically as follows: for the construction of labeled positive samples: randomly search a target area of ​​the same category from the batch of training images as the positive target area, perform data enhancement on the positive target area to obtain an enhanced positive target area, and randomly paste it to any position in background image 2 to obtain a positive second background image; finally, a labeled positive sample pair can be constructed based on the enhanced first background image and the positive second background image. For the construction of negative samples: randomly select target areas of different categories from the specific target area in the batch data or randomly cut out the background area as the negative target area. After the negative target area is subjected to data enhancement processing (rotation and translation, random cropping, color change, blurring and other data enhancement processing), it is randomly pasted to any position in the background image 2 (second background image) to obtain the negative second background image. According to the enhanced first background image and the negative second background image, a labeled negative sample pair is constructed.

[0077] In one embodiment of the present invention, constructing an unlabeled image training sample pair according to the unlabeled data and the background image includes:

[0078] Segmenting a segmentation region from the unlabeled data using a visual model;

[0079] extracting a background area from the background image using a background difference algorithm;

[0080] Determining whether the segmented area is a foreground area according to the background area; wherein the foreground area is an area including an object;

[0081] When it is determined that the segmented area is a foreground area, taking the segmented area as a target segmented area;

[0082] randomly acquiring a third background image from the background images;

[0083] Copying the third background image to obtain a fourth background image;

[0084] An unlabeled image training sample pair is constructed according to the target segmentation area, the third background image and the fourth background image.

[0085] Reference Figure 4, is a diagram of an unlabeled image contrast learning process provided in an embodiment of the present invention, and the process of constructing an unlabeled image training sample pair may include: for unlabeled training data, first, reading the unlabeled data (unlabeled image), using a visual model with a panoramic segmentation function (such as the SegAnything visual large model) to perform pixel-level panoramic segmentation on the unlabeled image, and separating each segmented area in the image (at this time, it is impossible to determine whether the segmented area belongs to the foreground area or the background area, and the specific category of the target); if it is determined that the segmented area belongs to the foreground area, the segmented area can be used as the target segmented area; randomly reading a background image, recorded as background image 3 (the third background image), and then copying the background image 3, recorded as background image 4 (the fourth background image); then, according to the target segmented area, the third background image and the fourth background image, an unlabeled image training sample pair can be constructed for model training of the target detection model.

[0086] In an embodiment of the present invention, determining whether the segmented area is a foreground area according to the background area includes:

[0087] Calculate the pixel area intersection-over-union ratio of the segmented area and the background area;

[0088] Determining whether the segmented area is a foreground area according to the pixel area intersection-and-combination ratio;

[0089] If the pixel area intersection-over-union ratio is less than a preset threshold, the segmented area is determined to be a foreground area.

[0090] In an embodiment of the present invention, a background image is read, a background area of ​​the background image is extracted using a background difference algorithm, and the pixel area intersection-and-union ratio between the target segmentation area and the background area segmented by the large visual model is calculated. When the pixel area intersection-and-union ratio is less than a preset threshold value T, the target segmentation area is considered to be a foreground area (target area). The foreground area contains a target to be detected, and the bounding box of the foreground area is the position of the target. Therefore, the segmentation area with a pixel area intersection-and-union ratio less than the preset threshold value T can be used as the target segmentation area.

[0091] In one embodiment of the present invention, the labeled image training sample pairs include unlabeled positive sample pairs and unlabeled negative sample pairs; constructing unlabeled image training sample pairs according to the target segmentation area, the third background image and the fourth background image includes:

[0092] Performing data enhancement processing on the target segmentation region to obtain an enhanced target segmentation region;

[0093] Randomly pasting the enhanced target segmented area to the third background image to obtain an enhanced third background image;

[0094] Randomly pasting the enhanced target segmented area to the fourth background image to obtain an enhanced fourth background image;

[0095] Constructing an unlabeled positive sample pair according to the enhanced third background image and the enhanced fourth background image;

[0096] Randomly acquiring a random background area as a non-foreground area from the segmented area;

[0097] Randomly pasting the random background area to the fourth background image to obtain a random fourth background image;

[0098] An unlabeled negative sample pair is constructed according to the enhanced third background image and the random fourth background image.

[0099] Reference Figure 4 , traverse each segmented target segmentation area, and after the target segmentation area is subjected to data enhancement processing (rotation and translation, random cropping, color change, blur and other data enhancement processing) to obtain an enhanced target segmentation area, it is randomly pasted to any position in the background image 3 to obtain an enhanced third background image; with a probability of 50% each, an unlabeled positive sample pair or an unlabeled negative sample pair is constructed for the target segmentation area: data enhancement is performed on the target segmentation area, and it is randomly pasted to any position in the background image 2 to obtain an enhanced fourth background image, and an unlabeled positive sample pair is constructed based on the enhanced third background image and the enhanced fourth background image; the background area of ​​the background image is randomly intercepted, and it is randomly pasted to any position in the background image 4 to obtain a random fourth background image, and an unlabeled negative sample pair is constructed based on the enhanced third background image and the random fourth background image.

[0100] Of course, the above-mentioned construction of labeled image training sample pairs and unlabeled image training sample pairs is only for example. The embodiment of the present invention can use other methods to construct positive and negative sample pairs for the target, and the embodiment of the present invention does not need to be limited to this.

[0101] In one embodiment of the present invention, the labeled image training sample pair and the unlabeled image training sample pair are used to train the target detection model to be trained to obtain a pre-trained model, including:

[0102] Inputting the labeled image training sample pair and the unlabeled image training sample into the target detection model to be trained to obtain a feature map;

[0103] Searching for features corresponding to the target area from the feature map according to the training image;

[0104] The features corresponding to the labeled image training samples are compared and learned, and the features corresponding to the unlabeled image training samples are compared and learned to obtain a pre-trained model.

[0105] In an embodiment of the present invention, the labeled image training sample pairs and the unlabeled image training sample pairs are input into the target detection model to be trained to obtain the feature graph of the sample pairs (labeled positive sample pairs, labeled negative sample pairs, unlabeled positive sample pairs, unlabeled negative sample pairs, etc.), and then the features corresponding to the sample pairs in the feature graph are extracted for comparative learning, the contrast loss between the features of the sample pairs is calculated, and the model parameters are adjusted by back propagation so that the distance between the features of the positive sample pairs is getting closer and closer, and the distance between the features of the negative sample pairs is getting farther and farther. The contrast loss calculation formula is shown in formula (1):

[0106]

[0107] In the formula, D W (X1,X2)=||X1-X2||2, represents the Euclidean distance between the two sample features X1 and X2, W is the model parameter, Y is the label of whether the two samples match, Y=1 represents that the two samples match, which is a positive sample pair, Y=0 represents a negative sample pair, m is the set threshold, N is the number of samples, Z is the label of whether the sample pair is a labeled sample, Z=1 represents that the sample pair is generated by labeled data, Z=0 represents that the sample pair is generated by unlabeled data, c1 and c2 are weight parameters used to control the impact of the loss of labeled data samples and unlabeled data samples on the model. Since the unlabeled data is far more than the labeled data, and the unlabeled data sample pairs are all obtained through data enhancement for the same target, the features are relatively similar, and the training is prone to overfitting. In order to prevent the model training from overfitting, the value of c1 needs to be larger than the value of c2 to reduce the impact of the loss of unlabeled data on the model parameters, and c1+c2=1.

[0108] After the target detection model to be trained is trained by the above-mentioned semi-supervised contrastive learning, a pre-trained model is obtained. The pre-trained model can extract features from the image, and make the features of targets (objects) of the same category closer in the feature space, and the features of objects of different categories and objects and backgrounds farther apart, thereby achieving the effect of target feature clustering; in addition, since the objects are randomly placed at different positions of the image during the training process, the calculation of the contrast loss takes into account the influence of the target position, and thus the pre-trained model of the embodiment of the present invention has a preliminary target positioning function.

[0109] In an embodiment of the present invention, searching for features corresponding to the target area from the feature map according to the training image includes:

[0110] Divide the corresponding training image into a plurality of regions having the same resolution as the feature map;

[0111] Taking the center point of the region as an anchor point;

[0112] Calculate the distance from the center point of the target bounding box in the training image to the anchor point;

[0113] The position of the anchor point closest to the training image in the area divided by the training image is used as the target position of the target area in the feature map;

[0114] The feature corresponding to the target position in the feature map is used as the feature corresponding to the target area.

[0115] In the embodiment of the present invention, refer to Figure 5 , is a schematic diagram of an object feature position search provided in an embodiment of the present invention. Background image 1, background image 2, background image 3 and background image 4 are processed to obtain images, which are input into the target detection model to be trained for feature extraction. Since the resolution of the image changes after being processed by the neural network, the feature position extracted from the target object also changes accordingly. The method for searching for target object features in the feature map is as follows Figure 5 As shown in the figure. According to the resolution of the feature map, the original image (training image) is divided into several regions with the same size as the feature map resolution. The center point of the region is the anchor point. The distance from the center point of the target bounding box to each anchor point is calculated, and the position of the nearest anchor point in the region divided by the original image is used as the position of the object feature in the feature map.

[0116] In one embodiment of the present invention, the method further comprises:

[0117] Collecting training images and background data in a source scene; wherein the training image includes a target area to be targeted, and the background data does not include the target area to be targeted;

[0118] Dividing the training image into to-be-labeled data and unlabeled data according to a preset ratio;

[0119] The labeled data is used to label the data to be labeled; wherein the labeled data includes a target bounding box and a category.

[0120] In an embodiment of the present invention, training data / training images (including targets) and background data (excluding targets) can be collected in a certain or a small number of source scenes (such as laboratory scenes or a few workstation scenes in a production line), and the training images are divided into data to be labeled and unlabeled data at a ratio of a:b. The data to be labeled is labeled (including manual labeling or semi-automatic labeling using auxiliary tools) to obtain labeled data. The labeled data may include target bounding boxes and categories. Usually, the proportion of data to be labeled is smaller than the proportion of unlabeled data.

[0121] In the pre-training stage, the embodiment of the present invention mainly uses a large amount of unlabeled data and a small amount of labeled data to perform semi-supervised training on the target detection model to be trained, and uses the contrastive learning training paradigm to obtain the pre-trained model; in the model fine-tuning stage, a small amount of background images that do not contain the target to be detected in the target scene are collected, and a small amount of labeled data of the source scene is migrated to the target scene by using data migration. The pre-trained model is adjusted with the migrated data to obtain the target detection model for the target scene, and finally the target detection model is deployed in the target scene for target detection.

[0122] In order to solve the problem that deep learning target detection model training in the field of automation relies on a large amount of labeled data, the embodiments of the present invention propose a semi-supervised model pre-training method based on contrastive learning. This method can perform model pre-training with the help of a large amount of unlabeled data and a small amount of labeled data, thereby reducing the amount of labeled data required for training. In addition, in order to solve the problem that model performance deteriorates due to changes in application scenarios (target scenarios) or large differences in environments when the model is deployed, the embodiments of the present invention generate pseudo samples in application scenarios through data migration, and combine model fine-tuning to improve the adaptability of target detection models to application scenarios, without the need to collect and label data again for different scenarios.

[0123] Reference Figure 6 , is a general framework diagram of model training and deployment provided in an embodiment of the present invention, and the main process may include: data collection; data division, that is, division into unlabeled data and data to be labeled, and the data to be labeled can be labeled to obtain labeled data; enter semi-supervised comparison learning, input the labeled data and the unlabeled data into the initialization model (the target detection model to be trained) for comparative learning and training to obtain a pre-trained model; based on the labeled data and the background data of the target scene (target background image), data migration is performed to obtain target scene data (including target area), and then the pre-trained model is fine-tuned according to the target scene data to obtain a trained target detection model, which can be deployed in the target scene for target detection.

[0124] Reference Figure 7, is a semi-supervised contrastive learning model training flow chart provided in an embodiment of the present invention, and the specific process includes: initializing model parameters to obtain a target detection model to be trained; reading training data (training images); determining whether the training data is labeled (i.e., labeled data or unlabeled data); if labeled, extracting the object labeled area (target area); constructing positive sample pairs for objects of the same category; constructing negative sample pairs for objects of different categories; data enhancement + random mapping, and finally obtaining labeled image training sample pairs; using the labeled image training sample pairs to perform contrastive learning training on the target detection model to be trained; if unlabeled, the SegAnything visual large model performs Perform panoramic segmentation; use background difference to extract foreground targets; enhance the data of the same foreground object to construct positive sample pairs; randomly crop the foreground area to construct negative sample pairs, and finally obtain unlabeled image training sample pairs; use the unlabeled image training sample pairs to perform comparative learning training on the target detection model to be trained; fuse the labeled image training sample pairs and the unlabeled image training sample pairs to obtain the model obtained by comparative learning training to obtain the pre-trained model; perform data scene migration based on the background image of the target scene to obtain the target scene data containing the target area of ​​the target scene; use the target scene data to fine-tune the model to obtain the trained target detection model for deployment in the target scene for target detection.

[0125] The embodiment of the present invention implements a target detection model training scheme commonly used in the field of automation. The target detection model is pre-trained through semi-supervised contrastive learning, which greatly reduces the workload of data labeling. Then, the performance of the target detection model in the target scene is improved by combining data migration and model fine-tuning technology, thereby realizing the rapid deployment of the model in different target scenes.

[0126] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0127] Reference Figure 8 , shows a structural block diagram of a target detection device provided in an embodiment of the present invention, which may specifically include the following modules:

[0128] The first acquisition module 801 is used to acquire a training image and a background image corresponding to a source scene; wherein the training image includes annotated data and unannotated data, the training image includes a target area, and the background image does not include the target area;

[0129] A first construction module 802, configured to construct a labeled image training sample pair according to the labeled data and the background image;

[0130] A second construction module 803 is used to construct an unlabeled image training sample pair according to the unlabeled data and the background image;

[0131] A pre-training module 804 is used to train the target detection model to be trained by using the labeled image training sample pair and the unlabeled image training sample pair to obtain a pre-training model;

[0132] The second acquisition module 805 is used to acquire a target background image of a target scene and migrate the target area to the target background image;

[0133] The adjustment module 806 is used to adjust the pre-trained model by adopting the target background image of the migrated target area, and use the adjusted pre-trained model as the trained target detection model to detect the target in the target scene according to the adjusted target detection model.

[0134] In an embodiment of the present invention, the target area includes a target bounding box and a category, and constructing a labeled image training sample pair according to the labeled data and the background image includes:

[0135] intercepting a target area in the annotated data;

[0136] Randomly obtain a first background image from the background images;

[0137] Copying the first background image to obtain a second background image;

[0138] A labeled image training sample pair is constructed according to the target area, the first background image, and the second background image.

[0139] In one embodiment of the present invention, the labeled image training sample pair includes a labeled positive sample pair and a labeled negative sample pair; constructing the labeled image training sample pair according to the target area, the first background image and the second background image includes:

[0140] Acquire a specific target area from the target areas;

[0141] Performing data enhancement processing on the specific target area to obtain an enhanced specific target area;

[0142] Randomly pasting the enhanced specific target area to the first background image to obtain an enhanced first background image;

[0143] Randomly acquire a positive target region from the target region that has the same category as the specific target region;

[0144] Performing data enhancement processing on the positive target region to obtain an enhanced positive target region;

[0145] Randomly pasting the enhanced positive target area to the second background image to obtain a positive second background image;

[0146] Constructing a labeled positive sample pair according to the enhanced first background image and the positive second background image;

[0147] Randomly acquire a negative target region from the target region that is different from the category of the specific target region;

[0148] Randomly pasting the negative target area to the second background image to obtain a negative second background image;

[0149] A labeled negative sample pair is constructed according to the enhanced first background image and the negative second background image.

[0150] In one embodiment of the present invention, constructing an unlabeled image training sample pair according to the unlabeled data and the background image includes:

[0151] Segmenting a segmentation region from the unlabeled data using a visual model;

[0152] extracting a background area from the background image using a background difference algorithm;

[0153] Determining whether the segmented area is a foreground area according to the background area; wherein the foreground area is an area including an object;

[0154] When it is determined that the segmented area is a foreground area, taking the segmented area as a target segmented area;

[0155] randomly acquiring a third background image from the background images;

[0156] Copying the third background image to obtain a fourth background image;

[0157] An unlabeled image training sample pair is constructed according to the target segmentation area, the third background image and the fourth background image.

[0158] In an embodiment of the present invention, determining whether the segmented area is a foreground area according to the background area includes:

[0159] Calculate the pixel area intersection-over-union ratio of the segmented area and the background area;

[0160] Determining whether the segmented area is a foreground area according to the pixel area intersection-and-combination ratio;

[0161] If the pixel area intersection-over-union ratio is less than a preset threshold, the segmented area is determined to be a foreground area.

[0162] In one embodiment of the present invention, the labeled image training sample pairs include unlabeled positive sample pairs and unlabeled negative sample pairs; constructing unlabeled image training sample pairs according to the target segmentation area, the third background image and the fourth background image includes:

[0163] Performing data enhancement processing on the target segmentation region to obtain an enhanced target segmentation region;

[0164] Randomly pasting the enhanced target segmented area to the third background image to obtain an enhanced third background image;

[0165] Randomly pasting the enhanced target segmented area to the fourth background image to obtain an enhanced fourth background image;

[0166] Constructing an unlabeled positive sample pair according to the enhanced third background image and the enhanced fourth background image;

[0167] Randomly acquiring a random background area as a non-foreground area from the segmented area;

[0168] Randomly pasting the random background area to the fourth background image to obtain a random fourth background image;

[0169] An unlabeled negative sample pair is constructed according to the enhanced third background image and the random fourth background image.

[0170] In one embodiment of the present invention, the labeled image training sample pair and the unlabeled image training sample pair are used to train the target detection model to be trained to obtain a pre-trained model, including:

[0171] Inputting the labeled image training sample pair and the unlabeled image training sample into the target detection model to be trained to obtain a feature map;

[0172] Searching for features corresponding to the target area from the feature map according to the training image;

[0173] The features corresponding to the labeled image training samples are compared and learned, and the features corresponding to the unlabeled image training samples are compared and learned to obtain a pre-trained model.

[0174] In an embodiment of the present invention, searching for features corresponding to the target area from the feature map according to the training image includes:

[0175] Divide the corresponding training image into a plurality of regions having the same resolution as the feature map;

[0176] Taking the center point of the region as an anchor point;

[0177] Calculate the distance from the center point of the target bounding box in the training image to the anchor point;

[0178] The position of the anchor point closest to the training image in the area divided by the training image is used as the target position of the target area in the feature map;

[0179] The feature corresponding to the target position in the feature map is used as the feature corresponding to the target area.

[0180] In an embodiment of the present invention, the data enhancement processing at least includes rotation and translation, random cropping, color change, blurring, random noise addition and object randomization.

[0181] In one embodiment of the present invention, the method further includes:

[0182] Collecting training images and background data in a source scene; wherein the training image includes a target area to be targeted, and the background data does not include the target area to be targeted;

[0183] Dividing the training image into to-be-labeled data and unlabeled data according to a preset ratio;

[0184] The labeled data is used to label the data to be labeled; wherein the labeled data includes a target bounding box and a category.

[0185] In one embodiment of the present invention, the device further comprises:

[0186] The image to be detected is input into the trained target detection model to obtain the category and position of the target in the image to be detected.

[0187] In an embodiment of the present invention, the labeled data is less than the unlabeled data.

[0188] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0189] In addition, an embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned target detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0190] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned target detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0191] An embodiment of the present invention also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned target detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0192] Fig. 9 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0193] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, a processor 910, and a power supply 911. Those skilled in the art will appreciate that Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0194] It should be understood that in the embodiment of the present invention, the radio frequency unit 901 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 910 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 901 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 901 can also communicate with the network and other devices through a wireless communication system.

[0195] The electronic device provides users with wireless broadband Internet access through the network module 902, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0196] The audio output unit 903 can convert the audio data received by the RF unit 901 or the network module 902 or stored in the memory 909 into an audio signal and output it as sound. Moreover, the audio output unit 903 can also provide audio output related to a specific function performed by the electronic device 900 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 903 includes a speaker, a buzzer, a receiver, etc.

[0197] The input unit 904 is used to receive audio or video signals. The input unit 904 may include a graphics processor (GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 906. The image frame processed by the graphics processor 9041 can be stored in the memory 909 (or other storage medium) or sent via the radio frequency unit 901 or the network module 902. The microphone 9042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 901 in the case of a telephone call mode.

[0198] The electronic device 900 also includes at least one sensor 905, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 9061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 9061 and / or the backlight when the electronic device 900 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 905 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0199] The display unit 906 is used to display information input by the user or information provided to the user. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0200] The user input unit 907 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 907 includes a touch panel 9071 and other input devices 9072. The touch panel 9071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 9071 or near the touch panel 9071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 9071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the contact point coordinates, and then sends it to the processor 910, receives the command sent by the processor 910 and executes it. In addition, the touch panel 9071 can be implemented using various types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch panel 9071, the user input unit 907 may also include other input devices 9072. Specifically, other input devices 9072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0201] Furthermore, the touch panel 9071 may be overlaid on the display panel 9061. When the touch panel 9071 detects a touch operation on or near it, it transmits the information to the processor 910 to determine the type of the touch event. Then, the processor 910 provides a corresponding visual output on the display panel 9061 according to the type of the touch event. Fig. 9 In the figure, the touch panel 9071 and the display panel 9061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 9071 and the display panel 9061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0202] The interface unit 908 is an interface for connecting an external device to the electronic device 900. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 908 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 900 or may be used to transmit data between the electronic device 900 and an external device.

[0203] The memory 909 can be used to store software programs and various data. The memory 909 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 909 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0204] The processor 910 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 909, and calling data stored in the memory 909, so as to monitor the electronic device as a whole. The processor 910 may include one or more processing units; preferably, the processor 910 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 910.

[0205] The electronic device 900 may also include a power supply 911 (such as a battery) for supplying power to each component. Preferably, the power supply 911 may be logically connected to the processor 910 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0206] In addition, the electronic device 900 includes some functional modules not shown, which will not be described in detail here.

[0207] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0208] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0209] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0210] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0212] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0215] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0216] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a training image and a background image corresponding to the source scene; wherein the training image includes labeled data and unlabeled data, the training image is an image including a target area, and the background image does not include an image of the target area; Constructing a labeled image training sample pair according to the labeled data and the background image; Constructing an unlabeled image training sample pair according to the unlabeled data and the background image; Using the labeled image training sample pair and the unlabeled image training sample pair to train the target detection model to be trained to obtain a pre-trained model; Acquire a target background image of a target scene, and migrate the target area to the target background image; The pre-trained model is adjusted by adopting the target background image of the migrated target area, and the adjusted pre-trained model is used as a trained target detection model to detect the target in the target scene according to the adjusted target detection model.

2. The method according to claim 1, characterized in that The target area includes a target bounding box and a category, and constructing a labeled image training sample pair according to the labeled data and the background image, including: intercepting a target area in the annotated data; Randomly acquiring a first background image from the background images; Copying the first background image to obtain a second background image; A labeled image training sample pair is constructed according to the target area, the first background image, and the second background image.

3. The method according to claim 2, characterized in that The labeled image training sample pairs include a labeled positive sample pair and a labeled negative sample pair; constructing the labeled image training sample pairs according to the target area, the first background image and the second background image, including: Acquire a specific target area from the target areas; Performing data enhancement processing on the specific target area to obtain an enhanced specific target area; Randomly pasting the enhanced specific target area to the first background image to obtain an enhanced first background image; Randomly acquire a positive target region from the target region that has the same category as the specific target region; Performing data enhancement processing on the positive target region to obtain an enhanced positive target region; Randomly pasting the enhanced positive target area to the second background image to obtain a positive second background image; Constructing a labeled positive sample pair according to the enhanced first background image and the positive second background image; Randomly acquire a negative target region from the target region that is different from the category of the specific target region; Randomly pasting the negative target area to the second background image to obtain a negative second background image; A labeled negative sample pair is constructed according to the enhanced first background image and the negative second background image.

4. The method according to claim 1, characterized in that Constructing an unlabeled image training sample pair according to the unlabeled data and the background image, comprising: Segmenting a segmentation region from the unlabeled data using a visual model; extracting a background area from the background image using a background difference algorithm; Determining whether the segmented area is a foreground area according to the background area; wherein the foreground area is an area including an object; When it is determined that the segmented area is a foreground area, taking the segmented area as a target segmented area; randomly acquiring a third background image from the background images; Copying the third background image to obtain a fourth background image; An unlabeled image training sample pair is constructed according to the target segmentation area, the third background image and the fourth background image.

5. The method according to claim 4, characterized in that Determining whether the segmented area is a foreground area according to the background area includes: Calculate the pixel area intersection-over-union ratio of the segmented area and the background area; Determining whether the segmented area is a foreground area according to the pixel area intersection-and-combination ratio; If the pixel area intersection-over-union ratio is less than a preset threshold, the segmented area is determined to be a foreground area.

6. The method according to claim 4, characterized in that The labeled image training sample pairs include unlabeled positive sample pairs and unlabeled negative sample pairs; constructing unlabeled image training sample pairs according to the target segmentation area, the third background image and the fourth background image, including: Performing data enhancement processing on the target segmentation region to obtain an enhanced target segmentation region; Randomly pasting the enhanced target segmented area to the third background image to obtain an enhanced third background image; Randomly pasting the enhanced target segmented area to the fourth background image to obtain an enhanced fourth background image; Constructing an unlabeled positive sample pair according to the enhanced third background image and the enhanced fourth background image; Randomly acquiring a random background area as a non-foreground area from the segmented area; Randomly pasting the random background area to the fourth background image to obtain a random fourth background image; An unlabeled negative sample pair is constructed according to the enhanced third background image and the random fourth background image.

7. The method according to claim 1, characterized in that The labeled image training sample pair and the unlabeled image training sample pair are used to train the target detection model to be trained to obtain a pre-trained model, including: Inputting the labeled image training sample pair and the unlabeled image training sample into the target detection model to be trained to obtain a feature map; Searching for features corresponding to the target area from the feature map according to the training image; The features corresponding to the labeled image training samples are compared and learned, and the features corresponding to the unlabeled image training samples are compared and learned to obtain a pre-trained model.

8. The method according to claim 7, characterized in that Searching for features corresponding to the target area from the feature map according to the training image includes: Divide the corresponding training image into a plurality of regions having the same resolution as the feature map; Taking the center point of the region as an anchor point; Calculate the distance from the center point of the target bounding box in the training image to the anchor point; The position of the anchor point closest to the training image in the area divided by the training image is used as the target position of the target area in the feature map; The feature corresponding to the target position in the feature map is used as the feature corresponding to the target area.

9. The method according to claim 3, characterized in that: The data enhancement processing at least includes rotation and translation, random cropping, color change, blurring, random noise addition and object randomization.

10. The method according to claim 1, characterized in that The method further comprises: Collecting training images and background data in a source scene; wherein the training image includes a target area to be targeted, and the background data does not include the target area to be targeted; Dividing the training image into to-be-labeled data and unlabeled data according to a preset ratio; The labeled data is used to label the data to be labeled; wherein the labeled data includes a target bounding box and a category.

11. The method according to claim 1, characterized in that: The method further comprises: The image to be detected is input into the trained target detection model to obtain the category and position of the target in the image to be detected.

12. The method according to claim 1, characterized in that The labeled data is less than the unlabeled data.

13. A target detection model training device, characterized in that: The device comprises: A first acquisition module is used to acquire a training image and a background image corresponding to a source scene; wherein the training image includes annotated data and unannotated data, the training image includes a target area, and the background image does not include the target area; A first construction module, configured to construct a labeled image training sample pair according to the labeled data and the background image; A second construction module is used to construct an unlabeled image training sample pair according to the unlabeled data and the background image; A pre-training module, used to train the target detection model to be trained by using the labeled image training sample pair and the unlabeled image training sample pair to obtain a pre-training model; A second acquisition module is used to acquire a target background image of a target scene and migrate the target area to the target background image; An adjustment module is used to adjust the pre-trained model by adopting the target background image of the migrated target area, and use the adjusted pre-trained model as a trained target detection model to detect the target in the target scene according to the adjusted target detection model.

14. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 12 when executing the program stored in the memory.

15. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 12.

Images