Mask-based automatic generation and hybrid feature enhancement for occluded object detection

Abstract

The application relates to the technical field of target detection, in particular to a method for occluded target detection based on mask automatic generation and mixed feature enhancement, which comprises the following steps: constructing an occluded sample library, wherein the samples in the occluded sample library comprise semantic information; using the occluded sample library and a mask automatic generation algorithm to perform data preprocessing on pictures in a training set to obtain a preprocessed training set, wherein the training set comprises a plurality of first pictures, the preprocessed training set comprises a plurality of second pictures, the first pictures are in one-to-one correspondence with the second pictures; inputting the first pictures and the second pictures into a first target detection model to perform training and obtain a second target detection model. By constructing the occluded sample library comprising semantic information, the method effectively solves the problems of low occluded generation quality and lack of complete semantic information in the prior art. The data pairs of the first images and the second images are formed, the first target detection model is trained by using the data pairs, and the precision of the model in general occluded target detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a method for occluded target detection based on automatic generation of masks and mixed feature enhancement. BACKGROUND

[0002] Target detection is a research problem in the field of computer vision that has attracted much attention, and has wide application prospects in various systems such as unmanned systems, security monitoring, and military equipment. However, in actual scenarios, the target detection system will be affected by various factors, resulting in a decrease in detection accuracy. The presence of occlusion is an important factor affecting accuracy, which can cause the loss of part of the information of the target, including appearance and texture, making it difficult for the target detection system to identify such objects, resulting in missed detection or false detection.

[0003] Existing methods can simulate the occlusion situation by adding noise to the original image to prevent overfitting of the convolutional neural network and enhance the robustness of the model to occlusion. For example, random erasing, that is, covering the original image by a probability matrix and randomly assigning pixels to the covered area. In addition, there are methods of image enhancement by randomly cutting and pasting occlusion blocks between training images and methods of directly adding occlusion images to the detection box of the target to be identified. Existing methods also divide the target by using the structural information of the target and detect by using the visible part, and infer the occluded target according to the overall score through a voting mechanism.

[0004] However, these methods have the following problems: the randomly generated occlusion region is quite different from the occlusion situation that occurs in real scenarios, affecting the generalization performance of the model. Directly constructing a dataset containing occlusions before training loses part of the information of the original dataset, damaging the performance of the model. The method of directly adding occlusions in the target detection box ignores the semantic information that may exist between the occlusion and the occluded object, which can mislead the model to some extent during training. SUMMARY

[0005] (I) Invention purpose

[0006] The purpose of the present application is to provide an occluded target detection method that can improve the accuracy of target detection and take into account the semantic information between the occlusion and the occluded object.

[0007] (II) Technical solution

[0008] To solve the above problems, the present application provides an occluded target detection method based on automatic generation of masks and mixed feature enhancement, comprising:

[0009] S100: Construct an occlusion template library, the samples in the occlusion template library including semantic information;

[0010] S200: Label the dataset to obtain the labeled dataset, and divide the labeled dataset into training set, validation set and test set;

[0011] S300: Using the occlusion template library and the automatic mask generation algorithm, perform data preprocessing on the images in the training set to obtain a preprocessed training set. The training set includes multiple first images, and the preprocessed training set includes multiple second images. The first images and the second images correspond one-to-one.

[0012] S400: Constructing a first-target detection model based on hybrid feature enhancement;

[0013] S500: Input the first image and the second image into the first object detection model for training to obtain the second object detection model;

[0014] S600: Using the test set, the second target detection model is used for detection, and the detected second target detection model is used for target detection.

[0015] In another aspect of the present invention, preferably, the semantic information includes the target category and the positional relationship between the occluder and the occluded object;

[0016] The positional relationship between the occluding object and the occluded object includes strong semantic occlusion relationship and weak semantic occlusion relationship;

[0017] The strong semantic occlusion relationship indicates that the positional relationship between the occluding object and the occluded object is fixed;

[0018] The weak semantic occlusion relationship indicates that the positional relationship between the occluding object and the occluded object is not fixed;

[0019] The annotations include the target detection box and the target category.

[0020] In another aspect of the present invention, preferably,

[0021] The first target detection model includes a backbone network, an invisible feature recovery module, and a visible feature extraction and enhancement module;

[0022] The step S400 of constructing the first target detection model based on hybrid feature enhancement includes:

[0023] S401: Input the first image into the backbone network for feature extraction to obtain the first feature;

[0024] S402: Input the second image into the backbone network and the invisible feature recovery module for feature extraction to obtain the second feature;

[0025] S403: The first feature and the second feature are fused to obtain the third feature;

[0026] S404: inputting the third feature into a visible feature extraction and enhancement module for feature extraction to obtain a fourth feature;

[0027] S405: performing target detection according to the fourth feature.

[0028] In another aspect of the present application, preferably, the step S300 includes: performing data preprocessing on the pictures in the training set by using the occlusion template library and a mask automatic generation algorithm to obtain a preprocessed training set, and the data preprocessing includes:

[0029] S301: obtaining a target detection frame of a predetermined proportion of each picture in the first pictures;

[0030] S302: selecting an occlusion template from the occlusion template library and performing cropping on the occlusion template according to a minimum enclosing rectangle frame of the occlusion template;

[0031] S303: determining a position relationship between an occluder and an occluded object according to a target category of the obtained target detection frame and a target category of the cropped occlusion template;

[0032] S304: generating a corresponding second picture based on the mask automatic generation algorithm according to the determined position relationship between the occluder and the occluded object;

[0033] S305: and supplementing a label of the occlusion template to a corresponding picture in the first pictures.

[0034] In another aspect of the present application, preferably, the step S500 includes: inputting the first pictures and the second pictures into the first target detection model to perform training to obtain a second target detection model, and the training includes:

[0035] S501: configuring a training environment of the first target detection model;

[0036] S502: configuring a training parameter of the first target detection model;

[0037] S503: constructing a loss function and performing back propagation;

[0038] S504: testing a verification set by using the first target detection model in the training, and ending the training when the loss function tends to be constant.

[0039] In another aspect of the present application, preferably, the visible feature extraction and enhancement module sequentially includes a multiscale receptive field deformable convolution submodule, a progressive feature fusion network submodule and a context semantic enhancement submodule.

[0040] The multiscale receptive field deformable convolution submodule comprises a global attention layer, a plurality of convolution kernels of different sizes and an adaptive weighted dynamic deformable convolution.

[0041] In another aspect of the present application, preferably, the adaptive weighted dynamic deformable convolution is expressed by the following formula:

[0042]

[0043] wherein x and y respectively represent input features and output features, O represents a learned offset direction, T represents a learned offset weight, m is a position of a convolution kernel, m0 represents a point in a feature map, and w represents a weight of a convolution kernel.

[0044] In another aspect of the present application, preferably, the progressive feature fusion network submodule is expressed by the following formula:

[0045]

[0046] wherein, represents features after multiscale adaptive spatial fusion, x ij n-＞l represents a feature vector from Level n to Level l, represents an adaptive spatial fusion coefficient of Level l.

[0047] In another aspect of the present application, preferably, the context semantic enhancement submodule is expressed by the following formula:

[0048]

[0049] wherein AdaptiveConv1D represents an adaptive one-dimensional convolution, the convolution kernel size is k; GAP represents global average pooling, C represents the number of channels of an input feature map, and odd represents taking an odd number downward, F in represents an input feature of the context semantic enhancement submodule, F out represents an output feature of the context semantic enhancement submodule, and sigmoid represents an activation function.

[0050] In another aspect of the present application, preferably, the loss function comprises a target confidence and classification loss function, a prediction box regression loss function and a feature comparison loss function.

[0051] wherein the target confidence and classification loss function comprises a cross-entropy loss function; the prediction box regression loss function comprises a Wise-IoU loss function; and the feature comparison loss function comprises a cosine similarity loss function.

[0052] (Three) beneficial effects

[0053] The above technical solutions of the present application have the following beneficial technical effects:

[0054] The present application effectively solves the problems of low generation quality and lack of complete semantic information in the prior art by constructing a semantic information containing occlusion template library, pre-processing the training set using the occlusion template library and a mask automatic generation algorithm, forming a data pair of the first image and the second image, training the first target detection model using the data pair, and improving the precision of the model in detecting general occlusion targets. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The overall flowchart of one embodiment of the present application;

[0056] Figure 2 The flowchart of data preprocessing of one embodiment of the present application;

[0057] Figure 3 The structure diagram of the first target detection model of one embodiment of the present application;

[0058] Figure 4 The structure diagram of the invisible feature recovery module of one embodiment of the present application;

[0059] Figure 5 The structure diagram of the visible feature extraction and enhancement module of one embodiment of the present application;

[0060] Figure 6 The structure diagram of the multiscale receptive field deformable convolution submodule of one embodiment of the present application;

[0061] Figure 7 The structure diagram of the context semantic enhancement submodule of one embodiment of the present application;

[0062] Figure 8 The training flowchart of one embodiment of the present application. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of the present application clearer, further detailed description will be given below in conjunction with specific embodiments and reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present application. In addition, in the following description, the description of well-known structures and techniques is omitted to avoid unnecessary confusion of the concept of the present application.

[0064] Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0065] In the description of the present application, it should be noted that the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0066] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0067] Embodiment one

[0068] A method for automatically generating and mixing feature-enhanced occluded target detection based on a mask, Figure 1 The overall flowchart of one embodiment of the present application is shown as Figure 1 As shown, it comprises:

[0069] S100: Construct an occlusion template library, the samples in the occlusion template library include semantic information;

[0070] Here, the specific way of constructing the occlusion template library is not limited, and in the present embodiment, the way of constructing the occlusion template library comprises: using a Mask-RCNN model pre-trained on a COCO dataset, inputting a test set picture of the COCO dataset, and outputting a target mask according to the target category (such as a car, a person, etc.). i Then, a black background is created according to the picture size, and an occlusion template is pasted on the black background according to the original picture and the mask of the target to generate an occlusion template, and the construction of the occlusion template library is completed according to the above steps.

[0071] subimage occ =subimage⊙mask i

[0072] Wherein, subimage is a sub-image cut according to the target bounding box on the original picture, subimage occ is an occlusion template after occlusion enhancement.

[0073] Further, the semantic information herein includes a target category and a position relationship of an occluder and an occluded object; the position relationship of the occluder and the occluded object includes a strong semantic occlusion relationship and a weak semantic occlusion relationship; the strong semantic occlusion relationship represents a fixed position relationship of the occluder and the occluded object; the weak semantic occlusion relationship represents an unfixed position relationship of the occluder and the occluded object; and the strong semantic occlusion relationship (such as a vehicle) and the weak semantic occlusion relationship set (such as a camouflage net). The strong semantic occlusion relationship refers to that the occluder and the occluded object generally have a strong position relationship in the real world, for example, a car is generally occluded at the bottom of the target, and the occlusion position is located in the lower half of the target detection box to be occluded. The weak semantic occlusion relationship refers to that the position relationship between the occluder and the occluded object is weak, and the occlusion position can be located at any position of the target detection box to be occluded.

[0074] S200: labeling a data set to obtain a labeled data set, and dividing the labeled data set into a training set, a validation set and a test set; the labeling includes a target detection box and a target category; after the data set is labeled, the labeled data set is divided into the training set, the validation set and the test set according to a ratio of 7:1:2.

[0075] S300: performing data preprocessing on pictures in the training set by using the occlusion template library and a mask automatic generation algorithm to obtain a preprocessed training set, the training set including a plurality of first pictures, and the preprocessed training set including a plurality of second pictures; the first picture and the second picture correspond to each other in a one-to-one manner; Figure 2 A flowchart of data preprocessing of one embodiment of the application is shown as shown in the figure, and includes: Figure 2

[0076] S301: obtaining a target detection box of a predetermined proportion in each picture in the first picture; the obtained target detection box information includes coordinate information and a target category of the target detection box;

[0077] S302: selecting an occlusion template in the occlusion template library, and cropping the occlusion template according to a minimum enclosing rectangle frame of the occlusion template; the selected occlusion template includes a target category of the occlusion template;

[0078] S303: determining a position relationship of an occluder and an occluded object between the target category of the obtained target detection box and the target category of the cropped occlusion template;

[0079] S304: generating a corresponding second picture based on the mask automatic generation algorithm according to the determined position relationship of the occluder and the occluded object;

[0080] ​If the position relationship between the occluder and the occluded object is a strong semantic occlusion relationship, the pasting is performed according to a strong semantic containing relationship, and if the position relationship between the occluder and the occluded object is a weak semantic occlusion relationship, the pasting is performed at an arbitrary position in the target detection frame;

[0081] S305: The label of the occlusion template is supplemented into the corresponding picture in the first picture.

[0082] The position determination is performed according to the following rules:

[0083]

[0084] wherein W, H are the width and height of the obtained target detection frame, and w0, h0 are the width and height of the occlusion template.

[0085] The mask automatic generation algorithm includes the following steps:

[0086] The selected occlusion template is scaled according to the size of the obtained target detection frame, and is changed to 20%-60% of the size of the target detection frame; and then the scaling is pasted on the obtained target detection frame;

[0087] x op =ψ(x ob ,ε)+M⊙ρ(x i )

[0088] wherein x op represents the second image, ψ(·,·) is a resizing operation, M represents an occlusion binary mask sampled from the resized occlusion x ob , and ρ(·) represents a clipping operation; x i represents the first image; x i and x op form a sample pair (x i , x op ).

[0089] S400: A first target detection model based on mixed feature enhancement is constructed; Figure 3 Fig. 1 shows a first target detection model structure schematic diagram of one embodiment of the present application, which includes the following:

[0090] The first target detection model includes a backbone network, an invisible feature recovery module, and a visible feature extraction and enhancement module.

[0091] S401: The first picture is input into the backbone network to extract features, and a first feature is obtained;

[0092] In this embodiment, the backbone network can adopt the CSPDarknet53 structure of YOLOX. The specific operation of the Focus module is to extract pixels in the row and column directions of an image by one pixel, form a new feature layer, and recombine each image into 4 feature layers. Then, the 4 feature layers are stacked, and the input channel is expanded to 4 times. The stacked feature layer changes from the original 3 channels to 12 channels. The CSPDarknet includes a plurality of residual network structures (Resblock body). The residual network is divided into two branches. The main branch performs a 1x1 convolution and a 3x3 convolution, and the residual edge part is not processed, which is equivalent to directly combining the input and output of the main branch. SPP (Spatial Pyramid Pooling) is used in the CSPDarknet. Different MaxPool kernel sizes are used for feature extraction to improve the receptive field of the network.

[0093] S402: inputting the second picture into the backbone network and the invisible feature recovery module for feature extraction to obtain a second feature, Figure 4 Fig. 2 shows a schematic diagram of the invisible feature recovery module of one embodiment of the present application. Figure 4 As shown in the figure, a lightweight invisible feature recovery module is designed by using two layers of depth separable convolution and an efficient channel attention module in this embodiment, which guarantees the feature extraction and generation ability of the model while maintaining the efficiency of the model.

[0094] F r =f2(F ori )

[0095] wherein F r is the second feature, and F ori is the first feature.

[0096] S403: performing fusion processing on the first feature and the second feature to obtain a third feature; the third feature is calculated by the following formula:

[0097]

[0098] wherein F r is the second feature, F ori is the first feature, and F is the third feature.

[0099] S404: inputting the third feature into the visible feature extraction and enhancement module for feature extraction to obtain a fourth feature; wherein, Figure 5 Fig. 3 shows a schematic diagram of the visible feature extraction and enhancement module of one embodiment of the present application. Figure 5As shown, the feature extraction and enhancement module sequentially includes a multiscale receptive field deformable convolution submodule, a progressive feature fusion network submodule and a context semantic enhancement submodule.

[0100] Figure 6 A schematic diagram of the multiscale receptive field deformable convolution submodule of one embodiment of the application is shown in FIG. 2. Figure 6 As shown, the multiscale receptive field deformable convolution submodule includes a global attention layer, a plurality of convolution kernels of different sizes and an adaptive weighted dynamic deformable convolution.

[0101] The convolution kernel sizes are 3x3, 5x5 and 7x7 respectively to obtain receptive fields of different scales, and the number of convolution kernels is 1 / 2, 1 / 4 and 1 / 4 of the input feature map channel respectively, and the channel attention information is used as the weight for channel dimension feature splicing.

[0102] The adaptive weighted dynamic deformable convolution is represented by the following formula:

[0103]

[0104] wherein x and y represent input features and output features respectively, O represents a learned offset direction, T represents a learned offset weight, m is the position of the convolution kernel, m0 represents a point in the feature map, and w represents the weight of the convolution kernel. The feature map containing the offset direction and the weight is obtained through the adaptive weighted dynamic deformable convolution. Specifically, the feature map has 3C channels, of which 2C channels represent the learned offset direction O, i.e. the direction (a, b), and the remaining C channels represent the learned offset weight T for updating the input feature map. The generation of the offset weight is inspired by the spatial attention mechanism, so that the network can not only learn the direction of various deformations, but also learn the influence of different deformations on the result. Finally, the learned offset direction and offset weight are used to update the input feature map, so as to maximize the extraction of available information in the feature map that is helpful for target detection.

[0105] The progressive feature fusion network submodule is represented by the following formula:

[0106]

[0107] wherein, represents the feature after multiscale adaptive spatial fusion, x ij n-＞l represents the feature vector from Level n to Level l, represents the adaptive spatial fusion coefficient of Level l.

[0108] In order to minimize the loss of information of feature maps of different resolution layers in the fusion process, an incremental feature fusion method is adopted, and new shallow feature maps are constantly transmitted into the feature fusion network through multi-scale feature fusion. In order to filter conflicting information in the fusion process, reduce the inconsistency between different scale features, adaptively select beneficial information in multi-scale fusion, avoid information confusion caused by simple element addition, and improve the effectiveness of feature fusion. Specifically, in the initial stage of the incremental feature pyramid network, two low-level features are first fused, higher-level features are fused in the subsequent stage, and top-level features are added to participate in the fusion in the final stage. Take the latter half stage as an example, as shown in FIG. 3, wherein the horizontal arrow represents direct connection input, the upward arrow represents different down-sampling ratios, and the downward arrow represents different up-sampling ratios. Figure 5

[0109] In order to supplement the semantic information lost in the feature fusion process and extract the relationship between the occluded target and other instances in the scene, the fused features are passed through a context semantic enhancement sub-module, Figure 7 The schematic diagram of the context semantic enhancement sub-module of one embodiment of the present application is shown in FIG. 4, as shown in FIG. 4, the context semantic enhancement sub-module is represented by the following formula: Figure 7

[0110]

[0111] Wherein, AdaptiveConv1D represents adaptive one-dimensional convolution, the convolution kernel size is k; GAP represents global average pooling, C represents the channel number of the input feature map, and odd represents k taking odd numbers downward, F in represents the input feature of the context semantic enhancement sub-module, F out represents the output feature of the context semantic enhancement sub-module, and sigmoid represents an activation function.

[0112] S405: Target detection is performed according to the fourth feature.

[0113] Figure 8 The schematic diagram of the training process of one embodiment of the present application is shown in FIG. 5.

[0114] S500: The first picture and the second picture are input into the first target detection model for training to obtain a second target detection model, including:

[0115] S501: The training environment of the first target detection model is configured;

[0116] S502: The training parameters of the first target detection model are configured;

[0117] S503: A loss function is constructed, and back propagation is performed;​​

[0118] S504: Test the validation set using the first target detection model in training, and end the training when the loss function tends to be constant.

[0119] Wherein, the loss function includes target confidence and classification loss function, prediction box regression loss function and feature comparison loss function;

[0120] Wherein, the target confidence and classification loss function includes cross entropy loss function; the prediction box regression loss function includes Wise-IoU loss function; the feature comparison loss function includes cosine similarity loss function.

[0121] In this embodiment, the mathematical expression of the classification confidence loss function is as follows:

[0122]

[0123] Where n represents the number of samples, t i represents the real label, p i represents the probability of the model predicting positive class.

[0124] The mathematical expression of the regression box loss function is as follows:

[0125] L WIoUv1 = R WIoU L IoU

[0126]

[0127] Where R WIoU is a weighting function, (x,y),(x gt ,y gt ) respectively represent the predicted upper left corner coordinates and real label coordinates, is the size of the minimum bounding box, in order to prevent R WIoU from producing gradient that hinders convergence, will be separated from the calculation graph (superscript * indicates this operation).

[0128] The mathematical expression of the cosine similarity loss function is as follows:

[0129]

[0130] L cosine = 1-CosineSimilarity(F ori ,F r )

[0131] Where A and B are two vectors, F ori represents the original feature, and F r represents the recovered feature.

[0132] S600: using the test set, the second target detection model is detected, and the second target detection model after detection is used for target detection. It includes: inputting the test set picture into the second target detection model, first, the second target detection model will adjust the picture to be detected to a uniform size, if the size of the picture to be detected is smaller than the set size, the picture to be detected is added with a gray bar around the edge, if it is larger than the set size, the picture to be detected is compressed. It should be noted that in the test stage, the sample pair (first image, second image) is replaced as (first image, first image) as input into the model. The output of the model will get three sizes of detection results, whose scales are (20x20), (40x40) and (80x80) respectively. Assuming that there are 10 categories to be detected, the trained second target detection model will express the output result as a two-dimensional vector (8400, 15). The 15 includes the target category number 10, the position parameters (x, y, w, h) of the detection frame and one confidence score. Then the soft-DIoU-NMS algorithm is used to suppress redundant detection frames, and the remaining detection frames are the final detection results.

[0133] The present application solves the problems of low quality of existing occlusion generation and lack of complete semantic information by constructing an occlusion template library containing semantic information, preprocessing the training set using the occlusion template library and mask automatic generation algorithm, and introducing occlusion annotations using additional occlusion strategies.

[0134] The present application proposes a method of combining invisible feature restoration and visible feature enhancement to generate hybrid features more robust to occlusion. In the aspect of invisible feature restoration, the present application proposes a feature restoration module containing a contrast learning loss; a set of sample pairs is formed by the first image and the second image after occlusion enhancement; the first image and the second image after occlusion enhancement are respectively input into a weight-shared backbone network, defined as the main branch and the occlusion branch; in the occlusion branch, the feature map of the second image is input into the invisible feature restoration module to obtain the generated second feature; and the third feature is obtained by fusing the first feature of the first image in the main branch and entering the subsequent module. In the aspect of visible feature extraction enhancement, multi-scale receptive field deformable convolution submodules are adaptively used to extract visible features according to feature maps from different levels, and then high-efficiency context semantic enhancement submodules are added to the improved progressive feature fusion network submodule to guide the model to adaptively learn visible features, thereby improving the detection performance of occluded targets and improving the precision of the model in detecting general occluded targets.

[0135] It should be understood that the foregoing detailed description of the application, rather than limiting the application, is provided as an illustrative example of the application, with the scope of the application being indicated by the appended claims and equivalents thereof. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the application should be included in the scope of the protection of the application. In addition, the appended claims of the present application are intended to cover all changes and modifications falling within the scope and boundary of the appended claims, or equivalents of such scope and boundary.

[0136] The present application has been described above with reference to the embodiments. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present application. The scope of the present application is defined by the appended claims and equivalents thereof. Various substitutions and modifications can be made to the embodiments by those skilled in the art without departing from the scope of the present application, and such substitutions and modifications should fall within the scope of the present application.

[0137] Although the embodiments of the present application have been described in detail, it should be understood that various changes, substitutions and alterations can be made to the embodiments without departing from the spirit and scope of the present application.

[0138] Obviously, the above-described embodiments are merely for illustrative purposes, and are not intended to limit the embodiments. Based on the above description, other different forms of changes or modifications can be made by those skilled in the art. Here, it is not necessary or possible to exhaust all the embodiments. The obvious changes or modifications derived therefrom are still within the scope of protection of the present application.

Claims

1. A method for automatically generating and mixing feature enhanced occluded object detection based on masks, characterized in that, The method comprises the following steps: S100: constructing a shielding template library, wherein samples in the shielding template library comprise semantic information; S200: labeling a data set to obtain a labeled data set, and dividing the labeled data set into a training set, a validation set and a test set; S300: using the shielding template library and a mask automatic generation algorithm to perform data preprocessing on pictures in the training set to obtain a preprocessed training set, wherein the training set comprises a plurality of first pictures, and the preprocessed training set comprises a plurality of second pictures; the first pictures and the second pictures correspond to each other in a one-to-one manner; S400: constructing a first target detection model based on mixed feature enhancement; S500: inputting the first pictures and the second pictures into the first target detection model for training to obtain a second target detection model; S600: using the test set to detect the second target detection model, and using the detected second target detection model to perform target detection; The semantic information comprises a target category and a position relationship between a shielding object and a shielded object. The position relationship between the shielding object and the shielded object comprises a strong semantic shielding relationship and a weak semantic shielding relationship. The strong semantic shielding relationship represents a fixed position relationship between the shielding object and the shielded object. The weak semantic shielding relationship represents an unfixed position relationship between the shielding object and the shielded object. The labeling comprises a target detection frame and a target category. The first target detection model comprises a backbone network, an invisible feature recovery module and a visible feature extraction and enhancement module. The step S400 of constructing the first target detection model based on mixed feature enhancement comprises: S401: inputting the first pictures into the backbone network for feature extraction to obtain first features; S402: inputting the second pictures into the backbone network and the invisible feature recovery module for feature extraction to obtain second features; S403: performing fusion processing on the first features and the second features to obtain third features; S404: inputting the third features into the visible feature extraction and enhancement module for feature extraction to obtain fourth features; S405: performing target detection according to the fourth features; The visible feature extraction and enhancement module comprises, in sequence, a multiscale receptive field deformable convolution submodule, a progressive feature fusion network submodule and a context semantic enhancement submodule. The multiscale receptive field deformable convolution submodule comprises a global attention layer, a plurality of convolution kernels of different sizes and an adaptive weighted dynamic deformable convolution. The adaptive weighted dynamic deformable convolution is represented by the following formula: wherein, and represent input and output features, respectively, represents a learned shift direction, represents a learned shift weight, is a position of a convolution kernel, represents a point in a feature map, represents a weight of a convolution kernel; The progressive feature fusion network submodule is represented by the following formula: wherein, represents the feature after multi-scale adaptive spatial fusion, represents the feature vector from Level to Level , represents the adaptive spatial fusion coefficient of Level .

2. The method of claim 1, wherein, The step S300 of using the shielding template library and the mask automatic generation algorithm to perform data preprocessing on the pictures in the training set comprises: S301: obtaining a target detection frame of a predetermined proportion in each picture in the first pictures; S302: selecting a shielding template from the shielding template library, and cropping the shielding template according to a minimum enclosing rectangle frame of the shielding template; S303: According to the target category of the obtained target detection frame and the target category of the cropped occlusion template, the position relationship between the occluder and the occluded object is determined; S304: According to the determined position relationship between the occluder and the occluded object, a corresponding second picture is generated based on a mask automatic generation algorithm; S305: And the label of the occlusion template is supplemented into the corresponding picture in the first picture.

3. The method of claim 1, wherein, The step S500 includes: S501: configuring the training environment of the first target detection model; S502: configuring the training parameters of the first target detection model; S503: constructing a loss function and performing back propagation; S504: using the first target detection model in training to test the verification set, and ending the training when the loss function tends to be constant.

4. The method of claim 1, wherein, The context semantic enhancement sub-module is represented by the following formula: wherein, denotes adaptive one-dimensional convolution, the convolution kernel size is k; GAP denotes global average pooling, denotes the number of channels of the input feature map, denotes k is taken as an odd number downward, F in denotes the input feature of the context semantic enhancement sub-module, F out denotes the output feature of the context semantic enhancement sub-module, sigmoid denotes an activation function.

5. The method of claim 3, wherein, The loss function includes target confidence and classification loss function, prediction box regression loss function and feature comparison loss function; Wherein the target confidence and classification loss function includes cross entropy loss function; the prediction box regression loss function includes Wise-IoU loss function; the feature comparison loss function includes cosine similarity loss function.

Images