Shielding target detection method based on automatic mask generation and mixed feature enhancement

Abstract

The invention relates to the technical field of target detection, in particular to a shielding target detection method based on automatic mask generation and mixed feature enhancement, which comprises the following steps: constructing a shielding template library, samples in the shielding template library comprising semantic information; performing data preprocessing on the pictures in the training set by using a shielding template library and a mask automatic generation algorithm to obtain a preprocessed training set, the training set comprising a plurality of first pictures, and the preprocessed training set comprising a plurality of second pictures; the first pictures and the second pictures are in one-to-one correspondence; and inputting the first picture and the second picture into the first target detection model for training to obtain a second target detection model. According to the invention, by constructing the shielding template library containing semantic information, the problems of low shielding generation quality, lack of semantic information and the like in the prior art are effectively solved. And forming a data pair of the first image and the second image, training the first target detection model by using the data pair, and improving the precision of the model for detecting the general occlusion target.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to an occluded target detection method based on automatic mask generation and hybrid feature enhancement. Background Art

[0002] Object detection is a research problem that has received much attention in the field of computer vision. It has broad application prospects in various systems such as unmanned systems, security monitoring, and military equipment. However, in actual scenarios, the object detection system will be affected by various factors, resulting in a decrease in detection accuracy. Among them, the existence of occlusion is an important factor affecting accuracy. It will cause the loss of part of the target information including appearance and texture, making it difficult for the object detection system to identify such objects, resulting in missed detection or false detection.

[0003] Existing methods can simulate occlusion by adding noise to the original image, prevent overfitting of the convolutional neural network, and enhance the robustness of the model to occlusion. For example, random erasing is to cover the original image with a probability matrix and assign random pixel values ​​to the covered area. In addition, there are methods for image enhancement by randomly cutting and pasting occlusion blocks between training images, and methods for directly adding the occluded object image to the detection frame of the target to be identified. Existing methods also divide the target by using the structural information of the target, and use the visible part for detection, and use a voting mechanism to infer the occluded target based on the overall score.

[0004] However, these methods have the following problems: the randomly generated occlusion areas are quite different from the occlusions that occur in real scenes, which affects the generalization performance of the model. Directly constructing a dataset containing occlusions before training loses some information of the original dataset, which impairs the performance of the model. The method of directly adding occlusions to the target detection box ignores the semantic information that may exist between the occlusion and the occluded object, which will mislead the model to a certain extent during training. Summary of the invention

[0005] (I) Purpose of the invention

[0006] The object of the present invention is to provide an occluded target detection method which can improve the accuracy of target detection and take into account the semantic information between the occluder and the occluded object.

[0007] (II) Technical solution

[0008] To solve the above problems, the present invention provides an occluded target detection method based on automatic mask generation and hybrid feature enhancement, comprising:

[0009] S100: constructing an occlusion template library, wherein samples in the occlusion template library include semantic information;

[0010] S200: labeling the data set to obtain a labeled data set, and dividing the labeled data set into a training set, a validation set, and a test set;

[0011] S300: using the occlusion template library and the automatic mask generation algorithm, performing data preprocessing on the images in the training set to obtain a preprocessed training set, wherein the training set includes a plurality of first images, and the preprocessed training set includes a plurality of second images; the first images correspond to the second images one by one;

[0012] S400: constructing a first target detection model based on hybrid feature enhancement;

[0013] S500: Inputting the first image and the second image into the first object detection model for training to obtain a second object detection model;

[0014] S600: Using the test set, the second target detection model is tested, and the tested second target detection model is used to perform target detection.

[0015] In another aspect of the present invention, preferably, the semantic information includes the positional relationship between the target category and the occluder and the occluded object;

[0016] The positional relationship between the occluder and the occluded object includes a strong semantic occlusion relationship and a weak semantic occlusion relationship;

[0017] The strong semantic occlusion relationship indicates that the positional relationship between the occluder and the occluded object is fixed;

[0018] The weak semantic occlusion relationship indicates that the positional relationship between the occluder and the occluded object is not fixed;

[0019] The annotation includes a target detection box and a target category.

[0020] In another aspect of the present invention, preferably,

[0021] The first target detection model includes a backbone network, an invisible feature recovery module and a visible feature extraction and enhancement module;

[0022] The step S400 of constructing a first target detection model based on hybrid feature enhancement includes:

[0023] S401: Inputting the first image into the backbone network to extract features and obtain a first feature;

[0024] S402: Inputting the second image into the backbone network and the invisible feature recovery module to extract features and obtain second features;

[0025] S403: fusing the first feature and the second feature to obtain a third feature;

[0026] S404: Inputting the third feature into a visible feature extraction and enhancement module for feature extraction to obtain a fourth feature;

[0027] S405: Perform target detection according to the fourth feature.

[0028] In another aspect of the present invention, preferably, the step S300: using the occlusion template library and the automatic mask generation algorithm to perform data preprocessing on the images in the training set, and obtaining the preprocessed training set comprises:

[0029] S301: Obtaining a target detection frame of a predetermined proportion for each image in a first image;

[0030] S302: selecting an occlusion template from the occlusion template library, and cutting the occlusion template according to the minimum enclosing rectangular frame of the occlusion template;

[0031] S303: Determine the positional relationship between the occluder and the occluded object according to the target category of the acquired target detection frame and the target category of the cropped occlusion template;

[0032] S304: Generate a corresponding second image based on the determined positional relationship between the occluder and the occluded object and based on an automatic mask generation algorithm;

[0033] S305: Add the label of the occlusion template to the corresponding picture in the first picture.

[0034] In another aspect of the present invention, preferably, in step S500, inputting the first image and the second image into the first object detection model for training, and obtaining the second object detection model includes:

[0035] S501: Configuring a training environment for a first object detection model;

[0036] S502: Configure first target detection model training parameters;

[0037] S503: Construct a loss function and perform back propagation;

[0038] S504: Use the first target detection model in training to test the validation set, and terminate the training when the loss function tends to be unchanged.

[0039] In another aspect of the present invention, preferably, the visible feature extraction and enhancement module includes, in sequence, a multi-scale receptive field deformable convolution submodule, a progressive feature fusion network submodule, and a contextual semantic enhancement submodule;

[0040] The multi-scale receptive field deformable convolution submodule includes: a global attention layer, a plurality of convolution kernels of different sizes and an adaptively weighted dynamic deformable convolution.

[0041] In another aspect of the present invention, preferably, the adaptive weighted dynamic deformable convolution is expressed by the following formula:

[0042]

[0043] Among them, x and y represent the input feature and output feature respectively, O represents the learned offset direction, T represents the learned offset weight, m ​​is the position of the convolution kernel, and m 0 Represents a point in the feature map, and w represents the weight of the convolution kernel.

[0044] In another aspect of the present invention, preferably, the progressive feature fusion network submodule is expressed by the following formula:

[0045]

[0046] in, represents the features after multi-scale adaptive spatial fusion, x ij n-＞l represents the feature vector from Leveln to Levell, Represents the adaptive spatial fusion coefficient of Level l.

[0047] In another aspect of the present invention, preferably, the contextual semantic enhancement submodule is represented by the following formula:

[0048]

[0049] Among them, AdaptiveConv1D represents adaptive one-dimensional convolution, and the convolution kernel size is k; GAP represents global average pooling, C represents the number of channels of the input feature map, || odd Indicates that k is an odd number downwards, F in represents the input features of the contextual semantic enhancement submodule, F out represents the output features of the contextual semantic enhancement submodule, and sigmoid represents the activation function.

[0050] In another aspect of the present invention, preferably, the loss function includes a target confidence and classification loss function, a prediction box regression loss function, and a feature contrast loss function;

[0051] Among them, the target confidence and classification loss functions include the cross entropy loss function; the prediction box regression loss function includes the Wise-IoU loss function; the feature contrast loss function includes the cosine similarity loss function.

[0052] (III) Beneficial effects

[0053] The above technical solution of the present invention has the following beneficial technical effects:

[0054] The present invention constructs an occlusion template library containing semantic information, and uses the occlusion template library and the automatic mask generation algorithm to preprocess the training set, effectively solving the problems of low occlusion generation quality and lack of complete semantic information in the prior art. A data pair of the first image and the second image is formed, and the data pair is used to train the first target detection model to improve the accuracy of the model for detecting universal occluded targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0056] Figure 2 A flowchart of data preprocessing according to an embodiment of the present invention;

[0057] Figure 3 A schematic diagram of the structure of a first target detection model according to an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the structure of an invisible feature recovery module according to an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of the structure of a visible feature extraction and enhancement module according to an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of the structure of a multi-scale receptive field deformable convolution submodule according to an embodiment of the present invention;

[0061] Figure 7 A schematic diagram of the structure of a contextual semantic enhancement submodule according to an embodiment of the present invention;

[0062] Figure 8 The figure is a schematic diagram of a training process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0064] Obviously, the described embodiments are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] In the description of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0066] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0067] Embodiment 1

[0068] A method for occluded target detection based on automatic mask generation and hybrid feature enhancement. Figure 1 FIG. 1 shows an overall flow chart of an embodiment of the present invention, as shown in FIG. Figure 1 As shown, including:

[0069] S100: constructing an occlusion template library, wherein samples in the occlusion template library include semantic information;

[0070] The specific method of constructing the occlusion template library is not limited here. In this embodiment, the method of constructing the occlusion template library includes: using the Mask-RCNN model pre-trained on the COCO dataset, inputting the test set pictures of the COCO dataset, and outputting the target mask according to the target category (such as car, person, etc.) i Then create a black background according to the image size, and paste the template on the black background according to the mask of the original image and the target to generate an occlusion template. Follow these steps to complete the construction of the occlusion template library.

[0071] subimage occ =subimage⊙mask i

[0072] Among them, subimage is a subimage cropped from the original image according to the target bounding box. occ This is the occlusion sample after occlusion enhancement.

[0073] Furthermore, the semantic information here includes the positional relationship between the target category and the occluder and the occluded object; the positional relationship between the occluder and the occluded object includes a strong semantic occlusion relationship and a weak semantic occlusion relationship; the strong semantic occlusion relationship indicates that the positional relationship between the occluder and the occluded object is fixed; the weak semantic occlusion relationship indicates that the positional relationship between the occluder and the occluded object is not fixed; strong semantic occlusion relationships (such as vehicles) and weak semantic occlusion relationships (such as camouflage nets) are included. Among them, the strong semantic occlusion relationship means that the occluder and the occluded object generally have a strong positional relationship in the real world. For example, a car generally occludes the bottom of the target, and the occlusion position is located in the lower half of the detection frame of the target to be occluded. The weak semantic occlusion relationship means that the positional relationship between the occluder and the occluded object is weak, and the occlusion position can be located at any position in the detection frame of the target to be occluded.

[0074] S200: Labeling the data set to obtain a labeled data set, and dividing the labeled data set into a training set, a validation set, and a test set; the labeling includes a target detection box and a target category; after the data set is labeled, the labeled data set is divided into a training set, a validation set, and a test set in a ratio of 7:1:2.

[0075] S300: using the occlusion template library and the automatic mask generation algorithm, performing data preprocessing on the images in the training set to obtain a preprocessed training set, wherein the training set includes a plurality of first images, and the preprocessed training set includes a plurality of second images; the first images correspond to the second images one by one; Figure 2 A flowchart of data preprocessing according to an embodiment of the present invention is shown. Figure 2 As shown, including:

[0076] S301: Obtain a target detection frame of a predetermined proportion of each image in the first image; the information of the acquired target detection frame includes coordinate information and a target category of the target detection frame;

[0077] S302: selecting an occlusion template from the occlusion template library, and cutting the occlusion template according to the minimum enclosing rectangular frame of the occlusion template; the selected occlusion template includes the target category of the occlusion template;

[0078] S303: Determine the positional relationship between the occluder and the occluded object according to the target category of the acquired target detection frame and the target category of the cropped occlusion template;

[0079] S304: Generate a corresponding second image based on the determined positional relationship between the occluder and the occluded object and based on an automatic mask generation algorithm;

[0080] If the positional relationship between the occluder and the occluded object is a strong semantic occlusion relationship, the object is pasted according to the strong semantic inclusion relationship. If the positional relationship between the occluder and the occluded object is a weak semantic occlusion relationship, the object is pasted at any position within the target detection frame.

[0081] S305: Add the label of the occlusion template to the corresponding picture in the first picture.

[0082] The position is determined according to the following rules:

[0083]

[0084] Where W, H are the width and height of the target detection box, w 0 ,h 0 The width and height of the masking template.

[0085] The automatic mask generation algorithm here includes:

[0086] Scale the selected occlusion template according to the size of the obtained target detection frame, changing it to 20%-60% of the size of the target detection frame; and then paste it on the obtained target detection frame respectively;

[0087] x op =ψ(x ob ,ε)+M⊙ρ(x i )

[0088] where x op represents the second image, ψ(·,·) is the resizing operation, and M represents the occlusion x from the resize ob The sampled occlusion binary mask, ρ(·) represents the clipping operation, x i represents the first image; x i and x op To form a sample pair (x i ,x op ).

[0089] S400: constructing a first target detection model based on hybrid feature enhancement; Figure 3 A schematic diagram of the structure of a first target detection model according to an embodiment of the present invention is shown, comprising:

[0090] The first target detection model includes a backbone network, an invisible feature recovery module and a visible feature extraction and enhancement module;

[0091] S401: Inputting the first image into the backbone network to extract features and obtain a first feature;

[0092] In this embodiment, the backbone network can adopt the CSPDarknet53 structure of YOLOX. The specific operation of the Focus module is to extract every other pixel in the direction of rows and columns in an image to form a new feature layer. Each image can be reorganized into 4 feature layers, and then the 4 feature layers are stacked to expand the input channel by 4 times. The stacked feature layer becomes 12 channels relative to the original 3 channels. CSPDarknet contains several residual network structures (Resblock body). The residual network is divided into two branches. The trunk branch performs a 1x1 convolution and a 3x3 convolution. The residual side part does not perform any processing, which is equivalent to directly combining the input and output of the trunk. SPP (Spatial Pyramid Pooling), in CSPDarknet, MaxPool with different pooling kernel sizes is used for feature extraction to improve the receptive field of the network.

[0093] S402: Input the second image into the backbone network and the invisible feature recovery module to extract features and obtain a second feature. Figure 4 A schematic diagram of an invisible feature recovery module according to an embodiment of the present invention is shown. Figure 4 As shown, in this embodiment, two layers of deep separable convolution and an efficient channel attention module are used to design a lightweight invisible feature recovery module, which maintains the efficiency of the model while ensuring the model's feature extraction and generation capabilities.

[0094] F r =f 2 (F ori )

[0095] where F r is the second feature, F ori The first feature.

[0096] S403: The first feature and the second feature are fused to obtain a third feature; the third feature is calculated using the following formula:

[0097]

[0098] where F r is the second feature, F ori is the first feature and F is the third feature.

[0099] S404: Input the third feature into the visible feature extraction and enhancement module for feature extraction to obtain a fourth feature; wherein, Figure 5 FIG. 4 is a schematic diagram showing a visible feature extraction and enhancement module according to an embodiment of the present invention. Figure 5As shown, the visible feature extraction and enhancement module includes a multi-scale receptive field deformable convolution submodule, a progressive feature fusion network submodule and a contextual semantic enhancement submodule in sequence;

[0100] Figure 6 FIG. 4 shows a schematic diagram of a multi-scale receptive field deformable convolution submodule according to an embodiment of the present invention. Figure 6 As shown in the figure, the multi-scale receptive field deformable convolution submodule includes: a global attention layer, several convolution kernels of different sizes, and an adaptively weighted dynamic deformable convolution.

[0101] The convolution kernel sizes are 3ⅹ3, 5ⅹ5, and 7ⅹ7 to obtain receptive fields of different scales. The number of convolution kernels is 1 / 2, 1 / 4, and 1 / 4 of the input feature map channels, and channel dimension features are concatenated with channel attention information as weight.

[0102] The adaptive weighted dynamic deformable convolution is expressed using the following formula:

[0103]

[0104] Among them, x and y represent the input feature and output feature respectively, O represents the learned offset direction, T represents the learned offset weight, m ​​is the position of the convolution kernel, and m 0 Represents a point in the feature map, and w represents the weight of the convolution kernel. A feature map containing offset direction and weight is obtained through adaptive weighted dynamic deformable convolution. Specifically, the feature map has 3C channels, of which 2C channels represent the learned offset direction O, that is, the direction (a, b), and the remaining C channels represent the learned offset weight T, which is used to update the input feature map. The generation of the offset weight draws on the spatial attention mechanism, so that the network can not only learn the directions of various deformations, but also learn the impact of different deformations on the results. Finally, the learned offset direction and offset weight are used to update the input feature map to maximize the extraction of available information in the feature map that is helpful for target detection.

[0105] The progressive feature fusion network submodule is expressed using the following formula:

[0106]

[0107] in, represents the features after multi-scale adaptive spatial fusion, x ij n-＞l represents the feature vector from Leveln to Levell, Represents the adaptive spatial fusion coefficient of Level l.

[0108] In order to minimize the information lost in the fusion process of feature maps of different resolution layers, a progressive feature fusion method is adopted to continuously pass new shallow feature maps into the feature fusion network using multi-scale feature fusion. In order to filter conflicting information and reduce the inconsistency between features of different scales during the fusion process, the beneficial information in multi-scale fusion is adaptively screened out to avoid information confusion that may be caused by simple element addition, thereby improving the effectiveness of feature fusion. Specifically, in the initial stage of the progressive feature pyramid network, two low-level features are fused first, and then higher-level features are fused in subsequent stages. In the final stage, top-level features are added to participate in the fusion. Take the second half of the stage as an example to illustrate, such as Figure 5 As shown, the horizontal arrow represents a direct connection input, the upward arrow represents downsampling at different magnifications, and the downward arrow represents upsampling at different magnifications.

[0109] In order to supplement the semantic information lost in the feature fusion process and extract the relationship between the occluded object and other instances in the scene, the fused features are passed through a contextual semantic enhancement submodule. Figure 7 A schematic diagram of a contextual semantic enhancement submodule according to an embodiment of the present invention is shown. Figure 7 As shown, the contextual semantic enhancement submodule is expressed using the following formula:

[0110]

[0111] Among them, AdaptiveConv1D represents adaptive one-dimensional convolution, and the convolution kernel size is k; GAP represents global average pooling, C represents the number of channels of the input feature map, || odd Indicates that k is an odd number downwards, F in represents the input features of the contextual semantic enhancement submodule, F out represents the output features of the contextual semantic enhancement submodule, and sigmoid represents the activation function.

[0112] S405: Perform target detection according to the fourth feature.

[0113] Figure 8 The figure is a schematic diagram of a training process according to an embodiment of the present invention.

[0114] S500: Inputting the first image and the second image into the first object detection model for training to obtain a second object detection model, including:

[0115] S501: Configuring a training environment for a first object detection model;

[0116] S502: Configure first target detection model training parameters;

[0117] S503: Construct a loss function and perform back propagation;

[0118] S504: Use the first target detection model in training to test the validation set, and terminate the training when the loss function tends to be unchanged.

[0119] The loss function includes target confidence and classification loss function, prediction box regression loss function and feature comparison loss function;

[0120] Among them, the target confidence and classification loss functions include the cross entropy loss function; the prediction box regression loss function includes the Wise-IoU loss function; the feature contrast loss function includes the cosine similarity loss function.

[0121] In this embodiment, the mathematical expression of the classification confidence loss function is as follows:

[0122]

[0123] Where n represents the number of samples, t i represents the true label, p i Represents the probability that the model predicts the positive class.

[0124] The mathematical expression of the regression box loss function is as follows:

[0125] L WIoUv1 =R WIoU L IoU

[0126]

[0127] Among them, R WIoU represents the weighting function, (x,y), (x gt ,y gt ) represent the predicted upper left corner coordinates and the true label coordinates, respectively. is the size of the minimum bounding box. In order to prevent R WIoU Produces gradients that hinder convergence, Separate from the computation graph (superscript * denotes this operation).

[0128] The mathematical expression of the cosine similarity loss function is as follows:

[0129]

[0130] L cosine =1-CosineSimilarity(F ori ,F r )

[0131] Among them, A and B are two vectors, F ori represents the original feature, F r Represents the restored features.

[0132] S600: Use the test set to detect the second target detection model, and use the detected second target detection model to detect the target. It includes: inputting the test set image into the second target detection model, firstly, the second target detection model will adjust the image to be detected to a uniform size, if the size of the image to be detected is smaller than the set size, then add a gray bar around the image to be detected, if it is larger than the set size, then compress the image to be detected. It should be noted that in the test phase, the sample pair (first image, second image) is replaced by (first image, first image) as input to the model. The output of the model will obtain detection results of three sizes, whose scales are (20×20), (40×40) and (80×80) respectively. Assuming that there are 10 categories to be detected, the trained second target detection model will represent the output result as a two-dimensional vector (8400,15). The 15 includes the number of target categories 10, the position parameters of the detection box (x, y, w, h) and 1 confidence score. The soft-DIoU-NMS algorithm is then used to suppress redundant detection frames, and the retained detection frames are the final detection results.

[0133] The present invention constructs an occlusion template library containing semantic information, uses the occlusion template library and the automatic mask generation algorithm to preprocess the training set, uses additional occlusion strategies and introduces occluder annotations, and effectively solves the problems of low occlusion generation quality and lack of complete semantic information in the existing technology.

[0134] The present invention proposes a method of recovering invisible features and enhancing visible features to jointly generate hybrid features that are more robust to occlusion. In terms of invisible feature recovery, the present invention proposes a feature recovery module that includes contrastive learning loss; a first image and a second image enhanced by occlusion form a set of sample pairs; the first image and the second image enhanced by occlusion are respectively passed through a weight-sharing backbone network, which are defined as a main branch and an occlusion branch; in the occlusion branch, the feature map of the second image is input into the invisible feature recovery module to obtain the generated second feature; and the third feature is fused with the first feature of the first image in the main branch to obtain the third feature, and enter the subsequent module. In terms of visible feature extraction and enhancement, a multi-scale receptive field deformable convolution submodule is adaptively used according to feature maps from different levels to extract visible features, and then an efficient context semantic enhancement submodule is added to the improved progressive feature fusion network submodule to guide the model to adaptively learn visible features, thereby improving the detection performance of occluded targets and the accuracy of the model in detecting general occluded targets.

[0135] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

[0136] The present invention has been described above with reference to the embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, a person skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

[0137] Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.

[0138] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for detecting occluded objects based on automatic mask generation and hybrid feature enhancement, characterized in that: The following steps are involved: S100: constructing an occlusion template library, wherein samples in the occlusion template library include semantic information; S200: labeling the data set to obtain a labeled data set, and dividing the labeled data set into a training set, a validation set, and a test set; S300: using the occlusion template library and the automatic mask generation algorithm, performing data preprocessing on the images in the training set to obtain a preprocessed training set, wherein the training set includes a plurality of first images, and the preprocessed training set includes a plurality of second images; the first images correspond to the second images one by one; S400: constructing a first target detection model based on hybrid feature enhancement; S500: Inputting the first image and the second image into the first object detection model for training to obtain a second object detection model; S600: Using the test set, the second target detection model is tested, and the tested second target detection model is used to perform target detection.

2. The method according to claim 1, characterized in that The semantic information includes the positional relationship between the target category and the occluder and the occluded object; The positional relationship between the occluder and the occluded object includes a strong semantic occlusion relationship and a weak semantic occlusion relationship; The strong semantic occlusion relationship indicates that the positional relationship between the occluder and the occluded object is fixed; The weak semantic occlusion relationship indicates that the positional relationship between the occluder and the occluded object is not fixed; The annotation includes a target detection box and a target category.

3. The method according to claim 1, characterized in that The first target detection model includes a backbone network, an invisible feature recovery module and a visible feature extraction and enhancement module; The step S400 of constructing a first target detection model based on hybrid feature enhancement includes: S401: Inputting the first image into the backbone network to extract features and obtain a first feature; S402: Inputting the second image into the backbone network and the invisible feature recovery module to extract features and obtain second features; S403: fusing the first feature and the second feature to obtain a third feature; S404: Inputting the third feature into a visible feature extraction and enhancement module for feature extraction to obtain a fourth feature; S405: Perform target detection according to the fourth feature.

4. The method according to claim 2, characterized in that: The step S300: using the occlusion template library and the automatic mask generation algorithm to perform data preprocessing on the images in the training set to obtain the preprocessed training set, includes: S301: Obtaining a target detection frame of a predetermined proportion for each image in a first image; S302: selecting an occlusion template from the occlusion template library, and cutting the occlusion template according to the minimum enclosing rectangular frame of the occlusion template; S303: Determine the positional relationship between the occluder and the occluded object according to the target category of the acquired target detection frame and the target category of the cropped occlusion template; S304: Generate a corresponding second image based on the determined positional relationship between the occluder and the occluded object and based on an automatic mask generation algorithm; S305: Add the label of the occlusion template to the corresponding picture in the first picture.

5. The method according to claim 1, characterized in that In step S500, the first image and the second image are input into the first object detection model for training, and obtaining the second object detection model includes: S501: Configuring a training environment for a first object detection model; S502: Configure first target detection model training parameters; S503: Construct a loss function and perform back propagation; S504: Use the first target detection model in training to test the validation set, and terminate the training when the loss function tends to be unchanged.

6. The method according to claim 3, characterized in that The visible feature extraction and enhancement module includes a multi-scale receptive field deformable convolution submodule, a progressive feature fusion network submodule and a contextual semantic enhancement submodule in sequence; The multi-scale receptive field deformable convolution submodule includes: a global attention layer, a plurality of convolution kernels of different sizes and an adaptively weighted dynamic deformable convolution.

7. The method according to claim 6, characterized in that The adaptive weighted dynamic deformable convolution is expressed using the following formula: Among them, x and y represent the input feature and output feature respectively, O represents the learned offset direction, T represents the learned offset weight, m ​​is the position of the convolution kernel, m0 represents the point in the feature map, and w represents the weight of the convolution kernel.

8. The method according to claim 6, characterized in that The progressive feature fusion network submodule is expressed using the following formula: in, represents the features after multi-scale adaptive spatial fusion, x ij n-＞l represents the feature vector from Level n to Level l, Represents the adaptive spatial fusion coefficient of Level l.

9. The method according to claim 6, characterized in that The contextual semantic enhancement submodule is represented by the following formula: Among them, AdaptiveConv1D represents adaptive one-dimensional convolution, and the convolution kernel size is k; GAP represents global average pooling, C represents the number of channels of the input feature map, | | odd Indicates that k is an odd number downwards, F in represents the input features of the contextual semantic enhancement submodule, F out represents the output features of the contextual semantic enhancement submodule, and sigmoid represents the activation function.

10. The method according to claim 5, characterized in that The loss function includes target confidence and classification loss function, prediction box regression loss function and feature comparison loss function; Among them, the target confidence and classification loss functions include the cross entropy loss function; the prediction box regression loss function includes the Wise-IoU loss function; the feature contrast loss function includes the cosine similarity loss function.

Images