Image target detection method based on hybrid minimum point-wasserstein distance

Abstract

The image target detection method based on hybrid minimum point-Wasserstein distance belongs to the technical field of target detection and solves the problem that weak and small targets are prone to be missed in the existing computer vision target detection process. The target image is obtained, the image is processed by using a target detection model to obtain an image marked with different scale targets; the target detection model is obtained by pre-training, and the specific training process is as follows: a global self-attention mechanism lightweight backbone network is used to perform multi-scale feature extraction on sample pictures, a dense nested feature fusion network is used to perform feature map size conversion, channel splicing through upsampling or depth separable convolution to obtain four final size feature maps; the feature maps after feature fusion are subjected to different scale position confidence and category prediction; the predicted values are summarized to obtain an image marked with different scale targets, one training is completed, until a preset training number is reached, and the training is completed. The present application is suitable for image target detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and particularly relates to a target detection method of computer vision. BACKGROUND

[0002] At present, a weak small target detection method based on deep learning gradually becomes a research hotspot. In the latest proposed aerial dataset, a weak small target is defined as being smaller than 12 pixels, which is often fuzzy in boundary and only has extremely limited appearance information, which increases the difficulty of learning discriminative features. As shown in FIG. 1, weak small target information is easy to be annihilated in the feature extraction process, which leads to difficulty in detecting micro objects. In addition to learning discriminative features, the quality of training sample selection also plays an important role in the weak small target detector based on anchor, as shown in FIG. 2, the sensitivity of traditional IoU to different scales of objects changes greatly. Figure 1 Figure 2 Figure 2 In the prior art, the orange frame represents a weak small target, and the blue frame represents a regular target. For the same pixel offset, the IoU (intersection over union) of the weak small target decreases sharply. At present, attention mechanism and multi-scale feature learning are becoming a research hotspot. A backbone network ResNeST based on self-attention mechanism is proposed. In order to reduce the calculation cost, an attempt is made to construct a feature pyramid of different scales. For example, Single Shot MultiBox Detector (SSD) detects objects from feature maps of different resolutions. Feature Pyramid Network (FPN) constructs a top-down structure with horizontal connection to combine feature information of different scales, so as to improve the weak small target detection performance. Then, a pyramid scene parsing network PSPNet is proposed, which uses context information to solve the micro object detection problem. In addition, a parallel multi-branch architecture TridentNet with different receptive fields is constructed to generate feature maps of specific scales. Region-based Recurrent Convolutional Neural Networks (R2CNN) proposes a specially designed backbone network TinyNet for detecting micro objects in large-scale aerial images. However, the above methods do not make the feature information of deep and shallow layers fully interact, and the detection performance of the weak small target is still far from that of the regular target. With the increase of the feature extraction layer, the feature of the weak small target is gradually reduced in the down-sampling process.

[0003] ​​Recently, many researchers believe that the reason why small targets are difficult to be detected is the lack of appropriate training strategies and evaluation metrics. Some people have proposed Single-shot scale-insensitive pedestrian detector (SNIP) network and SNIP with Efficient Refinement (SNIPER) network, which selectively trains objects within a certain scale range. In addition, some people have introduced a scale-aware network SAN, and mapped the features extracted from different spaces to a scale-invariant subspace, making the detector more robust to scale changes. A new NMS method has also been proposed, which is used to improve the detection accuracy of small objects in aerial images. Some people have proposed GWD (Gaussian Wasserstein Distance Loss) for oriented object detection by measuring the positional relationship of the oriented bounding box. It is undeniable that the above methods are committed to developing better training and evaluation strategies to enhance the recognition ability of small objects. However, due to the limitations of model generalization and computational complexity, they cannot be applied to scenarios that require fast response.

[0004] The detection of small targets is different from the detection of regular scale targets. On the one hand, many of the above attention mechanisms and multi-scale feature learning methods do not consider the difference between deep and shallow layers, and directly fuse multi-scale features, which may introduce features that are not conducive to small target detection, thereby reducing the detection performance. On the other hand, although some methods introduce attention mechanisms to enhance target features, they lack efficient evaluation and measurement standards, resulting in many small targets being detected but being mistakenly deleted by regular target measurement standards during NMS (Non-Maximum Suppression) and subsequent processing, leading to serious small target detection failure. The above is a problem that needs to be solved urgently in the current small target detection task. SUMMARY

[0005] The present application is to solve the problem of easy missed detection of small targets in the existing computer vision target detection process, and provides an image target detection method based on hybrid minimum point-Wasserstein distance.

[0006] The image target detection method based on hybrid minimum point-Wasserstein distance provided by the present application comprises:

[0007] Obtaining a target image, processing the target image using a target detection model to obtain an image labeled with different scale targets;

[0008] The target detection model is obtained by pre-training, and the training method of the target detection model comprises:

[0009] The global self-attention mechanism lightweight backbone network is used for multi-scale feature extraction of the sample picture to obtain multi-scale feature maps; the dense nested feature fusion network is used for feature map size conversion through upsampling or deep separable convolution, and channel splicing is performed to obtain four final size feature maps;

[0010] The hybrid minimum point-Wasserstein distance detector is used for position prediction of the feature fusion feature map at different scales, the BCELoss is used for confidence and category prediction of the feature fusion feature map, the position, category and confidence prediction values of the image target of interest at different scales are obtained, the position, category and confidence prediction values of the image target of interest at different scales are summarized, and the image marked with different scale targets is obtained; the training of the target detection model is completed once;

[0011] The target detection model is trained for a set number of times, and the finally obtained model is used as the target detection model.

[0012] Further, in the present application, the global self-attention mechanism lightweight backbone network uses a two-layer 3*3 convolutional network to sequentially extract features from the target image twice, and then inputs the extracted features into four global context extraction modules (GC Module) in sequence. The first three global context extraction modules receive the features, perform feature extraction again, and then transmit the extracted features to the next global context extraction module and the global self-attention mechanism lightweight backbone network module. The fourth global context extraction module directly transmits the extracted features to the global self-attention mechanism lightweight backbone network module after feature extraction.

[0013] Further, in the present application, the formula for feature extraction of the global context extraction module is:

[0014] GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F X )]}

[0015] Wherein, F rgb represents the feature of the input optical image, Conv rgb represents a convolution operation with a convolution kernel of 3 and a step of 2, CBS represents a Conv-BN-SiLU combined operation, and the Conv-BN-SiLU combined operation specifically includes Conv1*1 convolution operation, BN batch normalization processing and SiLU activation function serial calculation in sequence, and R and NR represent residual branch operation and non-residual branch operation, respectively.

[0016] Residual branch operation:

[0017] R(F rgb)=BottleNeck X {CBS[GCB(F rgb )]}

[0018] Non-residual branch operation:

[0019] NR(F rgb )=CBS[GCB(F rgb )]

[0020] Wherein, BottleNeck represents a bottleneck transformation calculation operation, X represents the number of residual blocks, X takes value with the increase of feature depth layer number, GCB(F rgb ) represents self-attention of input optical image features.

[0021] Further, in the application, the formula of self-attention GCB(F rgb ) of input optical image features is:

[0022]

[0023] Wherein, z i represents self-attention GCB(F rgb ) of input optical image features, is the weight of global attention pooling, W v2 ReLU{LN[W v1 (·)]} represents bottleneck transformation, ReLU represents activation function operation, LN represents layer normalization processing, W v1 represents initial 1×1 convolution linear transformation matrix, W v2 represents tail 1×1 convolution linear transformation matrix, x i represents the current input position, W k represents the transformation operation result of the matrix after 1×1 convolution linear transformation and softmax activation function respectively and input matrix multiplication, N P represents the total number of positions in the feature map, x m represents the mth input position, x j represents the jth input position.

[0024] Further, in the application, the dense nested feature fusion network receives four scale feature maps output by four global context extraction modules, respectively adopts depth separable convolution and up sampling to perform size conversion on the second scale feature map and the fourth scale feature map, and performs channel splicing and CSF feature fusion on the size-converted feature map and the third scale feature map; obtain the first fused feature map;

[0025] The first scale feature map is subjected to deep separable convolution, and is converted into a second scale feature map;

[0026] The first scale feature map is subjected to deep separable convolution, and is converted into a second scale feature map;

[0027] The original second scale feature map and the two second scale feature maps converted are subjected to channel splicing and CSF feature fusion, to obtain a second fusion feature map;

[0028] The second fusion feature map is subjected to deep separable convolution and upsampling, respectively, to convert into a third scale feature map and a first scale feature map;

[0029] The original first scale feature map and the converted first scale feature map are subjected to channel splicing, to obtain a final first scale feature map;

[0030] The second fusion feature map and the final first scale feature map are convolved to obtain a final second scale feature map;

[0031] The first fusion feature map and the converted third scale feature map are subjected to channel splicing, and then the fused feature map and the final second scale feature map are convolved, to obtain a final third scale feature map;

[0032] The original fourth scale feature map and the converted fourth scale feature map are subjected to channel splicing, and then the fused feature map and the final third scale feature map are convolved, to obtain a final fourth scale feature map.

[0033] Further, in the present application, the CSF feature fusion first adopts a convolution module CBL to extract features from the feature maps of the same scale, and transmits the extracted features to a feature superposition unit and a split feature reorganization unit, the feature reorganization unit reorganizes the received features and transmits the reorganized features to a first bottleneck unit BottleNeck and the feature superposition unit;

[0034] The first bottleneck unit BottleNeck is sequentially connected with multiple bottleneck units BottleNeck, each bottleneck unit BottleNeck successively performs twice convolution processing on the received signals and transmits the processed features to the next bottleneck unit BottleNeck and the feature superposition unit; until the last bottleneck unit BottleNeck performs twice convolution processing on the received signals and transmits the processed features to the feature superposition unit;

[0035] The feature superposition unit performs feature superposition on the received signal and transmits the result to another convolution module CBL. The other convolution module CBL performs feature extraction on the superimposed features again to obtain fused features.

[0036] Furthermore, in the present invention, a method for predicting positions at different scales of a feature map after feature fusion using a hybrid minimum point-Wasserstein distance detector is as follows:

[0037] First, the K-means clustering method is used to cluster the target boxes in the feature map. Anchor boxes of different sizes and proportions are obtained from the feature map. The intersection-over-union (MMPWIoU) ratio (MMPWIoU) between each anchor box and the corresponding true label in the sample image is calculated. The anchor boxes with an MMPWIoU greater than a set threshold (usually 0.35) are used as the final position prediction values.

[0038] The intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance is:

[0039]

[0040] Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance respectively. Since the value range of the loss function is [0,1], λ+μ=1, exp represents the exponential normalization operation, W2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, and adaptive adjustment is performed according to the average scale of the target in the dataset. IoU represents the original intersection-over-union calculation value, and the ground truth box N a The width is w a , height h a , the predicted box is N b , the distance between the upper left points of the two boxes is d1, and the distance between the lower right points is d2.

[0041] This paper addresses the characteristics of small and weak targets in remote sensing images and the difficulty of extracting their features. It proposes a lightweight backbone with a global self-attention mechanism to improve the reliability of information at each level. Secondly, to better leverage the complementarity of multi-level features, a densely nested feature fusion structure is designed. This structure can incorporate features from both its own layers and adjacent layers, enabling repeated fusion of multi-layer features. Finally, building on the advantages of both distances, a hybrid minimum point-Wasserstein distance is proposed, and this hybrid distance is used to construct an object detector for prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of the existing weak target feature extraction process;

[0043] Figure 2 This is a schematic diagram of the sensitivity of existing weak target positions;

[0044] Figure 3 A schematic diagram of the processing method of the present application is shown in Figure 1.

[0045] Figure 4 A schematic diagram of the dense nested feature fusion structure is shown in Figure 2.

[0046] Figure 5 A schematic diagram of the comparison of the minimum point distance and the Wasserstein distance metrics is shown in Figure 3.

[0047] Figure 6 A schematic diagram of the mixed minimum point-Wasserstein distance MMPW metric of the present application is shown in Figure 4. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0049] DETAILED DESCRIPTION Figure 3 The image target detection method based on the mixed minimum point-Wasserstein distance in the present embodiment includes:

[0050] An image is obtained, and a target detection model is used to process the image to obtain an image labeled with different scale targets.

[0051] The target detection model is obtained by pre-training, and the training method of the target detection model includes:

[0052] A global self-attention mechanism lightweight backbone network is used to perform multi-scale feature extraction on a sample picture to obtain a multi-scale feature map.

[0053] A dense nested feature fusion network is used to perform feature map size conversion through upsampling or depth separable convolution, and channel splicing is performed to obtain four final size feature maps.

[0054] A mixed minimum point-Wasserstein distance detector is used to perform position prediction of different scales on the feature map after feature fusion, and a BCELoss is used to perform confidence and category prediction on the feature map after feature fusion to obtain position, category and confidence prediction values of the target of interest in the image under different scales. The position, category and confidence prediction values of the target of interest in the image under different scales are summarized to obtain an image labeled with different scale targets. The training of the target detection model is completed.

[0055] The target detection model is trained for a certain number of times, and the finally obtained model is taken as the target detection model.

[0056] The images of different scales in the embodiment are small-scale target images (target scale < 12 pixels), small-scale target images (target scale 12-32 pixels), medium-scale target images (target scale 32-96 pixels) and large-scale target images (target scale > 96 pixels), as shown in the following table. Figure 3

[0057] Further, in the present application, the global self-attention mechanism lightweight backbone network adopts a two-layer 3*3 convolution network to sequentially perform feature extraction on the target image twice, and then input the extracted features to four global context extraction modules (GC Module) in turn. The first three global context extraction modules receive the features, perform feature extraction again, and then transmit the extracted features to the next global context extraction module and the global self-attention mechanism lightweight backbone network module. The fourth global context extraction module directly transmits the extracted features to the global self-attention mechanism lightweight backbone network module after feature extraction.

[0058] Further, in the present application, the formula for feature extraction of the global context extraction module is:

[0059] GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]}

[0060] Wherein, F rgb represents the feature of the input optical image, Conv 3-2 represents a convolution operation with a convolution kernel of 3 and a step of 2, CBS represents a Conv-BN-SiLU combined operation, and the Conv-BN-SiLU combined operation specifically includes Conv1*1 convolution operation, BN batch normalization processing and SiLU activation function serial calculation in turn, and R and NR represent residual branch operation and non-residual branch operation, respectively.

[0061] Residual branch operation:

[0062] R(F rgb )=BottleNeck X {CBS[GCB(F rgb )]}

[0063] Non-residual branch operation:

[0064] NR(F rgb )=CBS[GCB(F​rgb )]

[0065] wherein BottleNeck represents a bottleneck transformation calculation operation, X represents a residual block number, X takes a value increasing with the feature depth layer number, GCB(F rgb ) represents a self-attention of an input optical image feature.

[0066] Further, in the present application, the formula of the self-attention GCB(F rgb ) of the input optical image feature is:

[0067]

[0068] wherein z i represents the self-attention GCB(F rgb ) of the input optical image feature, is a weight of global attention pooling, W v2 ReLU{LN[W v1 (·)]} represents a bottleneck transformation, ReLU represents an activation function operation, LN represents a layer normalization processing, W v1 represents an initial 1×1 convolution linear transformation matrix, W v2 represents a tail 1×1 convolution linear transformation matrix, x i represents a current input position, W k represents a transformation operation result of a matrix after respectively passing through a 1×1 convolution linear transformation and a softmax activation function and the input matrix multiplication, N P represents a total number of positions in a feature map, x m represents an mth input position, x j represents a jth input position.

[0069] Further, in the present application, the dense nested feature fusion network receives four scale feature maps output by the four global context extraction modules, respectively uses a depth separable convolution and an up-sampling to perform size conversion on the second scale feature map and the fourth scale feature map, performs channel splicing and CSF feature fusion on the size-converted feature map and the third scale feature map, and obtains a first fusion feature map;

[0070] The first fusion feature map is respectively subjected to a depth separable convolution and an up-sampling for scale conversion; a fourth scale feature map and a second scale feature map are converted;

[0071] The first scale feature map is subjected to a depth separable convolution to be converted into a second scale feature map;

[0072] The original second scale feature map and the two converted second scale feature maps are subjected to channel splicing and CSF feature fusion to obtain a second fusion feature map;

[0073] The feature map after the second fusion is subjected to scale conversion by using a depth separable convolution and upsampling respectively; the feature map converted to the third scale and the feature map of the first scale are obtained.

[0074] The original feature map of the first scale is spliced with the feature map of the first scale obtained by conversion to obtain a final feature map of the first scale.

[0075] The feature map after the second fusion is convolved with the final feature map of the first scale to obtain a final feature map of the second scale.

[0076] The feature map after the first fusion is spliced with the feature map of the third scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the second scale to obtain a final feature map of the third scale.

[0077] The original feature map of the fourth scale is spliced with the feature map of the fourth scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the third scale to obtain a final feature map of the fourth scale.

[0078] Further, in the present application, the CSF feature fusion first uses a convolution module CBL to extract features from the feature map of the same scale, and transmits the extracted features to a feature superposition unit and a split feature reorganization unit. The feature reorganization unit reorganizes the received features and transmits the reorganized features to a first bottleneck unit BottleNeck and a feature superposition unit.

[0079] The first bottleneck unit BottleNeck is sequentially connected with multiple bottleneck units BottleNeck. Each bottleneck unit BottleNeck successively performs two convolution operations on the received signal and transmits the processed features to the next bottleneck unit BottleNeck and the feature superposition unit. Until the last bottleneck unit BottleNeck performs two convolution operations on the received signal and transmits the processed features to the feature superposition unit.

[0080] The feature superposition unit performs feature superposition on the received signal and transmits it to another convolution module CBL. The other convolution module CBL performs feature extraction on the superimposed features again to obtain fused features.

[0081] Further, in the present application, the method for position prediction of feature maps after feature fusion by using a hybrid minimum point-Wasserstein distance detector is as follows:

[0082] Firstly, the K-means clustering method is used to cluster the target frame in the feature map, to obtain anchor frames of different sizes and proportions in the feature map, and to calculate the intersection over union MMPWIoU defined by the mixed minimum point-Wasserstein distance of each anchor frame and the corresponding real label in the sample picture, and the anchor frame with the intersection over union greater than a set threshold (usually 0.35) is taken as the final position prediction value;

[0083] The intersection over union MMPWIoU defined by the mixed minimum point-Wasserstein distance is:

[0084]

[0085] Wherein, lambda and mu respectively represent the weights of the minimum point distance and the Wasserstein distance, since the value range of the loss function is in [0, 1], therefore lambda+mu=1, exp represents an exponential normalization operation, W2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, which is adaptively adjusted according to the average scale of the target in the data set, IoU represents the original intersection over union calculation value, the width of the true frame N a is w a , the height is h a , the predicted frame is N b , the distance between the upper left points of the two frames is d1, and the distance between the right lower points is d2.

[0086] The method disclosed by the application is based on three parts of feature extraction, feature fusion and detector, since the weak small target feature is scarce and easy to be annihilated in the feature extraction process, a lightweight backbone network GC-Backbone with global self-attention mechanism is used to extract small target deep features. Considering the difference between deep and shallow features, a dense nested feature fusion network DN-Neck is designed to fuse weak small target features from bottom to top and from top to bottom. In order to further obtain good detection performance, a mixed minimum point-Wasserstein distance MMPW is proposed to enhance the representation ability of weak small targets and reduce the effective detection results that are mistakenly deleted.

[0087] The lightweight backbone network with global self-attention mechanism uses an optical image as input to detect the network. In the feature extraction process, we use four global context information extraction modules in series to model the global context relationship through Non-Local (global self-attention method). After feature extraction, the calculation results of each stage of the network are taken as the input of the dense nested feature fusion network, all intermediate nodes and upper and lower layers in the network are closely connected with each other, forming a nested network. Each node can receive features from itself and adjacent layers, realizing repeated fusion of multi-layer features, and keeping the representation of weak small targets in the deep layer. Finally, the MMPW detector is used to detect the fusion features.

[0088] Application of global self-attention mechanism lightweight backbone network: due to the rich target feature information in optical images, in some challenging scenes (such as unmanned aerial vehicle images), the features of small weak targets are difficult to be effectively extracted and utilized. The backbone network is a basic information extractor of the target detection network. In some architectures, the feature extraction process includes convolution, pooling, downsampling and the like. However, for small weak targets with less feature information, effective information is easily lost in this long feature extraction process. Therefore, the global self-attention mechanism lightweight backbone network is adopted to better extract image features. Due to the different scales of targets, conventional targets often have more information; however, small weak targets are far away from the camera, and the edges between the targets and the background are often more blurred, lacking contour information. Therefore, a global context extraction module is used in the backbone network to supplement the information of feature maps between different scales and improve the quality of multi-scale image features.

[0089] Since more details and texture information in the image are distributed in the shallow layer of the network, and semantic information is more distributed in the deep layer of the network. In the method, the global self-attention method is adopted to supplement the global context information and enhance the feature extraction capability of the model. The backbone network is divided into four stages, and a global context extraction module is used in each stage to establish a one-way information path in the region. The global context extraction module combines a simplified non-local structure and a lightweight compression excitation structure, which greatly reduces the calculation amount of the model and improves the convergence speed of the network.

[0090] The global context extraction module described in the application includes two parallel branches, wherein the input feature map is separately processed by the self-attention calculation module in each branch before being connected. In the feature extraction process, the self-attention calculation process is added to capture the global context information, and the formula is:

[0091] GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]}

[0092] In the formula, F rgb represents the characteristics of the input optical image, Conv 3-2 represents a convolution operation with k (convolution kernel) being 3 and s (step length) being 2, CBS represents a Conv-BN-SiLU combined operation, specifically including a convolution operation with k (convolution kernel) being 1 and s (step length) being 1, BN (batch normalization processing), and SiLU (activation function) serial calculation, and R and NR represent residual branches and non-residual branches respectively, which can be calculated as:

[0093] R(F rgb )=BottleNeck X{CBS[GCB(F rgb )]

[0094] NR(F rgb )=CBS[GCB(F rgb )]

[0095] Where BottleNeck represents the bottleneck transform computing operation, and the number of residual blocks X is adaptively adjusted according to the number of layers. CBS represents the Conv-BN-SiLU combined operation. GCB represents the self-attention computing module. After performing self-attention computation on the features, we use CBS operation to obtain the spatial sampling matrix of the non-residual branch of the input information source, and use BottleNeck and CBS operation to obtain the spatial sampling matrix of the residual branch. The bottleneck transform is used to reduce the redundancy in the global context features. This is because we want the network to adapt to the characteristics of small weak target feature missing and construct a simple and efficient neural network.

[0096] In the present application, the self-attention computing module GCB is adopted, which combines the advantages of simplified non-local structure and light compression excitation structure, and improves on the basis of Non-Local (global self-attention method) channel attention, captures long-range dependence of feature information and helps network training. Since the target features concerned by different levels of features are not necessarily similar (shallow features may contain contour information, and deep features may contain detail information), direct feature supplement may interfere with the main branch. Therefore, a weighted way is used to supplement the self-attention information, and the self-attention calculation formula is:

[0097]

[0098] Wherein, is the weight of global attention pooling, W v2 ReLU{LN[W v1 (·)]} represents the bottleneck transform, ReLU represents the activation function operation, and LN represents the layer normalization processing. Specifically, our GCB block includes: global attention pooling for context modeling, bottleneck transform to capture channel dependence, and broadcast element addition for feature fusion. Since the GC block and GCB are lightweight, it can be applied in multiple layers to better capture long-range dependencies, with only a slight increase in computational cost.

[0099] In the present application, dense nested feature fusion network is also used to obtain the feature information of the input image. In order to obtain high-quality joint feature expression, a feature fusion network based on dense nested structure DN-Neck is used. The network contains multiple feature fusion structures. Each structure uses a cross-layer fusion module CSF to enhance the features of each scale layer by layer, strengthens the effective representation of small weak targets, and suppresses the interference information that is not conducive to fusion. Each group of feature fusion structures is similar, and the current layer and the upper and lower layers processed are composed of input (if there is no upper or lower layer, there are only two inputs). The upper layer after depth separable convolution and the lower layer after up sampling processing and the current layer have the same scale. The three are spliced and processed through the CSF structure. The detailed structure of CSF is shown in Figure 4

[0100] In the CSF, DenseNet is referenced to increase the rich skip-layer connection, cancel the convolution operation in the branch, and add an additional split operation. The feature information is more abundant, and the calculation amount is reduced, ensuring that both are taken into account. In order to integrate multi-scale features, taking the second layer branch as an example, the deep and shallow layer features are used for feature expansion and fused into the main branch. The processing layers d12 and u32 derived from the current layer g(2) and its shallow layer g(1) and deep layer g(3) are spliced and then fused through the CSF structure. The upper layer d12 processed by depth separable convolution and the lower layer u32 processed by up sampling are scaled to match the size of the current layer g(2). The calculation process of CSF is as follows:

[0101] output=CSF(Concat(g(2),DWConv(g(1)),Upsample(g(3)))

[0102] In the formula, Concat represents channel splicing operation, DWConv represents depth separable convolution operation, and Upsample represents up sampling operation. The design of CSF structure aims to further simplify the network while maintaining or even enhancing its feature extraction capability. It is composed of convolution modules CBL and multiple BottleNeck bottleneck units, where each bottleneck structure contains two consecutive 3x3 convolution blocks. By utilizing more skip connections and minimizing convolution operations, this architecture seamlessly fuses feature maps from different stages of the backbone network, thereby enriching and amplifying feature representations. This multi-scale feature fusion technique is crucial for enhancing the model's ability to detect different size targets and improving its overall performance and versatility in target detection tasks. The output of the CSF layer is then fed as input to the subsequent group of structures, which undergoes depth separable convolution or up sampling to enter the next group of structures. Through cross-modal feature enhancement, high-quality co-expression can be obtained, further improving detection performance.

[0103] ​The present invention also adopts a hybrid minimum point-Wasserstein distance detector, taking into account the difference between conventional scale and micro-scale target metrics, to form a new rectangular box metric method, which is applied to label assignment, NMS and loss function. The metric method based on this MMPW distance is not limited by the scale of the rectangular box, but also conforms to the distribution characteristics of micro-targets in the rectangular box, and has the characteristics of fast convergence and significant performance. Figure 6 and Figure 5 Compared with the two existing measurement methods, it has obvious advantages. Figure 6 As shown in , the width and height of the entire image are represented by w and h respectively, and the true value box is N a , the coordinates of the center point of the true value box can be expressed as (cx a ,cy a ), the predicted box is N b , the coordinates of the center point of the prediction box can be expressed as (cx b ,cy b ) The distance between the upper left points of the two boxes is d1, and the distance between the lower right points is d2.

[0104] Since the Wasserstein distance is not normalized, we use the property of the exponential to take its exponentially normalized value for calculation. Assume that the true value box N a The width is w a , height h a , prediction box N b The width is w b , height h b , then the calculation process of the intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance MMPW is as follows:

[0105]

[0106] Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance, respectively. Since the loss function ranges from [0, 1], λ + μ = 1, exp represents the exponential normalization operation, W2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, and adaptive adjustment is performed according to the average scale of the target in the dataset. IoU represents the original intersection-over-union calculation.

[0107] The experimental equipment adopts Intel(R) Xeon(R) Silver 4210R CPU, and the operating system is Ubuntu 18.04. The method is trained on an NVIDIA RTX 3090 GPU and is implemented under the Pytorch framework. 1024 is used as the input image size, the batch size is 16, and the initial learning rate is 0.01. The optimizer adopts a stochastic gradient descent (SGD) algorithm, the decay weight is 0.0001, and the momentum is 0.937. The present application evaluates each model by using mAP50 and mAP95.

[0108] While the application has been described with reference to particular embodiments, it will be understood that the examples are merely illustrative of the principles and applications of the application. It will be understood that various modifications can be made to the examples, and other arrangements can be devised, without departing from the spirit and scope of the application as defined by the appended claims. It will be understood that the features described with reference to individual examples can be used in other described examples.

Claims

1. An image target detection method based on a hybrid minimum point-Wasserstein distance, characterized in that: include: Obtain the target image, process the target image using the target detection model, and obtain images marked with targets of different scales; The target detection model is obtained through pre-training. The training methods of the target detection model include: A lightweight backbone network with a global self-attention mechanism is used to extract multi-scale features from sample images and obtain multi-scale feature maps; A densely nested feature fusion network is used to convert the feature map size through upsampling or depth-wise separable convolution, and channel splicing is performed to obtain the final feature maps of four sizes; A hybrid minimum point-Wasserstein distance detector is used to predict the positions of the feature maps after feature fusion at different scales. BCELoss is used to predict the confidence and category of the feature maps after feature fusion. The positions, categories, and confidence predictions of the objects of interest in images at different scales are obtained. The positions, categories, and confidence predictions of the objects of interest in images at different scales are summarized to obtain images marked with objects of different scales. This completes the training of the object detection model. The target detection model is trained a set number of times, and the final model is used as the target detection model; The global self-attention mechanism lightweight backbone network uses a two-layer 3×3 convolutional network to extract features from the target image twice in sequence, and then inputs the extracted features into four global context extraction modules in sequence. The first three global context extraction modules extract the received features again and then transmit them to the next global context extraction module and the global self-attention mechanism lightweight backbone network module. The fourth global context extraction module extracts the received features again and transmits the extracted features directly to the global self-attention mechanism lightweight backbone network module. The densely nested feature fusion network receives the feature maps of four scales output by the four global context extraction modules, and uses depthwise separable convolution and upsampling to resize the feature maps of the second scale and the fourth scale respectively. The resized feature maps are then channel-joined and CSF-fused with the feature maps of the third scale to obtain the first fused feature map. The feature map after the first fusion is scaled using depthwise separable convolution and upsampling respectively; it is converted into a feature map of the fourth scale and a feature map of the second scale; Perform depth-wise separable convolution on the feature map of the first scale to convert it into a feature map of the second scale; The original second-scale feature map and the two converted second-scale feature maps are subjected to channel splicing and CSF feature fusion to obtain the second fused feature map; The feature maps after the second fusion are scaled using depthwise separable convolution and upsampling respectively; converted into feature maps of the third scale and feature maps of the first scale; Perform channel splicing on the original first-scale feature map and the first-scale feature map obtained by conversion to obtain the final first-scale feature map; Convolve the feature map after the second fusion with the final feature map of the first scale to obtain the final feature map of the second scale; The feature map after the first fusion is concatenated with the feature map of the third scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the second scale to obtain the final feature map of the third scale; The original fourth-scale feature map and the converted fourth-scale feature map are channel-joined, and then the fused feature map is convolved with the final third-scale feature map to obtain the final fourth-scale feature map; CSF feature fusion first uses a convolution module CBL to extract features from the feature maps of the same scale, and transmits the extracted features to the feature superposition unit and the split feature reorganization unit. The feature reorganization unit reorganizes the received features and transmits the reorganized features to the first bottleneck unit BottleNeck and the feature superposition unit. The first bottleneck unit BottleNeck is connected to multiple bottleneck units BottleNeck in sequence. Each bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits the processed features to the next bottleneck unit BottleNeck and the feature superposition unit. This continues until the last bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits it to the feature superposition unit. The feature superposition unit performs feature superposition on the received signal and transmits the result to another convolution module CBL, which performs feature extraction on the superimposed features again to obtain fused features. The method of using the hybrid minimum point-Wasserstein distance detector to predict the position of the feature map at different scales after feature fusion is as follows: First, the K-means clustering method is used to cluster the target boxes in the feature map, and anchor boxes of different sizes and proportions are obtained in the feature map. The intersection-over-union (MMPWIoU) defined by the mixed minimum point-Wasserstein distance between each anchor box and the corresponding true label in the sample image is calculated. The anchor box with the MMPWIoU greater than the set threshold is used as the final position prediction value. The intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance is: Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance respectively. Since the value range of the loss function is [0,1], λ+μ=1, exp represents the exponential normalization operation, W2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, and adaptive adjustment is performed according to the average scale of the target in the dataset. IoU represents the original intersection-over-union calculation value, and the ground truth box N a The width is w a , height h a , the predicted box is N b , the distance between the upper left points of the two boxes is d1, and the distance between the lower right points is d2.

2. The image target detection system based on hybrid minimum point-Wasserstein distance according to claim 1, characterized in that: The formula for feature extraction of the global context extraction module is: GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]} Among them, F rgb Represents the features of the input optical image, Conv 3-2 It represents a convolution operation with a convolution kernel of 3 and a stride of 2. CBS represents the Conv-BN-SiLU combined operation, which includes the sequential Conv1×1 convolution, BN batch normalization processing, and SiLU activation function serial calculation. R and NR represent the residual branch operation and non-residual branch operation, respectively.

3. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 2, characterized in that: The residual branch operation R is: R(F rgb )=BottleNeck X {CBS[GCB(F rgb )]} The non-residual branch operation NR is: NR(F rgb )=CBS[GCB(F rgb )] Among them, BottleNeck represents the bottleneck transformation calculation operation, X represents the number of residual blocks, and the value of X increases with the number of feature depth layers. GCB(F rgb ) represents the self-attention of the input optical image features.

4. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 1, characterized in that: Self-attention GCB (F rgb ) is: Among them, z i The self-attention GCB (F rgb ), is the weight of the global attention pool, W v2 ReLU{LN[W v1 (·)]} represents bottleneck transformation, ReLU represents activation function operation, LN represents layer normalization processing, W v1 Represents the initial 1×1 convolution linear transformation matrix, W v2 Represents the final 1×1 convolution linear transformation matrix, x i Indicates the current input position, W k Represents the transformation result of multiplying the matrix after 1×1 convolution linear transformation and softmax activation function with the input matrix, N p represents the total number of locations in the feature map, x m represents the mth input position, x j represents the jth input position.

5. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 1, characterized in that: Set the threshold to 0.35.

Images