Image target detection method based on mixed minimum point-Warisstein distance

Abstract

The invention discloses an image target detection method based on a mixed minimum point-Warisstein distance, belongs to the technical field of target detection, and solves the problem that a weak and small target is easy to leak in a target detection process of existing computer vision. The method comprises the following steps: acquiring a target image, and processing the image by adopting a target detection model to obtain images marked with targets of different scales; the target detection model is obtained through pre-training, and the specific training process is that a global self-attention mechanism lightweight backbone network is adopted to carry out multi-scale feature extraction on sample pictures, a dense nested feature fusion network is adopted to carry out feature map size conversion and channel splicing through up-sampling or depth separable convolution, and the target detection model is obtained through image segmentation. Obtaining final feature maps of four sizes; performing position confidence and category prediction of different scales on the feature map after feature fusion; and summarizing the predicted values, obtaining images marked with targets of different scales, and completing one-time training until a preset number of training times is reached, thereby completing the training. The method is suitable for image target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a target detection method of computer vision. Background Art

[0002] At present, the detection method of small and weak targets based on deep learning has gradually become a research hotspot. In the latest proposed aerial dataset, small and weak targets are defined as smaller than 12 pixels. They often have fuzzy boundaries and only have very limited appearance information, which increases the difficulty of learning discriminative features. Figure 1 As shown in , the information of small targets is easily lost in the feature extraction process, which leads to difficulties in detecting small objects. In addition to learning discriminative features, the quality of training sample selection also plays an important role in anchor-based small target detectors, such as Figure 2 As shown in Figure 2, the sensitivity of traditional IoU to objects of different scales varies greatly. Figure 2 In the figure, the orange box represents a small target, the blue box represents a regular target, and the IoU (intersection over union) of the small target drops dramatically for the same pixel offset. At present, attention mechanisms and multi-scale feature learning are becoming research hotspots. A backbone network ResNeST based on the self-attention mechanism has been proposed. In order to reduce the computational cost, attempts are made to construct feature-level pyramids of different scales. For example, the Single Shot MultiBox Detector (SSD) detects objects from feature maps of different resolutions. The feature pyramid network FPN constructs a top-down structure with lateral connections to combine feature information of different scales to improve the performance of small target detection. Later, a pyramid scene parsing network PSPNet was proposed, which uses contextual information to solve the problem of small object detection. In addition, a parallel multi-branch architecture TridentNet with different receptive fields was constructed to generate feature maps of specific scales. Region-based Recurrent Convolutional Neural Networks (R2CNN) proposed a specially designed backbone network TinyNet for detecting small objects in large-scale aerial images. However, the above methods do not make the feature information of deep and shallow layers fully interact, and the detection performance of weak targets is still far inferior to that of conventional targets. As the number of feature extraction layers increases, the features of weak targets gradually decrease during the downsampling process.

[0003] Recently, many researchers believe that the reason why small objects are difficult to detect is the lack of suitable training strategies and evaluation metrics. Some people have proposed the Single-shot scale-insensitive pedestrian detector (SNIP) network and the SNIP with Efficient Refinement (SNIPER) network to selectively train objects within a certain scale range. In addition, some people have introduced the scale-aware network SAN and mapped the features extracted from different spaces to the scale-invariant subspace, making the detector more robust to scale changes. A new NMS method has also been proposed to improve the detection accuracy of small objects in aerial images. Some people have proposed GWD (Gauss-Wasserstein distance loss) for oriented object detection by measuring the positional relationship of oriented bounding boxes. There is no doubt that the above methods are all committed to developing better training and evaluation strategies to enhance the recognition ability of small objects. However, due to the limitations of model generalization and computational complexity, they cannot be applied to scenarios that require fast response.

[0004] The detection of small and weak targets is different from that of targets of regular scale. On the one hand, many of the above-mentioned attention mechanisms and multi-scale feature learning methods do not consider the difference between deep and shallow layers, and directly fuse multi-scale features, which may lead to the introduction of features that are not conducive to the detection of small and weak targets, thereby reducing the detection performance. On the other hand, although some methods introduce attention mechanisms to enhance target features, they lack efficient evaluation and metrics, resulting in many small and weak targets being detected, but being mistakenly deleted by the metrics of regular targets during NMS (non-maximum suppression) and subsequent processing, resulting in serious omissions of small targets. The above are the problems that need to be solved urgently in the current task of small and weak target detection. Summary of the invention

[0005] The present invention aims to solve the problem that small and weak targets are easily missed in the target detection process of existing computer vision, and now provides an image target detection method based on the hybrid minimum point-Wasserstein distance.

[0006] The image target detection method based on the hybrid minimum point-Wasserstein distance of the present invention comprises:

[0007] Obtain a target image, process the target image using a target detection model, and obtain images marked with targets of different scales;

[0008] The target detection model is obtained through pre-training. The training method of the target detection model includes:

[0009] A lightweight backbone network with a global self-attention mechanism is used to extract multi-scale features from sample images to obtain multi-scale feature maps. A densely nested feature fusion network is used to convert feature map sizes through upsampling or depthwise separable convolution, and channel splicing is performed to obtain feature maps of the final four sizes.

[0010] The hybrid minimum point-Wasserstein distance detector is used to predict the position of the feature map after feature fusion at different scales. The BCELoss is used to predict the confidence and category of the feature map after feature fusion to obtain the position, category and confidence prediction values ​​of the target of interest in the image at different scales. The position, category and confidence prediction values ​​of the target of interest in the image at different scales are summarized to obtain images marked with targets of different scales. The training of the target detection model is completed once.

[0011] The target detection model is trained a set number of times, and the final model is used as the target detection model.

[0012] Furthermore, in the present invention, the global self-attention mechanism lightweight backbone network adopts a two-layer 3×3 convolutional network to extract features of the target image twice in sequence, and then inputs the extracted features into four global context extraction modules (GC Module) in sequence. After receiving the features, the first three global context extraction modules perform feature extraction again and then transmit the extracted features to the next global context extraction module and the global self-attention mechanism lightweight backbone network module. After the fourth global context extraction module performs feature extraction, it directly transmits the extracted features to the global self-attention mechanism lightweight backbone network module.

[0013] Furthermore, in the present invention, the formula for feature extraction of the global context extraction module is:

[0014] GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]}

[0015] Among them, F rgb Represents the features of the input optical image, Conv 3-2 represents a convolution operation with a convolution kernel of 3 and a stride of 2. CBS represents a Conv-BN-SiLU combination operation. The Conv-BN-SiLU combination operation specifically includes sequentially performing a Conv1×1 convolution operation, a BN batch normalization process, and a SiLU activation function serial calculation. R and NR represent residual branch operations and non-residual branch operations, respectively.

[0016] Residual branch operation:

[0017] R(F rgb)=BottleNeck X {CBS[GCB(F rgb )]}

[0018] Non-residual branch operation:

[0019] NR(F rgb )=CBS[GCB(F rgb )]

[0020] Among them, BottleNeck represents the bottleneck transformation calculation operation, X represents the number of residual blocks, and the value of X increases with the number of feature depth layers. GCB(F rgb ) represents the self-attention of the input optical image features.

[0021] Furthermore, in the present invention, the self-attention GCB (F rgb ) is:

[0022]

[0023] Among them, z i The self-attention GCB (F rgb ), is the weight of the global attention pooling, W v2 ReLU{LN[W v1 (·)]} represents bottleneck transformation, ReLU represents activation function operation, LN represents layer normalization processing, W v1 represents the initial 1×1 convolution linear transformation matrix, W v2 Represents the final 1×1 convolution linear transformation matrix, x i Indicates the current input position, W k N represents the transformation result of multiplying the matrix after 1×1 convolution linear transformation and softmax activation function with the input matrix. P represents the total number of locations in the feature map, x m represents the mth input position, x j represents the jth input position.

[0024] Furthermore, in the present invention, the densely nested feature fusion network receives feature maps of four scales output by four global context extraction modules, respectively uses depthwise separable convolution and upsampling to perform size conversion on the feature map of the second scale and the feature map of the fourth scale, performs channel splicing and CSF feature fusion on the feature map after size conversion and the feature map of the third scale; obtains the feature map after the first fusion;

[0025] The feature maps after the first fusion are scaled using depthwise separable convolution and upsampling respectively; they are converted into feature maps of the fourth scale and feature maps of the second scale;

[0026] Perform depth-wise separable convolution on the feature map of the first scale and convert it into a feature map of the second scale;

[0027] The original second-scale feature map and the two converted second-scale feature maps are subjected to channel concatenation and CSF feature fusion to obtain the second fused feature map;

[0028] The feature maps after the second fusion are scaled by using depthwise separable convolution and upsampling respectively; they are converted into feature maps of the third scale and feature maps of the first scale;

[0029] Perform channel splicing on the original first-scale feature map and the first-scale feature map obtained by conversion to obtain a final first-scale feature map;

[0030] Convolve the feature map after the second fusion with the final feature map of the first scale to obtain the final feature map of the second scale;

[0031] The feature map after the first fusion is concatenated with the feature map of the third scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the second scale to obtain the final feature map of the third scale;

[0032] The original feature map of the fourth scale is concatenated with the feature map of the fourth scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the third scale to obtain the final feature map of the fourth scale.

[0033] Furthermore, in the present invention, CSF feature fusion first uses a convolution module CBL to extract features from feature maps of the same scale, and transmits the extracted features to a feature superposition unit and a split feature reorganization unit. The feature reorganization unit reorganizes the received features and transmits the reorganized features to the first bottleneck unit BottleNeck and the feature superposition unit.

[0034] The first bottleneck unit BottleNeck is connected to multiple bottleneck units BottleNeck in sequence. Each bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits the processed features to the next bottleneck unit BottleNeck and the feature superposition unit; until the last bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits it to the feature superposition unit;

[0035] The feature superposition unit performs feature superposition on the received signal and transmits the result to another convolution module CBL. The other convolution module CBL performs feature extraction on the superimposed features again to obtain fused features.

[0036] Furthermore, in the present invention, a method for predicting positions of different scales on a feature map after feature fusion using a hybrid minimum point-Wasserstein distance detector is as follows:

[0037] First, the K-means clustering method is used to cluster the target boxes in the feature map, and anchor boxes of different sizes and proportions in the feature map are obtained. The intersection-over-union ratio (MMPWIoU) defined by the mixed minimum point-Wasserstein distance between each anchor box and the corresponding true label in the sample image is calculated, and the anchor box with the intersection-over-union ratio greater than the set threshold (usually 0.35) is used as the final position prediction value;

[0038] The intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance is:

[0039]

[0040] Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance respectively. Since the value range of the loss function is [0,1], λ+μ=1, exp represents the exponential normalization operation, W 2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, and is adaptively adjusted according to the average scale of the target in the dataset. IoU represents the original intersection-over-union calculation value, and the true value box N a The width is w a , height h a , the predicted box is N b , the distance between the upper left points of the two boxes is d 1 , the distance between the lower right points is d 2 .

[0041] Aiming at the characteristics of small and weak targets in remote sensing images and considering the difficulty of extracting small and weak target features, this paper proposes a lightweight backbone with a global self-attention mechanism to improve the reliability of information at each level. Secondly, in order to better utilize the complementarity of multi-level features, a densely nested feature fusion structure is designed, which can receive features from itself and adjacent layers, thereby realizing repeated fusion of multi-layer features. Finally, based on the advantages of the two distances, a target detector based on the hybrid minimum point-Wasserstein distance is proposed and used to complete the prediction task. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of the existing weak target feature extraction process;

[0043] Figure 2This is a schematic diagram of the sensitivity of existing weak target positions;

[0044] Figure 3 This is a schematic diagram of the processing process of the method of the present invention;

[0045] Figure 4 It is a schematic diagram of densely nested feature fusion structure;

[0046] Figure 5 A schematic diagram showing the comparison between the minimum point distance and Wasserstein distance metrics;

[0047] Figure 6 Schematic diagram of the mixed minimum point-Wasserstein distance MMPW measurement of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0049] Specific implementation method 1: refer to Figure 3 Specifically describing this embodiment, the image target detection method based on the hybrid minimum point-Wasserstein distance described in this embodiment includes:

[0050] Obtain a target image, process the target image using a target detection model, and obtain images marked with targets of different scales;

[0051] The target detection model is obtained through pre-training. The training method of the target detection model includes:

[0052] A lightweight backbone network with a global self-attention mechanism is used to extract multi-scale features from sample images and obtain multi-scale feature maps.

[0053] A densely nested feature fusion network is used to convert the feature map size through upsampling or depth-wise separable convolution, and channel splicing is performed to obtain the final feature maps of four sizes;

[0054] The hybrid minimum point-Wasserstein distance detector is used to predict the position of the feature map after feature fusion at different scales. The BCELoss is used to predict the confidence and category of the feature map after feature fusion to obtain the position, category and confidence prediction values ​​of the target of interest in the image at different scales. The position, category and confidence prediction values ​​of the target of interest in the image at different scales are summarized to obtain images marked with targets of different scales. The training of the target detection model is completed once.

[0055] The target detection model is trained a set number of times, and the final model is used as the target detection model.

[0056] The images at different scales described in this embodiment are weak-scale target images (target scale <12 pixels), small-scale target images (target scale is 12-32 pixels), medium-scale target images (target scale is 32-96 pixels) and large-scale target images (target scale is >96 pixels). Figure 3 shown.

[0057] Furthermore, in the present invention, the global self-attention mechanism lightweight backbone network adopts a two-layer 3×3 convolutional network to extract features of the target image twice in sequence, and then inputs the extracted features into four global context extraction modules (GC Module) in sequence. After receiving the features, the first three global context extraction modules perform feature extraction again and then transmit the extracted features to the next global context extraction module and the global self-attention mechanism lightweight backbone network module. After the fourth global context extraction module performs feature extraction, it directly transmits the extracted features to the global self-attention mechanism lightweight backbone network module.

[0058] Furthermore, in the present invention, the formula for feature extraction of the global context extraction module is:

[0059] GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]}

[0060] Among them, F rgb Represents the features of the input optical image, Conv 3-2 represents a convolution operation with a convolution kernel of 3 and a stride of 2. CBS represents a Conv-BN-SiLU combination operation. The Conv-BN-SiLU combination operation specifically includes sequentially performing a Conv1×1 convolution operation, a BN batch normalization process, and a SiLU activation function serial calculation. R and NR represent residual branch operations and non-residual branch operations, respectively.

[0061] Residual branch operation:

[0062] R(F rgb )=BottleNeck X {CBS[GCB(F rgb )]}

[0063] Non-residual branch operation:

[0064] NR(F rgb )=CBS[GCB(F rgb )]

[0065] Among them, BottleNeck represents the bottleneck transformation calculation operation, X represents the number of residual blocks, and the value of X increases with the number of feature depth layers. GCB(F rgb ) represents the self-attention of the input optical image features.

[0066] Furthermore, in the present invention, the self-attention GCB (F rgb ) is:

[0067]

[0068] Among them, z i The self-attention GCB (F rgb ), is the weight of the global attention pooling, W v2 ReLU{LN[W v1 (·)]} represents bottleneck transformation, ReLU represents activation function operation, LN represents layer normalization processing, W v1 represents the initial 1×1 convolution linear transformation matrix, W v2 Represents the final 1×1 convolution linear transformation matrix, x i Indicates the current input position, W k N represents the transformation result of multiplying the matrix after 1×1 convolution linear transformation and softmax activation function with the input matrix. P represents the total number of locations in the feature map, x m represents the mth input position, x j represents the jth input position.

[0069] Furthermore, in the present invention, the densely nested feature fusion network receives feature maps of four scales output by four global context extraction modules, respectively uses depthwise separable convolution and upsampling to perform size conversion on the feature map of the second scale and the feature map of the fourth scale, performs channel splicing and CSF feature fusion on the feature map after size conversion and the feature map of the third scale; obtains the feature map after the first fusion;

[0070] The feature maps after the first fusion are scaled using depthwise separable convolution and upsampling respectively; they are converted into feature maps of the fourth scale and feature maps of the second scale;

[0071] Perform depth-wise separable convolution on the feature map of the first scale and convert it into a feature map of the second scale;

[0072] The original second-scale feature map and the two converted second-scale feature maps are subjected to channel concatenation and CSF feature fusion to obtain the second fused feature map;

[0073] The feature maps after the second fusion are scaled by using depthwise separable convolution and upsampling respectively; they are converted into feature maps of the third scale and feature maps of the first scale;

[0074] Perform channel splicing on the original first-scale feature map and the first-scale feature map obtained by conversion to obtain a final first-scale feature map;

[0075] Convolve the feature map after the second fusion with the final feature map of the first scale to obtain the final feature map of the second scale;

[0076] The feature map after the first fusion is concatenated with the feature map of the third scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the second scale to obtain the final feature map of the third scale;

[0077] The original feature map of the fourth scale is concatenated with the feature map of the fourth scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the third scale to obtain the final feature map of the fourth scale.

[0078] Furthermore, in the present invention, CSF feature fusion first uses a convolution module CBL to extract features from feature maps of the same scale, and transmits the extracted features to a feature superposition unit and a split feature reorganization unit. The feature reorganization unit reorganizes the received features and transmits the reorganized features to the first bottleneck unit BottleNeck and the feature superposition unit.

[0079] The first bottleneck unit BottleNeck is connected to multiple bottleneck units BottleNeck in sequence. Each bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits the processed features to the next bottleneck unit BottleNeck and the feature superposition unit; until the last bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits it to the feature superposition unit;

[0080] The feature superposition unit performs feature superposition on the received signal and transmits the result to another convolution module CBL. The other convolution module CBL performs feature extraction on the superimposed features again to obtain fused features.

[0081] Furthermore, in the present invention, a method for predicting positions of different scales on a feature map after feature fusion using a hybrid minimum point-Wasserstein distance detector is as follows:

[0082] First, the K-means clustering method is used to cluster the target boxes in the feature map, and anchor boxes of different sizes and proportions in the feature map are obtained. The intersection-over-union ratio (MMPWIoU) defined by the mixed minimum point-Wasserstein distance between each anchor box and the corresponding true label in the sample image is calculated, and the anchor box with the intersection-over-union ratio greater than the set threshold (usually 0.35) is used as the final position prediction value;

[0083] The intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance is:

[0084]

[0085] Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance respectively. Since the value range of the loss function is [0,1], λ+μ=1, exp represents the exponential normalization operation, W 2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, and is adaptively adjusted according to the average scale of the target in the dataset. IoU represents the original intersection-over-union calculation value, and the true value box N a The width is w a , height h a , the predicted box is N b , the distance between the upper left points of the two boxes is d 1 , the distance between the lower right points is d 2 .

[0086] The method described in the present invention is based on feature extraction, feature fusion and detector. Since the features of small and weak targets are scarce and easily annihilated in the feature extraction process, a lightweight backbone network GC-Backbone with a global self-attention mechanism is used to extract the deep features of small targets respectively. Considering the differences between deep and shallow features, a bottom-up and top-down densely nested feature fusion network DN-Neck is designed to fuse the features of small and weak targets. In order to further obtain good detection performance, a hybrid minimum point-Wasserstein distance MMPW is proposed to enhance the representation ability of small and weak targets and reduce the number of valid detection results that are mistakenly deleted.

[0087] The lightweight backbone network of the global self-attention mechanism uses optical images as input to detect the network. In the feature extraction process, we use four global context information extraction modules in series and extract the relationship of the global context through Non-Local (global self-attention method) modeling. After feature extraction, this method uses the calculation results of each stage of the network as the input of a densely nested feature fusion network, in which all intermediate nodes and upper and lower layers are closely connected to each other to form a nested shape network. Each node can receive features from itself and adjacent layers, realize repeated fusion of multi-layer features, and keep the representation of weak targets in the deep layer. Finally, the MMPW detector is used to detect the fused features.

[0088] Application of lightweight backbone network with global self-attention mechanism: Since optical images are rich in target feature information, in some challenging scenarios (such as drone images), the features of small and weak targets are difficult to be effectively extracted and utilized. The backbone network is the basic information extractor of the target detection network. In some architectures, feature extraction undergoes a multi-level processing process including convolution, pooling, downsampling, etc. However, for small and weak targets with less feature information, this lengthy feature extraction process is prone to gradually lose effective information. Therefore, the present invention adopts a lightweight and efficient backbone network with a global self-attention mechanism to better extract image features. Due to the different target scales, conventional targets often have richer information; however, small and weak targets are shot at a longer distance, and the edges between the target and the background are often more blurred and lack contour information. Therefore, a global context extraction module is used in the backbone network to supplement the information of feature maps between scales and improve the quality of multi-scale image features.

[0089] Since details and texture information in images are more distributed in the shallow layers of the network, while semantic information is more distributed in the deep layers of the network, this method uses a global self-attention method to supplement global context information and enhance the feature extraction ability of the model. The backbone network is divided into four stages. Each stage uses a global context extraction module to establish a unidirectional information path in the local area. The global context extraction module combines a simplified non-local structure and a lightweight compression excitation structure, which greatly reduces the computational complexity of the model and improves the convergence speed of the network.

[0090] The global context extraction module of the present invention contains two parallel branches, where the input feature map is processed separately by the self-attention calculation module in each branch before being connected. The self-attention calculation process is added to the feature extraction process to capture the global context information. The formula is:

[0091] GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]}

[0092] In the formula, F rgb Represents the characteristics of the input optical image, Conv 3-2 represents the convolution operation with k (convolution kernel) being 3 and s (step size) being 2. CBS represents the Conv-BN-SiLU combined operation, which includes the convolution operation with k (convolution kernel) being 1 and s (step size) being 1, BN (batch normalization), and SiLU (activation function) serial calculation. R and NR represent the residual branch and the non-residual branch, respectively, which can be calculated as:

[0093] R(F rgb )=BottleNeck X {CBS[GCB(F rgb )]}

[0094] NR(F rgb )=CBS[GCB(F rgb )]

[0095] Among them, BottleNeck represents the bottleneck transformation calculation operation, and the number of residual blocks X is adaptively adjusted according to the number of layers. CBS represents the Conv-BN-SiLU combination operation. GCB represents the self-attention calculation module. After performing self-attention calculation on the features, we use CBS operation to obtain the spatial sampling matrix of the non-residual branch of the input information source, and use BottleNeck and CBS operations to obtain the spatial sampling matrix of the residual branch. The bottleneck transformation is used to reduce the redundancy in the global context features. This is because we hope that the network can adapt to the characteristics of missing weak target features and build a simple and efficient neural network.

[0096] The present invention adopts the self-attention calculation module GCB, which combines the advantages of simplified non-local structure and lightweight compression excitation structure, improves on the basis of Non-Local (global self-attention method) channel attention, captures feature information remote dependency and helps network training. Since the target features of different levels of features are not necessarily similar (shallow features may contain contour information, while deep features may contain detail information), direct feature supplementation may interfere with the main branch. Therefore, a weighted method is used to supplement the self-attention information, and the self-attention calculation formula is:

[0097]

[0098] in, is the weight of the global attention pooling, W v2 ReLU{LN[W v1(·)]} represents bottleneck transformation, ReLU represents activation function operation, and LN represents layer normalization processing. Specifically, our GCB block includes: global attention pooling for context modeling, bottleneck transformation to capture channel dependencies, and broadcast element addition for feature fusion. Since the GC block and GCB are lightweight, it can be applied in multiple layers to better capture long-range dependencies with only a slight increase in computational cost.

[0099] The present invention also adopts a densely nested feature fusion network. After obtaining the feature information of the input image, in order to obtain high-quality joint feature expression, a feature fusion network DN-Neck based on a densely nested structure is adopted. The network contains multiple feature fusion structures. In each structure, the cross-layer fusion module CSF is used to enhance the features of each scale layer by layer, strengthen the effective representation of weak targets, and suppress interference information that is not conducive to fusion. Each group of feature fusion structures is similar, and the input is composed of the current layer and the layers after the upper and lower layers are processed (if there is no upper or lower layer, there are only two inputs). The upper layer after the depth-separable convolution, the lower layer after upsampling, and the current layer have the same scale. The three are spliced ​​and passed through the CSF structure. The detailed structure of CSF is as follows: Figure 4 shown.

[0100] In CSF, referring to DenseNet, abundant skip-layer connections are added, convolution operations in branches are cancelled, and additional split operations are added, so that feature information is richer while reducing the amount of calculation, ensuring both are taken into account. In order to integrate multi-scale features, taking the second-layer branch as an example, deep and shallow layer features are used for feature expansion and fused to the trunk branch. The processing layers d12 and u32 derived from the current layer g(2) and its shallow layer g(1) and deep layer g(3) are concatenated and then fused through the CSF structure. The upper layer d12 processed by the depthwise separable convolution and the upsampled lower layer u32 are scaled to match the size of the current layer g(2). The calculation process of CSF is as follows:

[0101] output=CSF(Concat(g(2),DWConv(g(1)),Upsample(g(3))))

[0102] Concat in the formula represents the channel concatenation operation, DWConv represents the depth-separable convolution operation, and Upsample represents the upsampling operation. The design of the CSF structure aims to further simplify the network while maintaining or even enhancing its feature extraction capabilities. It consists of a convolution module CBL and a combination of multiple bottleneck units BottleNeck, where each bottleneck structure contains two consecutive 3×3 convolution blocks. By utilizing more skip connections and minimizing convolution operations, the architecture seamlessly fuses feature maps from different stages of the backbone network, thereby enriching and amplifying feature representations. This multi-scale feature fusion technique is crucial to enhancing the model's ability to detect objects of different sizes and improving its overall performance and versatility in object detection tasks. The output of the CSF layer is then fed as input to a subsequent set of structures, undergoing depth-separable convolution or upsampling to enter the next set of structures. Through cross-modal feature enhancement, high-quality co-representation can be obtained, further improving detection performance.

[0103] The present invention also adopts a hybrid minimum point-Wasserstein distance detector, taking into account the difference between conventional scale and micro-scale target measurement, to form a new rectangular box measurement method, which is applied to label assignment, NMS and loss function. The measurement method based on this MMPW distance is not limited by the scale of the rectangular box, but also conforms to the distribution characteristics of micro-targets in the rectangular box, and has the characteristics of fast convergence and significant performance. And Figure 6 and Figure 5 Compared with the two existing measurement methods, it has obvious advantages. Figure 6 As shown in , the width and height of the entire image are represented by w and h respectively, and the true value box is N a , the coordinates of the center point of the true value box can be expressed as (cx a ,cy a ), the predicted box is N b , the coordinates of the center point of the prediction box can be expressed as (cx b ,cy b ) The distance between the upper left points of the two boxes is d 1 , the distance between the lower right points is d 2 .

[0104] Since the Wasserstein distance is not normalized, we use the property of the exponential to take its exponentially normalized value for calculation. Assume that the true value box N a The width is w a , height h a , prediction box N b The width is w b , height h b , then the calculation process of the intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance MMPW is as follows:

[0105]

[0106] Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance respectively. Since the value range of the loss function is [0,1], λ+μ=1, exp represents the exponential normalization operation, W 2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, which is adaptively adjusted according to the average scale of the target in the dataset, and IoU represents the original intersection over union calculation.

[0107] The experimental equipment uses Intel(R) Xeon(R) Silver 4210R CPU and the operating system is Ubuntu 18.04. The method was trained on NVIDIA RTX 3090 GPU and implemented in Pytorch framework. 1024 was used as the input image size, the batch size was 16, and the initial learning rate was 0.01. The optimizer used the stochastic gradient descent (SGD) algorithm, the decay weight was 0.0001, and the momentum was 0.937. The present invention uses mAP50 and mAP95 to evaluate each model.

[0108] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. An image object detection method based on a hybrid minimum point-Wasserstein distance, characterized in that: include: Obtain a target image, process the target image using a target detection model, and obtain images marked with targets of different scales; The target detection model is obtained through pre-training. The training method of the target detection model includes: A lightweight backbone network with a global self-attention mechanism is used to extract multi-scale features from sample images and obtain multi-scale feature maps. A densely nested feature fusion network is used to convert the feature map size through upsampling or depth-wise separable convolution, and channel splicing is performed to obtain the final feature maps of four sizes; The hybrid minimum point-Wasserstein distance detector is used to predict the position of the feature map after feature fusion at different scales. The BCELoss is used to predict the confidence and category of the feature map after feature fusion to obtain the position, category and confidence prediction values ​​of the target of interest in the image at different scales. The position, category and confidence prediction values ​​of the target of interest in the image at different scales are summarized to obtain images marked with targets of different scales. The training of the target detection model is completed once. The target detection model is trained a set number of times, and the final model is used as the target detection model.

2. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 1, characterized in that: The global self-attention mechanism lightweight backbone network uses a two-layer 3×3 convolutional network to extract features of the target image twice in sequence, and then inputs the extracted features into four global context extraction modules in sequence. The first three global context extraction modules extract the received features again and then transmit them to the next global context extraction module and the global self-attention mechanism lightweight backbone network module. The fourth global context extraction module extracts the received features again and transmits the extracted features directly to the global self-attention mechanism lightweight backbone network module.

3. The image target detection system based on hybrid minimum point-Wasserstein distance according to claim 2, characterized in that: The formula for feature extraction of the global context extraction module is: GC(F rgb )=Conv 3-2 {CBS[R(F rgb )+NR(F rgb )]} Among them, F rgb Represents the features of the input optical image, Conv 3-2 It represents a convolution operation with a convolution kernel of 3 and a stride of 2. CBS represents a Conv-BN-SiLU combined operation, which includes sequentially performing Conv1×1 convolution, BN batch normalization processing, and SiLU activation function serial calculation. R and NR represent residual branch operations and non-residual branch operations, respectively.

4. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 3 is characterized in that: The residual branch operation R is: R(F rgb )=BottleNeck X {CBS[GCB(F rgb )]} The non-residual branch operation NR is: NR(F rgb )=CBS[GCB(F rgb )] Among them, BottleNeck represents the bottleneck transformation calculation operation, X represents the number of residual blocks, and the value of X increases with the number of feature depth layers. GCB(F rgb ) represents the self-attention of the input optical image features.

5. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 1, characterized in that: Self-attention GCB (F rgb ) is: Among them, z i The self-attention GCB (F rgb ), is the weight of the global attention pooling, W v2 ReLU{LN[W v1 (·)]} represents bottleneck transformation, ReLU represents activation function operation, LN represents layer normalization processing, W v1 represents the initial 1×1 convolution linear transformation matrix, W v2 Represents the final 1×1 convolution linear transformation matrix, x i Indicates the current input position, W k N represents the transformation result of multiplying the matrix after 1×1 convolution linear transformation and softmax activation function with the input matrix. p represents the total number of locations in the feature map, x m represents the mth input position, x j represents the jth input position.

6. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 1 or 2, characterized in that: The densely nested feature fusion network receives the feature maps of four scales output by the four global context extraction modules, and uses depthwise separable convolution and upsampling to resize the feature maps of the second scale and the fourth scale, and performs channel concatenation and CSF feature fusion on the resized feature maps and the feature maps of the third scale. Obtain the first fused feature map; The feature maps after the first fusion are scaled using depthwise separable convolution and upsampling. Convert to the feature map of the fourth scale and the feature map of the second scale; Perform depth-wise separable convolution on the feature map of the first scale and convert it into a feature map of the second scale; The original second-scale feature map and the two converted second-scale feature maps are subjected to channel concatenation and CSF feature fusion to obtain the second fused feature map; The feature maps after the second fusion are scaled using depthwise separable convolution and upsampling respectively; Convert to a feature map of the third scale and a feature map of the first scale; Perform channel splicing on the original first-scale feature map and the first-scale feature map obtained by conversion to obtain a final first-scale feature map; Convolve the feature map after the second fusion with the final feature map of the first scale to obtain the final feature map of the second scale; The feature map after the first fusion is concatenated with the feature map of the third scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the second scale to obtain the final feature map of the third scale; The original feature map of the fourth scale is concatenated with the feature map of the fourth scale obtained by conversion, and then the fused feature map is convolved with the final feature map of the third scale to obtain the final feature map of the fourth scale.

7. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 6, characterized in that: CSF feature fusion first uses a convolution module CBL to extract features from feature maps of the same scale, and transmits the extracted features to the feature superposition unit and the split feature reorganization unit. The feature reorganization unit reorganizes the received features and transmits the reorganized features to the first bottleneck unit BottleNeck and the feature superposition unit. The first bottleneck unit BottleNeck is connected to multiple bottleneck units BottleNeck in sequence. Each bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits the processed features to the next bottleneck unit BottleNeck and the feature superposition unit; until the last bottleneck unit BottleNeck performs two consecutive convolutions on the received signal and transmits it to the feature superposition unit; The feature superposition unit performs feature superposition on the received signal and transmits the result to another convolution module CBL. The other convolution module CBL performs feature extraction on the superimposed features again to obtain fused features.

8. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 7, characterized in that: The method of using the hybrid minimum point-Wasserstein distance detector to predict the position of the feature map at different scales after feature fusion is as follows: First, the K-means clustering method is used to cluster the target boxes in the feature map, and anchor boxes of different sizes and proportions in the feature map are obtained. The intersection-over-union ratio (MMPWIoU) defined by the mixed minimum point-Wasserstein distance between each anchor box and the corresponding true label in the sample image is calculated, and the anchor box with the intersection-over-union ratio greater than the set threshold is taken as the final position prediction value; The intersection-over-union ratio MMPWIoU defined by the hybrid minimum point-Wasserstein distance is: Among them, λ and μ represent the weights of the minimum point distance and Wasserstein distance respectively. Since the value range of the loss function is [0,1], λ+μ=1, exp represents the exponential normalization operation, W2 represents the Wasserstein distance, C represents the normalization constant in the Wasserstein distance, and adaptive adjustment is performed according to the average scale of the target in the data set. IoU represents the original intersection-over-union calculation value, and the truth box N a The width is w a , height h a , the predicted box is N b , the distance between the upper left points of the two boxes is d1, and the distance between the lower right points is d2.

9. The image target detection method based on hybrid minimum point-Wasserstein distance according to claim 8, characterized in that: Set the threshold to 0.35.

Images