A target detection method, device, equipment, storage medium and product

Abstract

The application discloses a target detection method, device, equipment, storage medium and product. The method comprises the following steps: inputting a to-be-detected image into a feature extraction module to perform feature extraction, and obtaining a plurality of scale feature maps; inputting the plurality of scale feature maps into a feature fusion module to perform feature fusion, and obtaining a plurality of scale fusion feature maps; inputting the plurality of scale fusion feature maps into a target classification and bounding box regression module to obtain the position of a to-be-detected target in the to-be-detected image; wherein the feature fusion module adopts a bidirectional fusion path of a top-down path and a bottom-up path, and the mth bottom-up path is used for receiving a fusion feature map of one scale input in turn, a candidate output result of an (n-m+1)th top-down path and a candidate output result of an (n-m)th top-down path in addition to receiving the output result of the previous path. The embodiment of the application can improve the small target detection performance and precision.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a target detection method, apparatus, device, storage medium, and product. Background Technology

[0002] Object detection, as one of the core research directions in computer vision, primarily aims to identify and locate objects of interest in images, providing fundamental support for subsequent advanced visual tasks such as object tracking, behavior analysis, and scene understanding. However, object detection still faces many challenges in small object detection scenarios: such as small objects occupying a small proportion of the image, containing less feature information, and having weak semantic information representation, resulting in poor detection performance and low accuracy for small objects.

[0003] To address the challenges of small target detection, researchers proposed the Feature Pyramid Network (FPN) to achieve multi-scale feature fusion, thereby improving small target detection performance. FPN can effectively capture semantic and spatial information at different levels: high-level feature maps (i.e., small-scale feature maps) are rich in semantic information, which helps in identifying complex targets; low-level feature maps (i.e., large-scale feature maps) retain more spatial details, making them more suitable for detecting small targets and targets with rich details.

[0004] However, traditional FPN only completes feature fusion through a unidirectional flow of top-down path, which easily leads to an imbalance in the fusion of feature maps of different scales: the feature information of large-scale feature maps is easily weakened during the fusion process, making the features of small targets easily masked by large targets or backgrounds during the fusion process, thus restricting further improvement of small target detection performance. Summary of the Invention

[0005] This application provides a target detection method, apparatus, device, storage medium, and product to solve the problem in the prior art where FPN only completes feature fusion through a unidirectional flow of a top-down path, which easily leads to the imbalance of feature maps at different scales.

[0006] To achieve the above objectives, embodiments of this application provide a target detection method, including: Acquire the image to be detected; The image to be detected is input into the feature extraction module for feature extraction to obtain feature maps at multiple scales; Feature maps at multiple scales are input into the feature fusion module for feature fusion to obtain a fused feature map at multiple scales; The fused feature map at multiple scales is input into the target classification and bounding box regression module to obtain the location of the target to be detected in the image to be detected; The feature fusion module includes n top-down paths and n bottom-up paths connected in sequence. For n top-down paths, the fused feature maps of the remaining scales except the smallest scale are input into the corresponding top-down paths in order of scale size. Each top-down path receives a fused feature map of one scale. The first top-down path is also used to receive the fused feature map of the smallest scale. For n bottom-up paths, the fused feature maps of the remaining scales except the largest scale are input into the corresponding bottom-up paths in order of scale size. Each bottom-up path receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path; the candidate output results of the (n-m+1)-th top-down path, the candidate output results of the nm-th top-down path, and the fused feature map received by the m-th bottom-up path are at the same scale; m < n, where m is a positive integer and n is a positive integer.

[0007] As an improvement to the above scheme, the top-down path includes: The basic convolutional module, upsampling module, concatenation module, and convolutional block attention-cross-stage local module are connected in sequence. The fused feature maps at all scales except the smallest scale are sequentially input into the corresponding top-down paths in order of scale size, with each top-down path receiving a fused feature map of one scale; the first top-down path is also used to receive the fused feature map of the smallest scale, including: The fused feature maps of all scales except the smallest scale are input into the corresponding top-down path stitching modules in order of scale size. Each top-down path stitching module receives a fused feature map of one scale. The first top-down path of the basic convolutional module is used to receive the smallest scale fused feature map.

[0008] As an improvement to the above scheme, the bottom-up path includes: The standard convolutional module, the splicing module, and the convolutional block attention-cross-stage local module are connected in sequence; The fused feature maps at all scales except the largest scale are input sequentially into the corresponding bottom-up paths in scale order. Each bottom-up path receives a fused feature map at one scale, including: The fused feature maps of all scales except the largest scale are input into the corresponding bottom-up path stitching modules in order of scale size. Each bottom-up path stitching module receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path, including: The concatenation module of the m-th bottom-up path is also used to receive the output of the convolutional block attention-cross-stage local module of the n-m+1-th top-down path and the output of the base convolutional module of the nm-th top-down path.

[0009] As an improvement to the above scheme, the fused feature map of multiple scales includes: The final output of the nth top-down path; The final output of each bottom-up path.

[0010] As an improvement to the above scheme, the feature extraction module includes: A downsampling module, multiple first feature extraction units, and second feature extraction units are connected in sequence; The first feature extraction unit includes a standard convolutional module and a convolutional block attention-cross-stage local module connected in sequence; The second feature extraction unit includes a standard convolutional module, a spatial pyramid pooling module, and a convolutional block attention-cross-stage local module connected in sequence.

[0011] As an improvement to the above scheme, the target classification and bounding box regression module uses the full intersection-union loss function for target classification and bounding box regression.

[0012] To achieve the above objectives, embodiments of this application also provide a target detection device, comprising: The acquisition module is used to acquire the image to be detected; The feature extraction module is used to extract features from the image to be detected, and obtain feature maps at multiple scales; The feature fusion module is used to fuse feature maps from multiple scales to obtain a fused feature map from multiple scales. The target classification and bounding box regression module is used to perform target classification and bounding box regression processing on the fused feature map of multiple scales to obtain the location of the target to be detected in the image to be detected; The feature fusion module includes n top-down paths and n bottom-up paths connected in sequence. For n top-down paths, the fused feature maps of the remaining scales except the smallest scale are input into the corresponding top-down paths in order of scale size. Each top-down path receives a fused feature map of one scale. The first top-down path is also used to receive the fused feature map of the smallest scale. For n bottom-up paths, the fused feature maps of the remaining scales except the largest scale are input into the corresponding bottom-up paths in order of scale size. Each bottom-up path receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path; the candidate output results of the (n-m+1)-th top-down path, the candidate output results of the nm-th top-down path, and the fused feature map received by the m-th bottom-up path are at the same scale; m < n, where m is a positive integer and n is a positive integer.

[0013] To achieve the above objectives, embodiments of this application also provide a target detection device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the target detection method as described above when executing the computer program.

[0014] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the target detection method as described above.

[0015] To achieve the above objectives, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the target detection method as described above.

[0016] Compared with existing technologies, the target detection method, apparatus, device, storage medium, and product provided in this application embodiment extract features from the image to be detected through a feature extraction module to obtain feature maps at multiple scales. A feature fusion module then fuses these feature maps at multiple scales to obtain a fused feature map at multiple scales. Finally, a target classification and bounding box regression module performs target detection on the fused feature map at multiple scales to obtain the location of the target in the image to be detected. The feature fusion module in this application embodiment has n sequentially connected top-down paths and n bottom-up paths. Multi-scale feature fusion is performed through bidirectional paths, effectively avoiding the imbalance in fusion of feature maps at different scales caused by unidirectional flow through only the top-down path. Moreover, in the feature fusion module, the m-th bottom-up path, in addition to receiving the output of the previous path, is also used to receive the fused feature map of one scale, the candidate output of the n-m+1-th top-down path, and the candidate output of the nm-th top-down path. By using the residual connection method, the fusion imbalance of feature maps of different scales can be further alleviated, and the effective fusion and global optimization of feature maps of different scales can be achieved, which ultimately further improves the performance and accuracy of small target detection. Attached Figure Description

[0017] Figure 1This is a flowchart of a target detection method provided in an embodiment of this application; Figure 2 This is a block diagram of a target detection system provided in an embodiment of this application; Figure 3 This is a structural block diagram of a target detection device provided in an embodiment of this application; Figure 4 This is a structural block diagram of a target detection device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] In the description of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0020] In this application description, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0021] In this application description, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "based on" means "at least partially based on." The term "according to" means "at least partially according to." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." The term "and / or" means at least one of the connected objects, such as A and / or B, indicating three cases: including only A, only B, and both A and B. Unless otherwise stated, the term "multiple" means two or more.

[0022] See Figure 1 , Figure 1 This is a flowchart of a target detection method provided in an embodiment of this application. The target detection method includes: S1. Obtain the image to be detected; S2. Input the image to be detected into the feature extraction module for feature extraction to obtain feature maps at multiple scales; S3. Input the feature maps at multiple scales into the feature fusion module for feature fusion to obtain a fused feature map at multiple scales; S4. Input the fused feature map of multiple scales into the target classification and bounding box regression module to obtain the location of the target to be detected in the image to be detected; The feature fusion module includes n top-down paths and n bottom-up paths connected sequentially. These sequential connections extend not only between the n top-down paths but also between the n bottom-up paths. Furthermore, the nth top-down path may connect to the first bottom-up path; that is, the first top-down path connects to the second top-down path, the second top-down path connects to the third top-down path, and so on. The (n-1)th top-down path connects to the nth top-down path, then the nth top-down path connects to the first bottom-up path, the first bottom-up path connects to the second bottom-up path, the second bottom-up path connects to the third bottom-up path, and so on, until the (n-1)th bottom-up path connects to the nth bottom-up path.

[0023] For n top-down paths, the fused feature maps of the remaining scales except the smallest scale are input into the corresponding top-down paths in order of scale size. Each top-down path receives a fused feature map of one scale. The first top-down path is also used to receive the fused feature map of the smallest scale. In other words, the fused feature maps at all scales except the smallest scale are sequentially input into the corresponding top-down paths within the n top-down paths, in order of scale size. Each top-down path receives the fused feature map at one of the remaining scales. Furthermore, the fused feature map at the smallest scale is input into the first top-down path, so the first top-down path receives fused feature maps at two scales.

[0024] Specifically, the fused feature maps at multiple scales are sorted according to their scale size. The smallest-scale fused feature map is input into the first top-down path. The remaining fused feature maps at other scales are then sequentially input into the corresponding top-down paths within the n top-down paths, in ascending order of scale. That is, the second smallest fused feature map is input into the first top-down path, the third smallest into the second, and so on, until the largest-scale fused feature map is input into the nth top-down path. The scale size of the smallest, second smallest, third smallest, ..., largest fused feature map increases sequentially.

[0025] For n bottom-up paths, the fused feature maps of the remaining scales except the largest scale are input into the corresponding bottom-up paths in order of scale size. Each bottom-up path receives a fused feature map of one scale. In other words, the fused feature maps of all scales except the largest scale are sequentially input into the corresponding bottom-up paths of n bottom-up paths in order of scale size. Each bottom-up path receives the fused feature map of one of the other scales.

[0026] Specifically, the fused feature maps at multiple scales are sorted according to their scale size. The fused feature maps at all scales except the largest are then sequentially input into the corresponding bottom-up paths within n bottom-up paths, in descending order of scale. That is, the second largest fused feature map is input into the first bottom-up path, the third largest into the second bottom-down path, and so on, until the smallest fused feature map is input into the nth bottom-up path. The scale size of the fused feature map decreases sequentially from the largest to the smallest.

[0027] This application embodiment adds a bottom-up path while performing fusion via a top-down path, enabling feature information from large-scale feature maps to propagate back to higher levels, thereby improving the detection capability of high-level feature maps for small targets and effectively avoiding the fusion imbalance of feature maps at different scales caused by unidirectional flow through the top-down path alone.

[0028] The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path; the candidate output results of the (n-m+1)-th top-down path, the candidate output results of the nm-th top-down path, and the fused feature map received by the m-th bottom-up path are at the same scale; m < n, where m is a positive integer and n is a positive integer.

[0029] In addition, the candidate output results of the (n-m+1)th top-down path and the candidate output results of the nmth top-down path are also input into the mth bottom-up path.

[0030] In this context, the candidate output results of the (n-m+1)th top-down path, the candidate output results of the nmth top-down path, and the fused feature map received by the mth bottom-up path are at the same scale. For example, if the mth bottom-up path receives the second largest fused feature map, then the scale of the candidate output results of the (n-m+1)th top-down path and the candidate output results of the nmth top-down path is the same as the scale of the second largest fused feature map. This ensures the smooth implementation of residual connections, further alleviates the fusion imbalance of feature maps at different scales, achieves effective fusion and global optimization of feature maps at different scales, and ultimately further improves the performance and accuracy of small target detection.

[0031] In one alternative embodiment, the top-down path includes: The basic convolutional module, upsampling module, concatenation module, and convolutional block attention-cross-stage local module are connected in sequence. The fused feature maps at all scales except the smallest scale are sequentially input into the corresponding top-down paths in order of scale size, with each top-down path receiving a fused feature map of one scale; the first top-down path is also used to receive the fused feature map of the smallest scale, including: The fused feature maps of all scales except the smallest scale are input into the corresponding top-down path stitching modules in order of scale size. Each top-down path stitching module receives a fused feature map of one scale. The first top-down path of the basic convolutional module is used to receive the smallest scale fused feature map.

[0032] It is worth noting that the embodiments of this application utilize a top-down path constructed by sequentially connecting a base convolution module (BaseConv), an upsampling module (UP Sampling), a concatenation module (Concat), and a convolutional block attention-cross-stage local module (CBAM-CSP), and each top-down path includes these modules.

[0033] Here, the basic convolutional module performs basic convolutional operations, consisting of sequentially connected convolutional units, batch normalization units, and a first activation unit. The first activation unit uses either SiLU or ReLU as its activation function. The concatenation module performs concatenation operations. The convolutional block attention-cross-stage local module performs feature extraction and attention weighting operations. It is formed by inserting a convolutional block attention module (CBAM) after the cross-stage local module (CSP). The CBAM performs channel-dimensional and spatial-dimensional attention weighting on the feature map, enhancing the feature representation of the target region and guiding the network to focus more on the region where the target object is located, thereby improving the detection accuracy of small targets.

[0034] CBAM consists of two parts: channel attention and spatial attention. The specific steps are as follows: Channel attention mechanism: First, global average pooling and max pooling are performed on the input feature map (the specific input feature map is determined based on the actual situation; for example, it can be the feature map output by the feature extraction module, the fused feature map output by the feature fusion module, or other outputs from various modules in the object detection system). Then, channel attention maps are generated through a shared fully connected layer. Finally, the channel attention map is multiplied by the input feature map to obtain the enhanced feature map. The expression for the channel attention mechanism is as follows:

[0035]

[0036] in, Represents the Sigmoid function. The first and second results representing the network Represents a multi-sensor machine. This represents the output of the average pooling layer. This represents the output of the max pooling layer.

[0037] Spatial attention mechanism: First, global max pooling and global average pooling are performed on the input feature map to obtain two feature maps. These two feature maps are then concatenated and processed through a dimensionality-reduction convolutional layer, using the sigmoid function to generate a spatial attention feature map. Next, the weight distribution is multiplied element-wise with the input feature map to obtain a weighted feature map. Finally, the weighted feature map is passed through a convolutional operation to generate the output feature map. The spatial attention mechanism is expressed as follows:

[0038]

[0039] in, Represents the Sigmoid function. This represents a two-dimensional convolutional layer (Conv2d). Indicates the size of the convolution kernel. This indicates the input feature map Perform spatial dimension average pooling. This indicates the input feature map Perform max pooling in the spatial dimension. Represents a multi-sensor machine. This represents the output of the average pooling layer. This represents the output of the max pooling layer.

[0040] The expression for CBAM is as follows:

[0041] in, For the input feature map, For channel attention mechanism output, This is the output of the spatial attention mechanism.

[0042] Specifically, the top-down path's convolutional block attention-cross-stage local module is the second convolutional block attention-cross-stage local module (CBAM-CSP2_k), where k represents the number of residual processing steps. The specific processing flow of CBAM-CSP2_k is as follows: 1. Input feature map: Input feature map It is a feature map or fused feature map from the previous level network, with a dimension of .

[0043] 2. Channel separation and basic convolution processing: Input feature map It is divided into two parts, which are processed through two different paths.

[0044] Part 1 Input Feature Map Through a basic convolutional module and standard convolutional modules The feature map is obtained through processing. :

[0045] Part Two: Input Feature Map Through a basic convolutional module Obtain feature map :

[0046] 3. Multi-layer convolution processing: Feature map The data is further processed through multiple basic convolutional modules and standard convolutional modules for in-depth feature extraction.

[0047] Where k represents the number of iterations of the basic convolutional module and the standard convolutional module, which represents the number of residual processing steps.

[0048] 4. Feature map stitching: Feature maps processed through different paths and The features are then stitched together to form a new feature map. :

[0049] 5. Basic convolution processing: Feature map after stitching Features are further extracted and fused using a basic convolutional module:

[0050] 6. Channel attention and spatial attention processing: Feature map after basic convolution processing The data is passed to the convolutional block attention module, where channel attention and spatial attention mechanisms enhance the feature representation capability.

[0051] 7. Output feature map: The final output feature map It has stronger feature representation capabilities and target detection performance, and can be passed to the next level of network or used in target detection tasks.

[0052] In one alternative embodiment, the bottom-up path includes: The standard convolutional module, the splicing module, and the convolutional block attention-cross-stage local module are connected in sequence; The fused feature maps at all scales except the largest scale are input sequentially into the corresponding bottom-up paths in scale order. Each bottom-up path receives a fused feature map at one scale, including: The fused feature maps of all scales except the largest scale are input into the corresponding bottom-up path stitching modules in order of scale size. Each bottom-up path stitching module receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path, including: The concatenation module of the m-th bottom-up path is also used to receive the output of the convolutional block attention-cross-stage local module of the n-m+1-th top-down path and the output of the base convolutional module of the nm-th top-down path.

[0053] It is worth noting that the embodiments of this application utilize a bottom-up path constructed by sequentially connecting a standard convolutional module (CBL, conv bnleakyrelu), a concatenation module (Concat), and a convolutional block attention-cross-stage local module (CBAM-CSP), and each bottom-up path includes these modules.

[0054] Here, the standard convolution module performs standard convolution operations and includes sequentially connected convolutional units, batch normalization units, and a second activation function unit. The second activation unit uses the LeakyReLU activation function. The concatenation module performs concatenation operations. The convolutional block attention-cross-stage local module performs feature extraction and attention weighting operations. It is formed by inserting a convolutional block attention module after the cross-stage local module (CSP). This convolutional block attention-cross-stage local module can be a second convolutional block attention-cross-stage local module (CBAM-CSP2_k), as detailed above, and will not be repeated here.

[0055] In one optional embodiment, the fused feature map of multiple scales includes: The final output of the nth top-down path; The final output of each bottom-up path.

[0056] It is worth noting that, in this embodiment, the final output of the nth top-down path and the final output of each bottom-up path are used as the multi-scale fused feature map of the final output of the feature fusion module. This can fully preserve the semantic and detailed information of the feature maps at different levels, providing more accurate multi-scale feature representation for the subsequent target classification and bounding box regression modules, and further improving the accuracy of target detection.

[0057] In an optional embodiment, the feature extraction module includes: A downsampling module, multiple first feature extraction units, and second feature extraction units are connected in sequence; The first feature extraction unit includes a standard convolutional module and a convolutional block attention-cross-stage local module connected in sequence; The second feature extraction unit includes a standard convolutional module, a spatial pyramid pooling module, and a convolutional block attention-cross-stage local module connected in sequence.

[0058] It is worth noting that the embodiments of this application utilize a downsampling module, multiple first feature extraction units, and second feature extraction units connected in sequence to construct a feature extraction module. Each first feature extraction unit includes a standard convolution module and a convolutional block attention-cross-stage local module connected in sequence, and each second feature extraction unit includes a standard convolution module, a spatial pyramid pooling (SPP) module, and a convolutional block attention-cross-stage local module connected in sequence.

[0059] Here, the downsampling module performs downsampling operations, specifically the FOCUS module, which segments and stitches the image to be detected to perform downsampling, preserving the details of the original features to the maximum extent. The standard convolution module performs standard convolution operations, consisting of sequentially connected convolutional units, batch regularization units, and activation function units. The convolutional block attention-cross-stage local module performs feature extraction and attention weighting operations, formed by inserting a convolutional block attention module (CBAM) after the cross-stage local module (CSP). CBAM includes channel attention and spatial attention, as detailed above, and will not be repeated here.

[0060] Specifically, the convolutional block attention-cross-stage local module of the second feature extraction unit is the second convolutional block attention-cross-stage local module (CBAM-CSP2_k), as detailed above, and will not be repeated here. The convolutional block attention-cross-stage local module of the first feature extraction unit is the first convolutional block attention-cross-stage local module (CBAM-CSP1_k), where k represents the number of residual processing steps. The specific processing flow of CBAM-CSP1_k is as follows: 1. Input feature map: Input feature map It is a feature map from the previous level network, with dimension 1. .

[0061] 2. Channel separation and basic convolution processing: Input feature map It is divided into two parts, each processed by two basic convolutional modules. These two basic convolutional modules extract preliminary features and pass them to subsequent modules.

[0062] Part 1 Input Feature Map Through the first basic convolutional module Obtain feature map :

[0063] Part Two: Input Feature Map Through the second basic convolutional module Obtain feature map :

[0064] 3. Residual element processing: Feature map The gradient is then further passed to multiple residual units (ResUnits) for deeper feature extraction. The residual units enhance feature representation and gradient propagation through skip connections.

[0065]

[0066] Where k represents the number of residual units, and the number of residual units represents the number of residual processing steps.

[0067] 4. Feature map stitching: Feature map after residual unit processing Feature maps without residual element processing The features are then stitched together to form a new feature map. :

[0068] 5. Basic convolution processing: Feature map after stitching Features are further extracted and fused using a basic convolutional module:

[0069] 6. Channel attention and spatial attention processing: Feature map after basic convolution processing Passed to the convolutional block attention module In this study, feature representation capabilities are enhanced through channel attention and spatial attention mechanisms:

[0070] 7. Output feature map: The final output feature map It has stronger feature representation capabilities and target detection performance, and can be passed to the next level of network or used in target detection tasks.

[0071] In one alternative embodiment, the target classification and bounding box regression module uses the full intersection-union loss function for target classification and bounding box regression.

[0072] This application employs the Complete Intersection over Union (CIoU) loss function for object classification and bounding box regression. The CIoU method is based on IoU and considers the center point and diagonal distance factors. By calculating the center point distance between the ground truth bounding box and the predicted bounding box, as well as the diagonal distance of the minimum bounding box, CIoU can better approximate the ground truth bounding box. CIoU also has superior backpropagation capabilities. For example, when the ground truth bounding box and the target bounding box do not overlap, the IoU is equal to 0, which would prevent backpropagation from being performed; CIoU can effectively solve this problem. The specific implementation is as follows: IoU represents the intersection-union ratio between the predicted bounding box and the ground truth bounding box, and its formula is as follows:

[0073] in, It refers to the actual position of the frame within the image. It is the predicted bounding box's position in the image. Represents the area of ​​the overlapping portion. This indicates the total area enclosed.

[0074] The CIoU loss function is as follows:

[0075]

[0076]

[0077]

[0078] in, Represents the prediction box Center point and true frame The Euclidean distance between the center points, and These represent the coordinates of the center point of the prediction box. and These represent the coordinates of the center point of the actual bounding box. This represents the diagonal distance of the smallest closure region that can cover both the predicted bounding box and the ground truth bounding box. It is a weighting factor used to balance the effects of IoU and aspect ratio. This measures the consistency of aspect ratio between the predicted bounding box and the ground truth bounding box. and These represent the width and height of the prediction box, respectively. and These represent the width and height of the actual bounding box, respectively.

[0079] This application embodiment considers the distance between the center point, width, and height of the target box through the CIoU loss function and optimizes the proportional relationship of the target box, making the regression more accurate, reducing the false detection rate, and improving the overall accuracy of target detection.

[0080] In one specific embodiment, the block diagram of the entire target detection system is as follows: Figure 2 As shown, the process involves several steps. First, the image to be detected is processed by a feature extraction module to extract feature maps at multiple scales. Then, these feature maps are fused in a feature fusion module. Finally, the target is identified and located using a target classification and bounding box regression module. Throughout the process, the CIoU loss function is used to optimize bounding box regression, thereby improving detection accuracy.

[0081] The feature extraction module is the backbone of the entire object detection system, the feature fusion module is the neck network, and the object classification and bounding box regression module is the head.

[0082] The feature extraction module includes: Stem layer, Dark2 layer, Dark3 layer, Dark4 layer, and Dark5 layer; the Stem layer includes a downsampling module, specifically FOCUS; Dark2, Dark3, and Dark4 layers include: a first feature extraction unit, which includes a standard convolutional module (CBL) and a first convolutional block attention-cross-stage local module (CBAM-CSP1_k) connected in sequence, namely CBAM-CSP1_1, CBAM-CSP1_3, and CBAM-CSP1_3 respectively; the Dark5 layer includes a standard convolutional module (CBL), a spatial pyramid pooling module (SPP), and a second convolutional block attention-cross-stage local module (CBAM-CSP2_k), specifically CBAM-CSP2_1.

[0083] The feature fusion module receives three feature maps of different scales from the outputs of dark3, dark4, and dark5 layers as input, and fuses these three feature maps of different scales. Specifically: The feature map output from the dark5 layer is concatenated with the feature map output from the dark4 layer after passing through the base convolution module (BaseConv) and the upsampling module. The concatenated result is then input into CBAM-CSP2_1 for processing, and the output result is Res1.

[0084] Res1 is concatenated with the feature map output from the dark3 layer through the base convolution module (BaseConv) and the upsampling module (Upsampling), and the output fused feature map is processed by CBAM-CSP2_1 as the first scale of the fused feature map OUT2, and also as the input of the next scale.

[0085] After OUT2 is downsampled through the convolutional channels using a standard convolutional module (CBL), it is concatenated with the output of the dark4 layer, Res1, and the result of Res1 after passing through a base convolutional module (BaseConv). The resulting fused feature map is then processed by CBAM-CSP2_1 and used as the second-scale fused feature map OUT1, which also serves as the input for the next scale.

[0086] OUT1 is downsampled by a standard convolutional module (CBL) and concatenated with the feature map obtained by the dark5 convolution layer. The fused feature map output by a CBAM-CSP2_1 is used as the fused feature map OUT0 of the last scale.

[0087] The object classification and bounding box regression module receives three fused feature maps of different scales (OUT2, OUT1, and OUT0) output by the feature fusion module. The object classification and bounding box regression module comprises multiple object classification and bounding box units, the number of which is the same as the number of scales of the output fused feature maps. Each fused feature map is processed by inputting one object classification and bounding box.

[0088] Multiple object classification and bounding box units include: a base convolutional module (BaseConv), a standard convolutional module (CBL), and a convolutional module (Conv); among them, the convolutional module only retains a two-dimensional convolutional layer (Conv2d), which is used as the final prediction output layer of the detection head and is responsible for mapping the feature map to the detection result (category, location, confidence). The base convolutional module and the standard convolutional module can be referred to the above description, and will not be repeated here.

[0089] See Figure 3 , Figure 3 This is a structural block diagram of a target detection device 10 provided in an embodiment of this application. The target detection device 10 includes: Acquisition module 11 is used to acquire the image to be detected; Feature extraction module 12 is used to extract features from the image to be detected to obtain feature maps at multiple scales; The feature fusion module 13 is used to fuse features from multiple scales of input feature maps to obtain a fused feature map at multiple scales. The target classification and bounding box regression module 14 is used to perform target classification and bounding box regression processing on the fused feature map of multiple scales to obtain the position of the target to be detected in the image to be detected; The feature fusion module includes n top-down paths and n bottom-up paths connected in sequence. For n top-down paths, the fused feature maps of the remaining scales except the smallest scale are input into the corresponding top-down paths in order of scale size. Each top-down path receives a fused feature map of one scale. The first top-down path is also used to receive the fused feature map of the smallest scale. For n bottom-up paths, the fused feature maps of the remaining scales except the largest scale are input into the corresponding bottom-up paths in order of scale size. Each bottom-up path receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path; the candidate output results of the (n-m+1)-th top-down path, the candidate output results of the nm-th top-down path, and the fused feature map received by the m-th bottom-up path are at the same scale; m < n, where m is a positive integer and n is a positive integer.

[0090] It is worth noting that the working process of each module in the target detection device 10 described in this application embodiment can refer to the working process of the target detection method described in the above embodiment, and achieve the same beneficial effect, which will not be repeated here.

[0091] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the target detection method as described in any of the above embodiments.

[0092] Furthermore, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the target detection method as described in any of the above embodiments.

[0093] See Figure 4 , Figure 4 This is a structural block diagram of a target detection device 20 provided in an embodiment of this application. The target detection device 20 includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above-described target detection method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.

[0094] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the target detection device 20.

[0095] The target detection device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the target detection device 20 and does not constitute a limitation on the target detection device 20. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the target detection device 20 may also include input / output devices, network access devices, buses, etc.

[0096] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the target detection device 20, connecting all parts of the target detection device 20 via various interfaces and lines.

[0097] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the target detection device 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0098] If the modules / units integrated in the target detection device 20 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0099] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0100] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A target detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the feature extraction module for feature extraction to obtain feature maps at multiple scales; Feature maps at multiple scales are input into the feature fusion module for feature fusion to obtain a fused feature map at multiple scales. The fused feature map at multiple scales is input into the target classification and bounding box regression module to obtain the location of the target to be detected in the image to be detected; The feature fusion module includes n top-down paths and n bottom-up paths connected in sequence. For n top-down paths, the fused feature maps of the remaining scales except the smallest scale are input into the corresponding top-down paths in order of scale size. Each top-down path receives a fused feature map of one scale. The first top-down path is also used to receive the fused feature map of the smallest scale. For n bottom-up paths, the fused feature maps of the remaining scales except the largest scale are input into the corresponding bottom-up paths in order of scale size. Each bottom-up path receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path; the candidate output results of the (n-m+1)-th top-down path, the candidate output results of the nm-th top-down path, and the fused feature map received by the m-th bottom-up path are at the same scale. m < n, where m is a positive integer and n is a positive integer.

2. The target detection method as described in claim 1, characterized in that, The top-down path includes: The basic convolutional module, upsampling module, concatenation module, and convolutional block attention-cross-stage local module are connected in sequence. The fused feature maps at all scales except the smallest are sequentially input into the corresponding top-down paths in order of scale size, with each top-down path receiving a fused feature map of one scale; the first top-down path is also used to receive the fused feature map of the smallest scale, including: The fused feature maps of all scales except the smallest scale are input into the corresponding top-down path stitching modules in order of scale size. Each top-down path stitching module receives a fused feature map of one scale. The first top-down path of the basic convolutional module is used to receive the smallest scale fused feature map.

3. The target detection method as described in claim 1, characterized in that, The bottom-up path includes: The standard convolutional module, the splicing module, and the convolutional block attention-cross-stage local module are connected in sequence; The fused feature maps at all scales except the largest scale are input sequentially into the corresponding bottom-up paths in scale order. Each bottom-up path receives a fused feature map at one scale, including: The fused feature maps of all scales except the largest scale are input into the corresponding bottom-up path stitching modules in order of scale size. Each bottom-up path stitching module receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path, including: The concatenation module of the m-th bottom-up path is also used to receive the output of the convolutional block attention-cross-stage local module of the n-m+1-th top-down path and the output of the base convolutional module of the nm-th top-down path.

4. The target detection method as described in claim 1, characterized in that, The fused feature map across multiple scales includes: The final output of the nth top-down path; The final output of each bottom-up path.

5. The target detection method as described in claim 1, characterized in that, The feature extraction module includes: A downsampling module, multiple first feature extraction units, and second feature extraction units are connected in sequence; The first feature extraction unit includes a standard convolutional module and a convolutional block attention-cross-stage local module connected in sequence; The second feature extraction unit includes a standard convolutional module, a spatial pyramid pooling module, and a convolutional block attention-cross-stage local module connected in sequence.

6. The target detection method as described in claim 1, characterized in that, The target classification and bounding box regression module uses the full intersection-over-union loss function for target classification and bounding box regression.

7. A target detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; The feature extraction module is used to extract features from the image to be detected, and obtain feature maps at multiple scales; The feature fusion module is used to fuse feature maps from multiple scales to obtain a fused feature map from multiple scales. The target classification and bounding box regression module is used to perform target classification and bounding box regression processing on the fused feature map of multiple scales to obtain the location of the target to be detected in the image to be detected; The feature fusion module includes n top-down paths and n bottom-up paths connected in sequence. For n top-down paths, the fused feature maps of the remaining scales except the smallest scale are input into the corresponding top-down paths in order of scale size. Each top-down path receives a fused feature map of one scale. The first top-down path is also used to receive the fused feature map of the smallest scale. For n bottom-up paths, the fused feature maps of the remaining scales except the largest scale are input into the corresponding bottom-up paths in order of scale size. Each bottom-up path receives a fused feature map of one scale. The m-th bottom-up path is also used to receive the candidate output results of the (n-m+1)-th top-down path and the candidate output results of the nm-th top-down path; the candidate output results of the (n-m+1)-th top-down path, the candidate output results of the nm-th top-down path, and the fused feature map received by the m-th bottom-up path are at the same scale. m < n, where m is a positive integer and n is a positive integer.

8. A target detection device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the target detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the target detection method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program / instructions that, when executed by a processor, implement the target detection method as described in any one of claims 1 to 6.

Images