Target detection model, training method thereof, target detection method, device and medium

Abstract

Embodiments of the present application provide a target detection model and a training method thereof, a target detection method, a device and a medium, wherein the training method of the target detection model comprises: determining a target detection result corresponding to an image sample by using the target detection model; and obtaining model update parameters of the target detection model according to the target detection result corresponding to the image sample and a classification label. In the process of feature extraction of the image sample, the first image feature is subjected to down-sampling processing to obtain a corresponding second image feature; the second image feature is subjected to separation convolution processing to obtain a third image feature; the third image feature is segmented into a first branch feature and a second branch feature; the first branch feature is subjected to double-branch feature interaction processing; and the second separation convolution processing is performed on the splicing feature corresponding to the interaction processing feature and the second branch feature. The embodiments of the present application can improve the representation strength of the target detection model on the spatial detail information of small targets, and further improve the detection accuracy of small targets.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer vision technology, and in particular to an object detection model and its training method, object detection method, device and medium. Background Technology

[0002] With the rapid development of video platforms, the scale of video content data continues to grow, generating massive amounts of dramas, variety shows, movies, documentaries, and short videos every day. Automated video content parsing has become a core requirement for improving business capabilities and user experience. Object detection technology is the foundation of video content parsing, determining the depth and accuracy of content understanding, and is a key technology supporting the implementation of applications such as video recommendation, video search, video tag generation, content moderation, and video understanding and generation.

[0003] Current object detection methods typically employ object detection models. These models first extract features from the input video frames using a backbone network, obtaining features at different levels. Then, multi-scale fusion techniques are used to aggregate and enhance these features, forming fused features. Finally, based on these fused features, detection results such as the category and location of the target object in the video frame are output.

[0004] However, in video scenarios containing small targets such as distant faces, brand logos, rings, pills, and small dangerous items, the backbone network is prone to severe compression or even loss of information in the spatial dimension of the corresponding features of small targets due to multiple downsampling operations, thus limiting the detection accuracy of the target detection model for small targets. Summary of the Invention

[0005] This invention provides a target detection model and its training method, target detection method, device and medium, which can improve the representation strength of the target detection model for spatial details of small targets, thereby improving the detection accuracy of small targets.

[0006] In a first aspect, embodiments of the present invention disclose a method for training an object detection model, the method comprising: Obtain a training sample set; the training sample set includes: multiple image samples, and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; Using an object detection model, the object detection result corresponding to the image sample is determined. Specifically, during feature extraction of the image sample, the object detection model downsamples a first image feature to obtain a corresponding second image feature; performs a split convolution on the second image feature to obtain a third image feature; divides the third image feature into a first branch feature and a second branch feature; performs a bi-branch feature interaction process on the first branch feature to obtain an interactively processed feature; the bi-branch feature interaction process is used to divide the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and performs interactive processing on the first branch feature map and the second branch feature map; concatenates the interactively processed feature with the second branch feature to obtain a concatenated feature; performs a second split convolution on the concatenated feature to obtain an enhanced image feature; and the object detection model determines the object detection result corresponding to the image sample based on the enhanced image feature. Based on the target detection results and classification labels corresponding to the image samples, the loss information is determined; Based on the loss information, the model update parameters of the target detection model are obtained.

[0007] Secondly, embodiments of the present invention disclose a target detection model, including: a feature extraction network, a feature fusion network, and a detection network; The feature extraction network is used to extract features from the image to be detected, resulting in enhanced image features at multiple levels. The feature fusion network is used to fuse the enhanced image features at multiple levels to obtain fused features at multiple levels. The detection network is used to obtain target detection results based on the fusion features of the multiple levels; The feature extraction network includes multiple layers of feature extraction units; each layer of feature extraction unit includes a downsampling unit and a feature enhancement unit; the output features of the downsampling unit are input to the feature enhancement unit; the output features of the feature enhancement unit are used as the enhanced image features of the corresponding layer. The feature enhancement unit includes: a first depthwise separable convolutional unit, a channel splitting unit, multiple cascaded bi-branch interactive processing units, a channel fusion unit, and a second depthwise separable convolutional unit; wherein, the channel splitting unit is used to segment the output features of the first depthwise separable convolutional unit into a first branch and a second branch; the first branch is input to at least one bi-branch interactive processing unit, which performs bi-branch feature interaction processing on the corresponding input features; the bi-branch feature interaction processing is used to segment the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and perform interactive processing on the first branch feature map and the second branch feature map; the second branch is input to the channel fusion unit; the channel fusion unit is used to concatenate the output features of the at least one bi-branch interactive processing unit with the second branch, and output the concatenated features to the second depthwise separable convolutional unit.

[0008] Thirdly, embodiments of the present invention disclose a target detection method, the method comprising: Acquire the image to be detected; Using the target detection model trained according to the aforementioned method, target detection is performed on the image to be detected, and the target detection result corresponding to the target object in the image to be detected is obtained.

[0009] Fourthly, embodiments of the present invention disclose a training apparatus for an object detection model, the apparatus comprising: The acquisition module is used to acquire a training sample set; the training sample set includes: multiple image samples, and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; A detection module is used to determine the target detection result corresponding to the image sample using a target detection model. Specifically, during feature extraction of the image sample, the target detection model downsamples a first image feature to obtain a corresponding second image feature; performs a split convolution on the second image feature to obtain a third image feature; segments the third image feature into a first branch feature and a second branch feature; performs a bi-branch feature interaction process on the first branch feature to obtain an interactively processed feature; the bi-branch feature interaction process is used to segment the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and performs interactive processing on the first branch feature map and the second branch feature map; concatenates the interactively processed feature with the second branch feature to obtain a concatenated feature; performs a second split convolution on the concatenated feature to obtain an enhanced image feature; and the target detection model determines the target detection result corresponding to the image sample based on the enhanced image feature. The loss determination module is used to determine loss information based on the target detection results and classification labels corresponding to the image samples; The parameter update module is used to obtain the model update parameters of the target detection model based on the loss information.

[0010] Fifthly, embodiments of the present invention disclose a target detection device, the device comprising: The acquisition module is used to acquire the image to be detected; The target detection module is used to perform target detection on the image to be detected using the target detection model trained according to the aforementioned method, and obtain the target detection result.

[0011] In a sixth aspect, embodiments of the present invention disclose an electronic device, including a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the aforementioned method.

[0012] In a seventh aspect, embodiments of the present invention disclose a non-transitory computer-readable storage medium storing instructions that cause a processor to execute the aforementioned method.

[0013] Compared with the prior art, the embodiments of the present invention have the following advantages: In the feature extraction process of the object detection model, downsampling the first image features reduces spatial resolution, so the downsampled second image features may lose some spatial details of small objects. In this embodiment, after performing a first depthwise separable convolution on the second image features to obtain the third image features, the third image features are divided into a first branch feature and a second branch feature. The first branch feature undergoes bi-branch feature interaction processing to enhance the weakened spatial details of small objects in the second image features, while the second branch feature is passed as a bypass to retain the original spatial details of small objects in the second image features. Because the bi-branch feature interaction processing of the first branch feature enhances the weakened spatial details of small objects after downsampling, and the retention processing of the second branch feature preserves the original spatial details of small objects, the bi-branch feature interaction processing of the first branch feature and the retention processing of the second branch feature work together. Therefore, this embodiment can effectively alleviate the problem of loss of spatial details of small objects caused by downsampling, thereby improving the representation strength of spatial details of small objects in the feature extraction stage of the object detection model. Based on this, this embodiment can improve the accuracy of object detection results for small objects. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the steps of a training method for an object detection model according to an embodiment of the present invention. Figure 2This is a schematic diagram of the structure of a target detection model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the feature enhancement unit according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a feature extraction network according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the feature fusion network according to an embodiment of the present invention; Figure 6 This is a schematic flowchart of the target detection method according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a training device for an object detection model according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the target detection device according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an electronic device 1100 according to an embodiment of the present invention. Detailed Implementation

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] Current object detection methods typically employ object detection models. These models first extract features from the input video frames using a backbone network, obtaining features at different levels. Then, multi-scale fusion techniques are used to aggregate and enhance these features, forming fused features. Finally, based on these fused features, detection results such as the object's category and location in the video frame are output.

[0017] However, in video scenarios containing small targets such as distant faces, brand logos, rings, pills, and small dangerous items, the backbone network is prone to severe compression or even loss of information in the spatial dimension of the corresponding features of small targets due to multiple downsampling operations, thus limiting the model's detection accuracy for small targets.

[0018] To address the technical problem of multiple downsampling operations limiting the detection accuracy of small targets in current target detection methods, this invention provides a training method for a target detection model. The method specifically includes: acquiring a training sample set; the training sample set includes multiple image samples and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; using a target detection model to determine the target detection result corresponding to the image sample; wherein, during feature extraction of the image sample, the target detection model performs downsampling processing on a first image feature to obtain a corresponding second image feature; performs a split convolution processing on the second image feature to obtain a third image feature; divides the third image feature into a first branch feature and a second branch feature; performs bi-branch feature interaction processing on the first branch feature to obtain an interaction-processed feature; concatenates the interaction-processed feature with the second branch feature to obtain a concatenated feature; performs a second split convolution processing on the concatenated feature to obtain an enhanced image feature; determines loss information based on the target detection result and classification label corresponding to the image sample; and updates the model parameters of the target detection model based on the loss information.

[0019] In the feature extraction process, the target detection model of this invention performs downsampling on the first image features. Downsampling reduces the spatial resolution of the feature map, and the resulting second image features may lose some detail information. Next, the target detection model performs a first depthwise separable convolution on the second image features to achieve lightweight feature transformation and obtain the third image features.

[0020] Subsequently, the object detection model segments the third image feature into a first branch feature and a second branch feature. The first branch feature is processed through interaction between the two branches to enhance feature representation; the second branch feature is passed as a bypass, preserving the information in the third image feature.

[0021] Subsequently, the object detection model concatenates the interactive processing features with the second branch features passed by the bypass, combines them in the channel dimension, and then performs channel integration and transformation through a second depthwise separable convolution process, finally outputting enhanced image features.

[0022] Therefore, the above-mentioned feature extraction process of the target detection model enhances the third image features through the first branch to strengthen the weakened small target spatial details in the second image features, and transmits the third image features through the second branch to retain the original small target spatial details in the second image features. This collaborative processing method can effectively alleviate the problem of loss of small target spatial details during downsampling, thereby improving the representation strength of small target spatial details in the feature extraction stage of the target detection model. On this basis, it can improve the accuracy of target detection results for small targets.

[0023] It should be noted that in the field of object detection, spatial detail information can refer to the specific features of the object, such as its edges, contours, and textures, which can reflect its shape and position. These are precisely the key information that is easily lost during the downsampling process.

[0024] In video-related scenarios, a small target is an entity that exists independently in a single frame or consecutive frames of video footage and is smaller than a set threshold. The set threshold can be determined by those skilled in the art based on actual application requirements. Spatial detail information refers to the specific features such as edges, contours, and textures that reflect the shape and position of a target. The spatial detail information of a small target refers to the visual features of the small target, such as its edges, contours, and textures, that reflect its shape and position. For example, the facial contours of a person in the distance in a video frame, the local texture of a small prop in the corner of the frame, and the edges of a brand logo shot from a distance all belong to the spatial detail information of a small target.

[0025] In the feature extraction process of the object detection model, downsampling the first image features reduces spatial resolution, so the downsampled second image features may lose some spatial details of small objects. In this embodiment, after performing a first depthwise separable convolution on the second image features to obtain the third image features, the third image features are divided into a first branch feature and a second branch feature. The first branch feature undergoes bi-branch feature interaction processing to enhance the weakened spatial details of small objects in the second image features, while the second branch feature is passed as a bypass to retain the original spatial details of small objects in the second image features. Because the bi-branch feature interaction processing of the first branch feature enhances the weakened spatial details of small objects after downsampling, and the retention processing of the second branch feature preserves the original spatial details of small objects, the bi-branch feature interaction processing of the first branch feature and the retention processing of the second branch feature work together. Therefore, this embodiment can effectively alleviate the problem of loss of spatial details of small objects caused by downsampling, thereby improving the representation strength of spatial details of small objects in the feature extraction stage of the object detection model. Based on this, this embodiment can improve the accuracy of object detection results for small objects.

[0026] The training method of the target detection model of the present invention will be described below through specific embodiments.

[0027] Reference Figure 1 The diagram illustrates a step-by-step flowchart of a training method for an object detection model according to an embodiment of the present invention. The method may specifically include the following steps: Step 101: Obtain the training sample set; the training sample set includes: multiple image samples and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; Step 102: Using a target detection model, determine the target detection result corresponding to the image sample; wherein, during the feature extraction process of the image sample, the target detection model performs downsampling processing on the first image feature to obtain the corresponding second image feature; performs separable convolution processing on the second image feature to obtain the third image feature; divides the third image feature into a first branch feature and a second branch feature; performs bi-branch feature interaction processing on the first branch feature to obtain an interactive processing feature; the bi-branch feature interaction processing is used to divide the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and performs interactive processing on the first branch feature map and the second branch feature map; concatenates the interactive processing feature with the second branch feature to obtain a concatenated feature; performs a second separable convolution processing on the concatenated feature to obtain an enhanced image feature; the target detection model determines the target detection result corresponding to the image sample based on the enhanced image feature; Step 103: Determine the loss information based on the target detection results and classification labels corresponding to the image samples; Step 104: Obtain the model update parameters of the target detection model based on the loss information.

[0028] In step 101, the training sample set can be sourced from publicly available standard datasets or real images collected in specific application scenarios. To improve the generalization ability of the object detection model, the training sample set can be collected based on rules such as scene diversity, object category, object scale (especially small objects), lighting, and occlusion conditions.

[0029] Classification labels are a set of information formed by annotating targets in image samples. They include bounding box coordinates, the category to which the target belongs, and in some scenarios, auxiliary information such as target size and occlusion status. They are used to compare with the target detection results output by the target detection model to guide model training.

[0030] A classification label is a discrete identifier assigned to each target to indicate its semantic category, such as "actor's face," "sneaker prop," or "beverage logo." This classification label typically appears in pairs with bounding boxes, together forming a complete target annotation. During training, the classification label is used as the ground truth and, along with the category confidence distribution output by the detection network, to calculate the loss and guide model learning. During inference, the classification label corresponds to the category confidence distribution output by the detection network as the prediction result. Here, "target" refers to an independent entity existing in a single frame or consecutive frames, such as a single person's face, a single prop, or a single brand logo.

[0031] In step 102, the target detection model performs forward computation on the input image samples to generate detection results.

[0032] In the feature extraction stage, the object detection model first downsamples the first image features, reducing their spatial resolution while expanding the receptive field to obtain the second image features. This process may weaken the fine structural information of small targets. Subsequently, the object detection model performs a first depthwise separable convolution on the second image features to achieve lightweight feature transformation and channel information integration, resulting in the third image features. Then, the third image features are segmented into first branch features and second branch features.

[0033] For example, a third image feature with N channel dimensions is segmented into a first branch feature and a second branch feature; the number of channels in the first branch feature and the second branch feature is C / 2.

[0034] Channel dimension is an independent dimension in image features that carries different visual information, while the number of channels refers to the total number of channel dimensions contained in the image feature. For example, a third image feature with 4 channels, after being divided equally along the channel dimensions, has the features of the first 2 channel dimensions as the first branch feature and the features of the last 2 channel dimensions as the second branch feature. After the division, the number of channels in both the first and second branch features is 2.

[0035] The first branch features undergo interactive processing with the second branch features to enhance their expressive power; the second branch features are then passed as a bypass. Next, the processed first and second branch features are concatenated along the channel dimension, and then processed by a second depthwise separable convolution for channel integration and dimensionality adjustment, ultimately outputting the enhanced image features for this stage. Based on these enhanced image features and subsequent network layers (such as feature fusion networks and detection networks), the object detection model ultimately outputs object detection results containing predicted categories and bounding boxes.

[0036] A bounding box is a rectangular region used to mark the spatial location of an object in an image sample; its coordinates are typically defined using the pixel coordinate system of the image sample. The first image feature refers to the intermediate feature representation generated after processing the image sample in the feature extraction network.

[0037] Step 103 uses a predefined loss function to quantify the deviation between the detection result and the classification label. The loss function is typically a weighted combination of one or more losses, including a classification loss that measures the deviation between the predicted and true classes, and a regression loss that measures the geometric position deviation between the predicted bounding box and the ground truth bounding box. Through calculation, a scalar form of loss information is obtained, which objectively reflects the overall prediction error of the object detection model on the current batch of image samples.

[0038] Step 104, based on the loss information calculated in Step 103, uses the backpropagation algorithm to calculate the gradient of the loss information relative to model parameters such as convolutional kernel weights and biases. The optimizer combines the gradient, preset learning rate, and other hyperparameters to calculate the update amount of each model parameter. The model parameters are adjusted according to the update amount to obtain the updated model parameters. Continuous adjustment of the model parameters enables the object detection model to continuously reduce the loss in subsequent predictions, thereby improving the detection performance. The training process iteratively executes steps 102 to 104, continuously optimizing the model parameters.

[0039] Optionally, the process of performing bi-branch feature interaction processing on the first branch features specifically includes: performing convolution processing on the first branch features to obtain a feature map; dividing the feature map into a third branch feature map and a fourth branch feature map according to the channel dimension; performing convolution transformation on the third branch feature map and the fourth branch feature map respectively to obtain two transformed feature maps; performing element-wise multiplication on the two transformed feature maps to obtain a fused feature map; performing dimensionality reduction processing on the fused feature map, and performing residual connection between the dimensionality-reduced fused feature map and the first branch features to obtain the interactive processing features.

[0040] Convolutional processing of the first branch features adjusts their dimension and representation, generating feature maps suitable for subsequent branch interaction operations. This feature map is then split into third and fourth branch feature maps, and convolutional transformations are performed on each branch to obtain two transformed feature maps. Element-wise multiplication of these two transformed feature maps enables deep information interaction and correlation modeling between the two branches, effectively strengthening effective correlation features and suppressing invalid noise. The resulting fused feature map is then dimensionality-reduced to match the dimension of the first branch features. Finally, a residual connection is established between the dimensionality-reduced fused feature map and the first branch features. This retains the information from the first branch features while incorporating fine-grained correlation features learned from the two-branch interaction, significantly enhancing the feature representation and detail modeling capabilities, and improving the model's detection accuracy and robustness for small and weakly characterized targets.

[0041] In this embodiment of the invention, optionally, the enhanced image features include: first-level enhanced image features, second-level image features, ..., Nth-level enhanced image features arranged sequentially according to their generation time, where N is a positive integer; The target detection model further fuses the enhanced image features of multiple levels in the order from level N to level 2 to obtain fused intermediate features, and fuses the fused intermediate features and the enhanced image features of multiple levels in the order from level 1 to level N to obtain fused features of multiple levels. In the process of fusing enhanced image features from multiple levels in the order from level N to level 2, the enhanced image features of two adjacent levels are fused to obtain an intermediate feature map, and the intermediate feature map is then subjected to coordinate attention weighting processing.

[0042] The coordinate attention weighting process is as follows: First, one-dimensional global average pooling is performed on the intermediate feature map along both the horizontal and vertical directions to obtain horizontal and vertical feature vectors, respectively, to aggregate global spatial context information in the two orthogonal directions while preserving positional information in each direction. Next, the obtained horizontal and vertical feature vectors are concatenated to obtain a concatenated feature vector, which is then transformed and fused with cross-channel information through a 1×1 convolutional layer to obtain a fused feature vector. Subsequently, the fused feature vector is split into two independent branches: a horizontal branch feature vector and a vertical branch feature vector. Each branch is passed through a 1×1 convolutional layer and a non-linear activation unit to generate horizontal and vertical attention weight maps that match the corresponding channel dimensions of the intermediate feature map. Finally, the generated horizontal and vertical attention weight maps are multiplied element-wise with the original intermediate feature map to obtain a horizontally weighted feature map and a vertically weighted feature map, achieving targeted weighting of features in the horizontal and vertical directions.

[0043] Coordinate attention weighting can be performed by this coordinate attention unit. This unit calculates horizontal and vertical attention weight maps along the horizontal and vertical directions respectively for the intermediate feature maps obtained by fusing adjacent layers. This models the spatial location dependencies of features, effectively enhancing the response of features at key spatial locations and improving the accuracy of feature representation. It is particularly helpful for object detection models to capture the position and structural information of targets in complex scenes, improving the accuracy of multi-scale feature fusion and making object detection models more suitable for the high-precision detection needs of small targets on long video platforms.

[0044] The spatial location dependency of features refers to the association and constraint relationships between feature points at different spatial locations in an intermediate feature map. This relationship typically manifests as follows: features within the same target exhibit spatial consistency and continuity; features at different locations are correlated with each other and jointly characterize the target's structure and shape; features in adjacent locations often show strong correlation, exhibiting a locally smooth or structurally continuous feature distribution pattern; while features between the target region and the background or irrelevant regions usually have significant distinguishability, manifested as significant differences in feature responses.

[0045] The coordinate attention unit, by modeling the spatial positional dependencies in the horizontal and vertical directions respectively, can guide the model to focus on the precise spatial location of the target, perceive its spatial range and structural distribution, thereby enhancing the feature response to the target-related region, while suppressing the interference of the background and irrelevant regions.

[0046] In this embodiment of the invention, optionally, the target detection model includes a fusion unit, which is used to fuse multiple input features during the fusion of enhanced image features at multiple levels, and the fusion unit configures corresponding weight parameters for each input feature.

[0047] Specifically, each input feature includes: the transformed image feature obtained by performing image transformation on the enhanced image features of one level, or the transformed image feature obtained by performing image transformation on the fusion result of enhanced image features of multiple levels.

[0048] The step of updating the model parameters of the target detection model based on the loss information includes updating the weight parameters corresponding to each input feature based on the loss information.

[0049] By adaptively learning the weight parameters of each input feature, the object detection model can achieve a more reasonable weighting allocation during feature fusion, strengthening the shallow detail information and spatial location information that play a crucial role in the detection of small objects. Combined with a bidirectional fusion path, the adaptive learning of weight parameters achieves full interaction and balanced integration of deep semantic information and shallow detail information, improving the rationality and effectiveness of multi-scale feature fusion, and further enhancing the model's detection accuracy and robustness for small objects.

[0050] This invention also provides a target detection model.

[0051] Reference Figure 2 The diagram illustrates the structure of a target detection model according to an embodiment of the present invention. The target detection model specifically includes: a feature extraction network 201, a feature fusion network 202, and a detection network 203. The feature extraction network is used to extract features from the image to be detected, obtaining enhanced image features at multiple levels. The feature fusion network is used to fuse the enhanced image features at multiple levels, obtaining fused features at multiple levels. The detection network is used to obtain a target detection result based on the fused features at multiple levels.

[0052] The feature extraction network 201 specifically includes: multiple layers of feature extraction units 211; each layer of feature extraction unit 211 includes: a downsampling unit 2111 and a feature enhancement unit 2112; the output features of the downsampling unit 2111 are input to the feature enhancement unit 2112; the output features of the feature enhancement unit 2111 are used as the enhanced image features of the corresponding layer.

[0053] The downsampling unit 2111 is used to perform spatial dimension downsampling processing on the input features, which reduces the resolution of the feature map while expanding the channel dimension to improve the expressive power of the features.

[0054] The downsampling unit 2111 can be structured as a multi-scale feature convolutional unit. This multi-scale feature convolutional unit performs convolution operations on the input features with different receptive fields through parallel processing of multiple convolutional branches of different scales, so as to capture multi-scale contextual information; then, the output features of each convolutional branch are concatenated and fused along the channel dimension, thereby enhancing the multi-scale representation capability of the features while achieving downsampling.

[0055] In addition to the above structures, downsampling units can also adopt structures such as convolutional downsampling with a stride greater than 1, depthwise separable convolutional downsampling, or max pooling downsampling to adapt to different model design requirements.

[0056] Reference Figure 3 The diagram shows a schematic of the feature enhancement unit according to an embodiment of the present invention. The feature enhancement unit includes: a first depthwise separable convolutional unit 301, a channel splitting unit 302, multiple serially connected dual-branch interactive processing units 303, a channel fusion unit 304, and a second depthwise separable convolutional unit 305.

[0057] In this system, the input features of the feature enhancement unit first enter the first depthwise separable convolutional unit 301. The output of the first depthwise separable convolutional unit 301 is connected to the input of the channel splitting unit 302. Of the two parallel outputs of the channel splitting unit 302, the first output, corresponding to the direct shortcut branch of the second branch, is connected to the first input of the channel fusion unit 304. The second output, corresponding to the feature extraction branch of the first branch, is connected to the input of the first dual-branch interactive processing unit of the series of multiple dual-branch interactive processing units 303. The output of the last dual-branch interactive processing unit of the series of multiple dual-branch interactive processing units 303 is connected to the second input of the channel fusion unit 304. The output of the channel fusion unit 304 is connected to the input of the second depthwise separable convolutional unit 305, which is the overall output of the entire feature enhancement unit.

[0058] The first depthwise separable convolutional unit 301 employs a depthwise separable convolutional structure to perform lightweight feature extraction and dimension adaptation on the input features. This reduces computational cost while preserving key feature information, preparing for subsequent branch processing. The first depthwise separable convolutional unit 301 can sequentially perform channel-wise convolution and pointwise convolution on the input features, and output to the channel splitting unit 302 after completing feature transformation and dimension adjustment.

[0059] The channel splitting unit 302 is used to divide the input features into two parallel processing paths. It can divide the input features into two equal parts based on the total number of channels. For example, if the total number of channels is C, then each path receives features from C / 2 channels. One part serves as a direct shortcut branch, transmitting the features to the channel fusion unit 304; the other part serves as a feature extraction branch, inputting to multiple serially connected dual-branch interactive processing units 303. This segmentation method can reduce computational load while simultaneously ensuring both direct feature transmission and deep feature extraction.

[0060] The cascaded dual-branch interactive processing units 303 may include n (n is a positive integer) dual-branch interactive processing units connected in series. Each dual-branch interactive processing unit performs convolution processing on the corresponding input features to obtain a feature map, divides the feature map into a third branch feature map and a fourth branch feature map, performs convolution transformation on the third branch feature map and the fourth branch feature map respectively to obtain two transformed feature maps, performs element-wise multiplication on the two transformed feature maps to obtain a fused feature map, performs dimensionality reduction processing on the fused feature map, and performs residual connection between the dimensionality-reduced fused feature map and the input features to obtain the output features. By performing residual connection between the dimensionality-reduced fused feature map and the original input features, each dual-branch interactive processing unit can avoid the dilution of small target features in deep structures, making the model training process more stable and helping to improve the detection effect of small targets.

[0061] Multiple cascaded dual-branch interactive processing units 303 divide the feature map into a third-branch feature map and a fourth-branch feature map along the channel dimension. After performing convolutional transformations on the third-branch and fourth-branch feature maps respectively, element-wise multiplication is performed, allowing the channel features of the third-branch and fourth-branch feature maps to interact. That is, the strength of the channel features of the third-branch feature map affects the output of the corresponding channel features of the fourth-branch feature map, thereby achieving cross-channel feature interaction, enhancing the texture and edge expression of small targets, and enabling the feature extraction network to strengthen the information of small-sized targets. Moreover, the convolutional transformation process of the third-branch and fourth-branch feature maps increases the channel dimension of the features, thereby expanding the local features to a higher-dimensional space and improving the expressive power. This branch segmentation and parallel convolutional structure design reduces redundant computation and improves the detection sensitivity of small targets.

[0062] The channel fusion unit 304 can be an operation layer that splices features according to the channel dimension. It is used to fuse the features of the direct shortcut branch and the feature extraction branch according to the channel dimension, and integrate multi-path feature information. The process is to splice and merge the direct shortcut features from the channel splitting unit 302 and the enhanced features from multiple serial dual-branch interaction processing units 303 according to the channel dimension, and output the spliced ​​features to the second depth separable convolutional unit 305.

[0063] The second depthwise separable convolutional unit 305 employs a depthwise separable convolution structure to perform lightweight feature adaptation and output regularization on the stitched features, optimizing feature representation and outputting enhanced image features at the corresponding level. The second depthwise separable convolutional unit 305 can sequentially perform channel-wise convolution and pointwise convolution on the stitched features output by the channel fusion unit 304, completing feature extraction and dimensionality adjustment before serving as the final output of the entire feature enhancement unit.

[0064] In one implementation, the feature extraction network specifically includes: four levels of feature extraction units, wherein the output features of the first three levels of feature enhancement units are used as the enhanced image features of the corresponding levels.

[0065] The feature extraction network may further include a fast spatial pyramid pooling unit connected after the feature extraction unit of the fourth level. The fast spatial pyramid pooling unit is used to perform multi-resolution feature fusion processing on the output features of the feature enhancement unit of the fourth level, and the resulting output features are used as the enhanced image features of the corresponding level.

[0066] Suppose that the P3 feature map represents the first-level enhanced image features output by the first-level feature extraction unit. The P3 feature map is the shallowest feature layer, with the highest resolution and the smallest receptive field, which can preserve the fine-grained details of the first-size target (minimal target) and serve as a feature source for small target detection.

[0067] The P4 feature map represents the second-level enhanced image features output by the second-level feature extraction unit. As a mid-level feature map, the P4 feature map has moderate resolution and receptive field, capable of capturing both fine-grained details and semantic information of second-sized targets, making it suitable for detecting medium-scale targets.

[0068] The P5 feature map represents the third-level enhanced image features output by the third-level feature extraction unit. The P5 feature map is a deep feature map with low resolution, a large receptive field, and rich semantic information. After further enhancement by the fast spatial pyramid pooling unit, it serves as the deep feature received by the feature fusion network.

[0069] It contains multiple parallel pooling layers of different scales, each followed by a 1×1 convolutional layer, and finally a feature concatenation layer. The input features are simultaneously fed into each parallel pooling layer to extract spatial features with different receptive fields. The output features of each pooling layer are individually adjusted for the number of channels by the corresponding 1×1 convolutional layer. These multi-scale features, after convolution, are then combined with the original input features and concatenated along the channel dimension by the concatenation layer. The resulting fused enhanced features serve as a supplementary source for the P5 feature map.

[0070] Reference Figure 4 The diagram illustrates the structure of a feature extraction network according to an embodiment of the present invention. The feature extraction network specifically includes: an initial feature extraction unit 401, a first-level downsampling unit 402, a first-level feature enhancement unit 403, a second-level downsampling unit 404, a second-level feature enhancement unit 405, a third-level downsampling unit 406, a third-level feature enhancement unit 407, a fourth-level downsampling unit 408, a fourth-level feature enhancement unit 409, and a spatial pyramid pooling unit 410, which are connected in series.

[0071] In one example, the image to be detected may originate from video. Specifically, the image to be detected is a single frame of the original video image after the video stream / video file has been parsed by a decoder. This image to be detected can be two-dimensional image data in raw pixel format. Optionally, preprocessing operations can be performed on the detected image, including size normalization, pixel value normalization, color space conversion, and data augmentation (motion blur simulation, compression artifact simulation, random brightness perturbation) based on the characteristics of the video scene, outputting a standardized image tensor that conforms to the model input specifications. A standardized image tensor is a multidimensional data array generated after a series of preprocessing steps on the detected image (such as size normalization, pixel value normalization, color space conversion, and data augmentation), whose format and numerical range meet the model's input requirements. This tensor is typically floating-point type, with dimensions arranged as batch × number of channels × height × width, or batch × height × width × number of channels. It has a uniform size, standardized numerical range, and data type, and can be input into a feature extraction network for forward computation.

[0072] The initial feature extraction unit 401 is used to perform initial feature extraction on the input features (normalized image tensor).

[0073] The first-level downsampling unit 402 is used to perform spatial downsampling and channel expansion on the output features of the initial feature extraction unit 401.

[0074] The first-level feature enhancement unit 403 is used to enhance the output features of the first-level downsampling unit 402 and output the first-level enhanced image features.

[0075] The second-level downsampling unit 404 is used to perform spatial downsampling and channel expansion on the output features of the first-level feature enhancement unit 403.

[0076] The second-level feature enhancement unit 405 is used to enhance the output features of the second-level downsampling unit 404 and output the second-level enhanced image features.

[0077] The third-level downsampling unit 406 is used to perform spatial downsampling and channel expansion on the output features of the second-level feature enhancement unit 405.

[0078] The third-level feature enhancement unit 407 is used to enhance the output features of the third-level downsampling unit 406 and output the third-level enhanced image features.

[0079] The fourth-level downsampling unit 408 is used to perform spatial downsampling and channel expansion on the output features of the third-level feature enhancement unit 407.

[0080] The fourth-level feature enhancement unit 409 is used to enhance the output features of the fourth-level downsampling unit 408.

[0081] The spatial pyramid pooling unit 410 is used to perform multi-scale feature fusion and representation enhancement on the output features of the fourth-level feature enhancement unit 409, and output the fourth-level enhanced image features.

[0082] In summary, the feature enhancement unit of this invention utilizes depthwise separable convolution, parallel feature slicing processing of channel splitting units, and cross-layer residual structures to better preserve the texture and structural information of small targets in shallow and mid-layer features. This generates feature maps with different spatial resolutions, such as P3, P4, and P5, each containing local and semantic information of targets at different scales. This allows the target detection model to enhance its fine-grained feature representation capabilities while remaining lightweight. Compared to traditional target detection models, it can more effectively preserve the local texture information of small targets such as distant faces, tiny logos, and key plot props, significantly improving model training stability and small target recall rate, while still achieving higher detection accuracy with a parameter reduction of approximately 40% to 60%.

[0083] Furthermore, the first depthwise separable convolutional unit and the second depthwise separable convolutional unit used in this embodiment of the invention both include spatial convolution and pointwise convolution. Spatial convolution independently performs spatial convolution operations on each channel of the input feature map, extracting spatial features within each channel without cross-channel information interaction; pointwise convolution, on the other hand, linearly combines the multi-channel features output by spatial convolution in the channel dimension through 1×1 convolution, achieving cross-channel feature fusion.

[0084] Unlike standard convolution, which simultaneously extracts spatial features and fuses channel information, depthwise separable convolution decomposes these two functions into two independent steps. In terms of computational complexity, standard convolution can be represented as the product of the number of input channels multiplied by the number of output channels, then multiplied by the square of the kernel space size and the feature map space size. In contrast, depthwise separable convolution involves two parts: spatial convolution and pointwise convolution. The spatial convolution part is the product of the number of input channels multiplied by the square of the kernel space size and the feature map space size, while the pointwise convolution part is the product of the number of input channels multiplied by the number of output channels and the feature map space size. Since in most cases the kernel space size is much smaller than the product of the number of output channels and the square of the kernel space size, this decomposition significantly reduces the overall computational complexity, improving computational efficiency while maintaining the model's expressive power.

[0085] The feature fusion network in this embodiment of the invention is used to fuse the enhanced image features of the multiple levels to obtain fused features of the multiple levels.

[0086] In one optional implementation, the multiple levels of enhanced image features include: first-level enhanced image features, second-level enhanced image features, ..., Nth-level enhanced image features arranged in chronological order of generation, where N is a positive integer; The feature fusion network specifically includes: top-down fusion links and bottom-up fusion links; The top-down fusion link is used to fuse enhanced image features from multiple levels in order from level N to level 2 to obtain intermediate fusion features; the bottom-up fusion link is used to fuse the intermediate fusion features and enhanced image features from multiple levels in order from level 1 to level N to obtain fused features from multiple levels. The top-down fusion link includes a coordinate attention unit, which is used to perform coordinate attention weighting processing on the intermediate feature map after fusing the enhanced image features of two adjacent layers.

[0087] The top-down fusion link, along the direction from level N to level 2, gradually merges high-level features with corresponding low-level features, injecting rich semantic information into shallow features, enhancing the shallow features' ability to represent details of small targets, and providing more refined semantic support for small target detection.

[0088] The bottom-up fusion link, moving from level 1 to level N, progressively merges shallow features with deep features that have already been fused semantic information. This process transmits precise spatial location information to deeper layers, enabling deep features to enhance the perception of small target locations while maintaining high semantic quality, thereby improving positioning accuracy.

[0089] Furthermore, a coordinate attention unit is introduced into the top-down fusion process. The coordinate attention unit processes the intermediate feature maps obtained by fusing adjacent layers to enhance the object detection model's ability to perceive spatial location information.

[0090] The processing procedure of the coordinate attention unit is as follows: First, one-dimensional global average pooling is performed on the intermediate feature map along the horizontal and vertical directions respectively to obtain horizontal and vertical feature vectors, thus aggregating global spatial context information in two orthogonal directions while preserving positional information in each direction. Next, the obtained horizontal and vertical feature vectors are concatenated to obtain a concatenated feature vector, which is then transformed and fused with cross-channel information through a 1×1 convolutional layer to obtain a fused feature vector. Subsequently, the fused feature vector is split into two independent branches: a horizontal branch feature vector and a vertical branch feature vector. Each branch is processed by a 1×1 convolutional layer and a non-linear activation unit to generate a horizontal attention weight map and a vertical attention weight map that match the corresponding channel dimensions of the intermediate feature map. Finally, the generated horizontal and vertical attention weight maps are multiplied element-wise with the original intermediate feature map to obtain a horizontally weighted feature map and a vertically weighted feature map, achieving targeted weighting of features in the horizontal and vertical directions.

[0091] This coordinate attention unit calculates horizontal attention weight maps and vertical attention weight maps along the horizontal and vertical directions respectively for the intermediate feature maps obtained by fusing adjacent layers. This models the spatial position dependency of features, which can effectively enhance the response of features at key spatial locations and improve the accuracy of feature representation. In particular, it helps the target detection model capture the position and structural information of targets in complex scenes, improve the accuracy of multi-scale feature fusion, and make the target detection model more suitable for the high-precision detection needs of small targets on long video platforms.

[0092] The spatial location dependency of features refers to the association and constraint relationships between feature points at different spatial locations in an intermediate feature map. This relationship typically manifests as follows: features within the same target exhibit spatial consistency and continuity; features at different locations are correlated with each other and jointly characterize the target's structure and shape; features in adjacent locations often show strong correlation, exhibiting a locally smooth or structurally continuous feature distribution pattern; while features between the target region and the background or irrelevant regions usually have significant distinguishability, manifested as significant differences in feature responses.

[0093] The coordinate attention unit, by modeling the spatial positional dependencies in the horizontal and vertical directions respectively, can guide the model to focus on the precise spatial location of the target, perceive its spatial range and structural distribution, thereby enhancing the feature response to the target-related region, while suppressing the interference of the background and irrelevant regions.

[0094] In one implementation, N is 4, and the top-down fusion link specifically includes: a first convolutional unit, a first upsampling unit, a first fusion unit, a second convolutional unit, a first feature enhancement unit, a coordinate attention unit, a second upsampling unit, a third convolutional unit, a second fusion unit, and a second feature enhancement unit; The first convolutional unit performs convolution processing on the enhanced image features of the fourth level; the first upsampling unit upsamples the output features of the first convolutional unit; the second convolutional unit performs convolution processing on the enhanced image features of the third level; the first fusion unit concatenates the output features of the first upsampling unit and the output features of the second convolutional unit; the first feature enhancement unit enhances the output features of the first fusion unit; the coordinate attention unit performs coordinate attention weighting processing on the intermediate feature map output by the first feature enhancement unit; the second upsampling unit performs convolution processing on the output features of the coordinate attention unit; the third convolutional unit performs convolution processing on the enhanced image features of the second level; the second fusion unit concatenates the output features of the second upsampling unit and the output features of the third convolutional unit; and the second feature enhancement unit enhances the output features of the second fusion unit to obtain fused intermediate features.

[0095] The first, second, and third convolutional units perform convolution processing on the enhanced image features of their respective levels, extracting local spatial features and unifying or adjusting the channel dimensions of the features.

[0096] The first and second upsampling units can employ structures with upsampling capabilities (such as transposed convolution structures, bilinear interpolation structures, etc.) to enhance the spatial resolution of the input features, aligning them with the resolution of the features at the level to be fused. For example, features at the fourth level can be upsampled to match the spatial size of features at the third level.

[0097] The feature enhancement processes corresponding to the first feature enhancement unit and the second feature enhancement unit specifically include: downsampling the input first image features to obtain the corresponding second image features; performing a split convolution process on the second image features to obtain the third image features; dividing the third image features into a first branch feature and a second branch feature; performing a bi-branch feature interaction process on the first branch feature to obtain an interaction-processed feature; concatenating the interaction-processed feature with the second branch feature to obtain a concatenated feature; and performing a second split convolution process on the concatenated feature to obtain the enhanced image features.

[0098] The first feature enhancement unit and the second feature enhancement unit can adopt... Figure 3 The structure of the feature enhancement unit shown enhances the input features, improves their feature representation ability, and focuses on preserving the texture and structural information of small targets.

[0099] In the top-down fusion link processing flow, the fourth-level enhanced image features are processed by the first convolutional unit, then upsampled by the first upsampling unit, and concatenated with the third-level enhanced image features processed by the second convolutional unit in the first fusion unit. The concatenated features are then successively enhanced by the feature enhancement of the first feature enhancement unit and weighted by the spatial position of the coordinate attention unit, then upsampled by the second upsampling unit, and concatenated again with the second-level features processed by the third convolutional unit in the second fusion unit. Finally, the fused intermediate features are obtained through the second feature enhancement unit.

[0100] The top-down fusion link achieves the transfer and fusion of high-level semantic information to low-level features through hierarchical top-down fusion. Combined with feature enhancement and coordinate attention mechanisms, it strengthens the spatial location perception and detail preservation capabilities of features, thereby improving the accuracy of multi-scale feature fusion and enhancing the target detection model's performance in detecting small targets.

[0101] In another implementation, the bottom-up fusion link includes: a fourth convolutional unit, a third fusion unit, a third feature enhancement unit, a first downsampling unit, a fourth fusion unit, a fourth feature enhancement unit, a second downsampling unit, a fifth fusion unit, and a fifth feature enhancement unit; The fourth convolutional unit is used to perform convolution processing on the first-level enhanced image features; the third fusion unit is used to concatenate the fused intermediate features with the output features of the fourth convolutional unit; the third feature enhancement unit is used to perform feature enhancement processing on the output features of the third fusion unit; the first downsampling unit is used to downsample the output features of the third feature enhancement unit; the fourth fusion unit is used to concatenate the output features of the first downsampling unit with the output features of the first feature enhancement unit; the fourth feature enhancement unit is used to perform feature enhancement processing on the output features of the fourth fusion unit; the second downsampling unit is used to downsample the output features of the fourth feature enhancement unit; the fifth fusion unit is used to concatenate the output features of the second downsampling unit with the output features of the first convolutional unit; the fifth feature enhancement unit is used to perform feature enhancement processing on the output features of the fifth fusion unit. The fusion features of the multiple levels include: the output features of the third feature enhancement unit, the output features of the fourth feature enhancement unit, and the output features of the fifth feature enhancement unit.

[0102] In the bottom-up fusion chain, the enhanced image features at each level are gradually fused through operations such as convolution, stitching, enhancement, and downsampling, realizing the step-by-step information transmission and aggregation from shallow detail features to deep semantic features.

[0103] The fourth convolutional unit is used to perform convolution processing on the enhanced image features of the corresponding level, extract local spatial features, and unify or adjust the channel dimensions of the features.

[0104] The first and second downsampling units employ a structure with downsampling functionality to reduce the spatial resolution of the input features, aligning them spatially with the higher-level enhanced image features to be fused. For example, the first-level enhanced image features are downsampled to match the size of the second-level enhanced image features.

[0105] The third, fourth, and fifth feature enhancement units adopt the feature enhancement unit structure shown in Figure 3 to enhance the spliced ​​fused features, improve the feature expression capability, and retain the texture and structural details of small targets.

[0106] The bottom-up fusion link's processing flow is as follows: The first-level enhanced image features are processed by the fourth convolutional unit and then concatenated with the intermediate fusion features output from the top-down fusion link in the third fusion unit. The concatenated features are then enhanced by the third feature enhancement unit, then their resolution is reduced by the first downsampling unit, and finally concatenated with the output features of the first feature enhancement unit in the fourth fusion unit. This concatenated feature is processed by the fourth feature enhancement unit, further downsampled by the second downsampling unit, and then concatenated with the output features of the first convolutional unit in the fifth fusion unit. Finally, the fifth feature enhancement unit processes the fused output features. This link ultimately outputs multiple levels of fused features, namely the output features of the third, fourth, and fifth feature enhancement units.

[0107] The bottom-up fusion path achieves the transfer and aggregation of low-level detailed information to high-level features through bottom-up, step-by-step fusion. Combined with multiple feature enhancement processes, it improves the expressive power and detail preservation ability of features. It complements the top-down fusion path and together improves the model's detection performance for multi-scale targets, especially small targets.

[0108] Optionally, in this embodiment of the invention, the top-down fusion link and the bottom-up fusion link include fusion units, such as the first fusion unit, the second fusion unit, the third fusion unit, the fourth fusion unit, and the fifth fusion unit mentioned above.

[0109] The fusion unit configures corresponding weight parameters for each input feature; during the training of the object detection model, the weight parameters corresponding to each input feature are iteratively updated. Specifically, the fusion unit includes: a first and second fusion unit in a top-down fusion chain, and a third, fourth, and fifth fusion unit in a bottom-up fusion chain.

[0110] The input features for each path include: transformed image features obtained by performing image transformation on enhanced image features of one level, or transformed image features obtained by performing image transformation on the fusion result of enhanced image features of multiple levels.

[0111] For example, Figure 5 In this process, each input feature of the first fusion unit 503 includes: the output feature of the first upsampling unit 502 and the output feature of the second convolution unit 504. The output feature of the first upsampling unit 502 is the transformed image feature corresponding to the enhanced image feature of the fourth level; the output feature of the second convolution unit 504 is the transformed image feature corresponding to the enhanced image feature of the third level.

[0112] Each input feature of the second fusion unit 509 includes: the output feature of the second upsampling unit 507 and the output feature of the third convolution unit 508. The output feature of the second upsampling unit 507 is the transformed image feature corresponding to the fusion result of the enhanced image features of the third level and the enhanced image features of the fourth level. The output feature of the third convolution unit 508 is the transformed image feature corresponding to the enhanced image features of the second level.

[0113] In this embodiment of the invention, in each fusion unit of the top-down and bottom-up fusion chain, a corresponding learnable weight parameter is set for each input feature. During model training, these weight parameters are iteratively updated through backpropagation of the loss function, thereby adaptively learning the relative importance of different input features in the fusion process.

[0114] In the process of weighted fusion of two features A and B in the fusion unit, a weighted normalization fusion method can be adopted: F = (W1) A + W2 B) / (W1 + W2 + ε), where W1 and W2 are the learnable weight parameters for the corresponding input features A and B, respectively, and ε is a very small positive number used to prevent the denominator from being zero. Through this mechanism, the object detection model can automatically optimize the distribution of weight parameters during training, so that features that contribute more to the detection of small objects receive higher fusion weights.

[0115] By adaptively learning the weight parameters of each input feature, the object detection model can achieve a more reasonable weighting allocation during feature fusion, strengthening the shallow detail information and spatial location information that play a crucial role in the detection of small objects. Combined with a bidirectional fusion path, the adaptive learning of weight parameters achieves full interaction and balanced integration of deep semantic information and shallow detail information, improving the rationality and effectiveness of multi-scale feature fusion, and further enhancing the model's detection accuracy and robustness for small objects.

[0116] Reference Figure 5 The diagram shows a schematic of the structure of a feature fusion network according to an embodiment of the present invention. The feature fusion network specifically includes: a top-down fusion link A and a bottom-up fusion link B. The top-to-bottom fusion link A specifically includes: a first convolutional unit 501, a first upsampling unit 502, a first fusion unit 503, a second convolutional unit 504, a first feature enhancement unit 505, a coordinate attention unit 506, a second upsampling unit 507, a third convolutional unit 508, a second fusion unit 509, and a second feature enhancement unit 510. The bottom-up fusion link B includes: a fourth convolutional unit 511, a third fusion unit 512, a third feature enhancement unit 513, a first downsampling unit 514, a fourth fusion unit 515, a fourth feature enhancement unit 516, a second downsampling unit 517, a fifth fusion unit 518, and a fifth feature enhancement unit 519.

[0117] In the top-down fusion link A, the first convolutional unit 501 is used to perform convolution processing on the enhanced image features of the fourth level; the first upsampling unit 502 is used to upsample the output features of the first convolutional unit 501; the second convolutional unit 504 is used to perform convolution processing on the enhanced image features of the third level; the first fusion unit 503 is used to concatenate the output features of the first upsampling unit 502 and the output features of the second convolutional unit 504; the first feature enhancement unit 505 is used to perform feature enhancement processing on the output features of the first fusion unit 503; the coordinate attention unit 506 is used to perform coordinate attention weighting processing on the intermediate feature map output by the first feature enhancement unit 505; the second upsampling unit 507 is used to perform convolution processing on the output features of the coordinate attention unit 506; the third convolutional unit 508 is used to perform convolution processing on the enhanced image features of the second level; the second fusion unit 509 is used to concatenate the output features of the second upsampling unit 507 and the output features of the third convolutional unit 508; the second feature enhancement unit 510... This is used to perform feature enhancement processing on the output features of the second fusion unit 509 to obtain fused intermediate features.

[0118] In the bottom-up fusion link B, the fourth convolutional unit 511 is used to perform convolution processing on the first-level enhanced image features; the third fusion unit 512 is used to concatenate the fused intermediate features with the output features of the fourth convolutional unit 511 and the output features of the third convolutional unit 508; the third feature enhancement unit 513 is used to perform feature enhancement processing on the output features of the third fusion unit 512; the first downsampling unit 514 is used to downsample the output features of the third feature enhancement unit 513; the fourth fusion unit 515 is used to concatenate the output features of the first downsampling unit 514 with the output features of the first feature enhancement unit 505 and the second convolutional unit; the fourth feature enhancement unit 516 is used to perform feature enhancement processing on the output features of the fourth fusion unit 515; the second downsampling unit 517 is used to downsampling the output features of the fourth feature enhancement unit 516; and the fifth fusion unit 518 is used to concatenate the output features of the second downsampling unit 517 with the output features of the first convolutional unit 501. The output features are spliced ​​together; the fifth feature enhancement unit 519 is used to perform feature enhancement processing on the output features of the fifth fusion unit 518.

[0119] The multiple levels of fusion features include: the first-level fusion feature output by the third feature enhancement unit 513, the second-level fusion feature output by the fourth feature enhancement unit 516, and the third-level fusion feature output by the fifth feature enhancement unit 519.

[0120] The detection network is used to obtain target detection results based on the fusion features of the multiple levels. The number of detection networks can be the same as the number of fusion features; for example, if the number of fusion features is 3, then there can be 3 detection networks.

[0121] In one alternative implementation, this invention provides a lightweight detection network to achieve accurate localization and classification of small targets. This detection network combines the concept of a decoupled head, separating the classification and regression branches, which not only improves the stability of model training but also accelerates the inference process, further improving the detection accuracy and efficiency of small targets such as distant faces, logos, and props.

[0122] The detection network specifically includes: a channel adaptation unit, a target feature extraction unit, a classification branch unit, a bounding box regression unit, and a prediction output unit; The channel adaptation unit includes a first pointwise convolutional unit, a first batch normalization unit, and a first activation unit, used to perform channel adaptation on the fusion features of the corresponding level. The target feature extraction unit includes: a third depthwise separable convolutional unit, a second batch normalization unit, and a second activation unit, used to determine semantic features and spatial features based on the output features of the channel adaptation unit; The classification branch unit includes: a fourth depthwise separable convolutional unit, a channel attention unit, and a second pointwise convolutional unit, used to extract semantic features of the target category and output the category prediction result based on the output features of the target feature extraction unit; The bounding box regression unit includes a fifth depthwise separable convolutional unit and a third pointwise convolutional unit, which are used to output the bounding box regression result of the target category based on the output features of the target feature extraction unit. The prediction output unit is used to output the target detection result based on the output features of the classification branch unit and the output features of the bounding box regression unit.

[0123] The channel adaptation unit is used to adapt and integrate the channel dimensions of the input fused features at different levels. This unit sequentially includes a first pointwise convolutional unit, a first batch of normalization units, and a first activation unit. The first pointwise convolutional unit adjusts the input features to a uniform target number of channels, achieving channel standardization of fused features at different levels. The first batch of normalization units normalizes the output of the first pointwise convolutional unit, reducing internal covariate bias and improving model training stability. The first activation unit introduces a non-linear transformation to further enhance feature representation and filter redundant information. The final output is the channel-adapted feature, which is used by the downstream target feature extraction unit. The first pointwise convolutional unit can be a 1×1 pointwise convolution.

[0124] The target feature extraction unit takes the output of the channel adaptation unit as input and extracts discriminative semantic and spatial features. This unit specifically includes: a third depthwise separable convolutional unit, a second batch normalization unit, and a second activation unit. The third depthwise separable convolutional unit efficiently extracts spatial structure information and inter-channel relationships from features with low computational cost, balancing lightweight design with effective feature extraction. The second batch normalization unit normalizes the output features of the third depthwise separable convolutional unit, accelerating model training convergence and preventing gradient explosion or vanishing. The second activation unit enhances the non-linear expression of features, improving the discriminative power of semantic and spatial features, facilitating feature utilization in subsequent classification and regression tasks. The final output is a target feature containing both semantic and spatial information, providing a basic feature representation for the classification branch unit and the bounding box regression unit.

[0125] The classification branch unit, based on the output features of the target feature extraction unit, focuses on predicting the target's category. This unit sequentially includes a fourth deep separable convolutional unit, a channel attention unit, and a second pointwise convolutional unit. The fourth deep separable convolutional unit selectively extracts semantic features relevant to category discrimination, effectively suppressing interference from irrelevant spatial noise; the channel attention unit adaptively weights features from different channels, highlighting feature channels that play a crucial role in category discrimination and weakening the influence of ineffective channels; the second pointwise convolutional unit maps the weighted semantic features to the target category dimension, outputting prediction scores for each category, providing a basis for target category determination.

[0126] The bounding box regression unit takes the output features of the target feature extraction unit as input and focuses on predicting the location parameters of the target bounding box. This unit sequentially contains a fifth depthwise separable convolutional unit and a third pointwise convolutional unit. The fifth depthwise separable convolutional unit focuses on extracting spatial features related to the target location, deliberately downplaying the interference of category semantic information to ensure the accuracy of location prediction; the third pointwise convolutional unit maps the extracted spatial features to bounding box regression parameters, including the target center point coordinates, width and height offsets, etc., to achieve precise target localization.

[0127] The prediction output unit integrates the category prediction results from the classification branch unit with the position parameters from the bounding box regression unit to generate the final target detection result. This unit first performs threshold filtering on the category prediction scores output by the classification branch unit, eliminating predictions with excessively low confidence. Then, it uses non-maximum suppression to eliminate overlapping and redundant bounding boxes. Finally, it combines the regression parameters output by the bounding box regression unit to decode the accurate coordinates of the target bounding box, ultimately outputting a structured detection result containing the target category, bounding box position, and confidence level, thus achieving accurate localization and classification of small targets.

[0128] This detection network effectively reduces the number of model parameters and computational complexity through a lightweight structure design that combines depthwise separable convolutions and pointwise convolutions. At the same time, it uses decoupled branches for classification and regression to extract task-specific features, reducing redundant computations and feature conflicts. This allows the network to achieve lightweight, lower computational load, and faster inference speed while ensuring the accuracy and stability of small target detection.

[0129] This invention also provides a target detection method, which can be used to identify targets of target categories from an image to be detected and determine the specific location of each target in the image. The location of each target in the output result is usually marked in the form of a bounding box. A bounding box is a rectangular area used to define the spatial range of a target in the image to be detected. It is usually defined by the coordinates of the upper left and lower right corners of the rectangle, thereby accurately representing the position and size of the target in the image to be detected.

[0130] The embodiments of the present invention do not limit the specific application scenarios of the object detection method. For example, when the image to be detected originates from a video, the object detection method can be applied to application scenarios such as video recommendation, video search, video tag generation, content moderation, and video understanding and generation.

[0131] In video recommendation scenarios, object detection provides video platforms with fine-grained content understanding by identifying key elements such as people, props, and brand logos that exist as small objects in the video. This supports accurate recommendations based on entity elements. The recall rate of small object detection directly affects the completeness of the video content structure and the upper limit of the recommendation effect.

[0132] In video search scenarios, object detection can locate and associate small targets such as distant faces and small props, and support spatiotemporal retrieval tasks such as "the first appearance location of a character", achieving shot-level accurate search.

[0133] In video tag generation scenarios, stable recognition of small targets helps video platforms capture long-tail visual entities such as distant figures and miniature props, thereby improving the coverage and accuracy of the tagging system and providing structured input for multimodal understanding.

[0134] In content moderation scenarios, high-precision small target detection can effectively identify risk elements such as tiny illegal items and sensitive symbols in the image, reduce the false negative rate, control the marginal cost of moderation, and is a key technology for large-scale risk control.

[0135] In video understanding and generation scenarios, continuous detection and cross-frame correlation of small targets can provide generative AI (Artificial Intelligence) with fine-grained visual information such as entity trajectories and temporal actions, supporting high-level semantic tasks such as plot summarization and character behavior question answering, and is one of the basic capabilities driving multimodal reasoning.

[0136] Reference Figure 6 The diagram illustrates a step-by-step flowchart of a target detection method according to an embodiment of the present invention. The method may specifically include the following steps: Step 601: Obtain the image to be detected; Step 602: Using the trained target detection model, perform target detection on the image to be detected to obtain the target detection result corresponding to the target object in the image to be detected.

[0137] The target object can refer to the entity in the image to be detected that needs to be identified and located.

[0138] For example, in video content understanding scenarios (such as video recommendation, video search, and video tag generation), common target objects include people, specific props, and brand logos appearing in the video. In content security review scenarios, target objects typically involve prohibited items, sensitive symbols, or inappropriate content elements in the video. Sensitive symbols refer to visual symbols, texts, signs, etc., in the video that contain illegal, misleading, or non-compliant content, such as illegal trademarks, inappropriate patterns, and prohibited symbols.

[0139] Figure 6 The target detection method embodiments shown can be executed by either a client or a server. It is understood that the embodiments of the present invention are for... Figure 6 The specific entity that performs the method embodiment shown is not limited.

[0140] Step 601 is used to acquire the image data to be analyzed. Depending on the actual application scenario, the source of the image to be detected may include: real-time acquisition by the client through a camera (such as real-time video surveillance footage, images containing small distant targets captured by mobile devices); image files uploaded by users and stored locally (such as product images uploaded by e-commerce platforms for logo detection, historical monitoring frames retrieved from security systems); or video frames extracted frame by frame from video platforms or video files (such as short video content review, detection of small target logo props in videos). The server can obtain input by receiving image data streams transmitted by the client, reading image files from the storage system, or parsing video streams, thereby providing a clear input object for subsequent processing.

[0141] Step 602: Perform target detection on the image to be detected based on the target detection model to obtain the target detection result corresponding to the target object in the image to be detected.

[0142] In addition, embodiments of the present invention can also adapt the target detection results to the application scenario.

[0143] In client-side scenarios, bounding boxes and category labels can be rendered and overlaid on the image to be detected through a graphical interface.

[0144] In server-side scenarios, target detection results, including target category, bounding box coordinates, and confidence level, can be written into a database for retrieval and analysis, or the target detection results can be returned to the client for further processing via an interface.

[0145] In the video file scenario of the video platform, the bounding box is mapped to the original video frame. The target position is selected on the original video frame, and the category and confidence are labeled. The processed video frame with the target category, bounding box position and confidence label is output. The processed video frame contains the precise location of the target, the category name and the confidence value, which can be directly connected to the target detection system.

[0146] Optionally, the process of performing bi-branch feature interaction processing on the first branch features specifically includes: performing convolution processing on the first branch features to obtain a feature map; dividing the feature map into a third branch feature map and a fourth branch feature map according to the channel dimension; performing convolution transformation on the third branch feature map and the fourth branch feature map respectively to obtain two transformed feature maps; performing element-wise multiplication on the two transformed feature maps to obtain a fused feature map; performing dimensionality reduction processing on the fused feature map, and performing residual connection between the dimensionality-reduced fused feature map and the first branch features to obtain the interactive processing features.

[0147] Optionally, the enhanced image features include: first-level enhanced image features, second-level image features, ..., Nth-level enhanced image features arranged in chronological order of generation time, where N is a positive integer; The target detection model further fuses the enhanced image features of multiple levels in the order from level N to level 2 to obtain fused intermediate features, and fuses the fused intermediate features and the enhanced image features of multiple levels in the order from level 1 to level N to obtain fused features of multiple levels. In the process of fusing enhanced image features from multiple levels in the order from level N to level 2, the enhanced image features of two adjacent levels are fused to obtain an intermediate feature map, and the intermediate feature map is then subjected to coordinate attention weighting processing.

[0148] In an example of a target detection method according to an embodiment of the present invention, the method may specifically include the following steps: Step S1, Input Processing: The original video frame after the video stream / video file is parsed by the decoder is used as input. The original video frame is two-dimensional image data in original pixel format.

[0149] Step S2, Image Preprocessing: Perform the set preprocessing operations on the input raw video frames, including size normalization, pixel value normalization, color space conversion, and data augmentation (motion blur simulation, compression artifact simulation, random brightness perturbation) according to the characteristics of the video scene, and output a standardized image tensor that conforms to the model input specifications.

[0150] Step S3, Feature Extraction: Input the standardized image tensor into the feature extraction network, and the feature extraction network outputs feature maps with different spatial resolutions such as P3, P4, and P5.

[0151] Step S4, Multi-scale Feature Fusion: Input P3, P4, and P5 into the feature fusion network, and perform bidirectional cross-scale weighted fusion from "top-down" and "bottom-up" (the fusion nodes are assigned learnable weights, and shallow features sensitive to small targets are assigned higher weights), outputting enhanced feature maps N3, N4, and N5 (corresponding one-to-one with the scales of P3, P4, and P5). These enhanced feature maps simultaneously possess detailed and semantic information.

[0152] Step S5, Detection Inference: Input N3, N4, and N5 into the corresponding three detection networks. The classification branch of each detection network outputs the class probability, and the regression branch outputs the bounding box position and confidence score. The three detection networks jointly generate the full-scale original detection result (including all bounding boxes, position parameters, confidence scores, and class probabilities). A bounding box is a rectangular label used to locate the target position in an image or video frame, and its coordinate parameters describe the area range of the target.

[0153] Step S6, Post-processing of results: Perform uniform processing on the original detection results at all scales. First, filter by confidence threshold (filter low confidence candidate boxes), then remove overlapping redundant boxes by non-maximum suppression, and finally obtain the target bounding box.

[0154] Step 7, Result Output: Map the target bounding box to the original video frame, select the target location on the original video frame, label the category and confidence level, and output the processed video frame with the target category, bounding box location and confidence level labels. The processed video frame contains the target's location, category name and confidence level value, and can be directly connected to the target detection system.

[0155] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the present invention.

[0156] Based on the description of the above method embodiments, the present invention also provides corresponding target detection device embodiments to achieve the content described in the above method embodiments.

[0157] Reference Figure 7 The diagram illustrates a structural schematic of a training device for an object detection model according to an embodiment of the present invention. The device specifically includes the following modules: The acquisition module 701 is used to acquire a training sample set; the training sample set includes: multiple image samples, and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; The detection module 702 is used to determine the target detection result corresponding to the image sample using a target detection model. Specifically, during feature extraction of the image sample, the target detection model downsamples a first image feature to obtain a corresponding second image feature; performs a split convolution on the second image feature to obtain a third image feature; divides the third image feature into a first branch feature and a second branch feature; performs a bi-branch feature interaction process on the first branch feature to obtain an interactively processed feature; the bi-branch feature interaction process is used to divide the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and performs interactive processing on the first branch feature map and the second branch feature map; concatenates the interactively processed feature with the second branch feature to obtain a concatenated feature; performs a second split convolution on the concatenated feature to obtain an enhanced image feature; and the target detection model determines the target detection result corresponding to the image sample based on the enhanced image feature. The loss determination module 703 is used to determine loss information based on the target detection result and classification label corresponding to the image sample; The parameter update module 704 is used to obtain the model update parameters of the target detection model based on the loss information.

[0158] Optionally, the process of performing bi-branch feature interaction processing on the first branch features specifically includes: performing convolution processing on the first branch features to obtain a feature map; dividing the feature map into a third branch feature map and a fourth branch feature map according to the channel dimension; performing convolution transformation on the third branch feature map and the fourth branch feature map respectively to obtain two transformed feature maps; performing element-wise multiplication on the two transformed feature maps to obtain a fused feature map; performing dimensionality reduction processing on the fused feature map, and performing residual connection between the dimensionality-reduced fused feature map and the first branch features to obtain the interactive processing features.

[0159] Optionally, the enhanced image features include: first-level enhanced image features, second-level image features, ..., Nth-level enhanced image features arranged in chronological order of generation time, where N is a positive integer; The target detection model further fuses the enhanced image features of multiple levels in the order from level N to level 2 to obtain fused intermediate features, and fuses the fused intermediate features and the enhanced image features of multiple levels in the order from level 1 to level N to obtain fused features of multiple levels. In the process of fusing enhanced image features from multiple levels in the order from level N to level 2, the enhanced image features of two adjacent levels are fused to obtain an intermediate feature map, and the intermediate feature map is then subjected to coordinate attention weighting processing.

[0160] Optionally, the target detection model includes a fusion unit, which is used to fuse multiple input features during the fusion of enhanced image features at multiple levels, and the fusion unit configures corresponding weight parameters for each input feature. The input features for each path include: transformed image features obtained by performing image transformation on enhanced image features of one level, or transformed image features obtained by performing image transformation on the fusion result of enhanced image features of multiple levels.

[0161] Optionally, the parameter update module is specifically used to update the weight parameters corresponding to each input feature based on the loss information.

[0162] Reference Figure 8 The diagram illustrates the structure of a target detection device according to an embodiment of the present invention, the device specifically comprising the following modules: The acquisition module 801 is used to acquire the image to be detected; The target detection module 802 is used to perform target detection on the image to be detected using a trained target detection model, and obtain the target detection result corresponding to the target object in the image to be detected.

[0163] This invention discloses an electronic device, including a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the aforementioned method.

[0164] This invention discloses a non-transitory computer-readable storage medium storing instructions that cause a processor to execute the aforementioned method.

[0165] Non-transitory memory: This refers to components or devices in computer hardware used for non-temporary storage of data and instructions. It emphasizes the "non-transitory" nature of the storage, meaning the data will not be lost due to momentary factors such as the disappearance of electrical signals. Examples include ROM (Read-Only Memory) and the storage chips in solid-state drives (SSDs), which can store data long-term.

[0166] Non-transitory computer-readable recording media: Physical carriers that can permanently store computer data and are readable. Data storage is stable and not temporary. Examples include hard drives, USB flash drives, and optical discs. They can store various types of data such as programs, documents, and videos. Even if the device is powered off or restarted, the data is still retained and can be accessed by the computer later.

[0167] Non-transitory computer-readable storage media: These are part of the computer storage system and are media that can permanently store computer-readable data. They include hard drives, solid-state drives, and flash memory. Unlike transient storage, they can permanently or long-term retain data. Computers can read the stored instructions and data through corresponding interfaces and protocols for use in program execution, data processing, and other scenarios. For example, they store operating systems, application code, and user files, providing stable data support for the computer system.

[0168] Devices used for target detection can be electronic devices. Figure 9 A schematic diagram of an electronic device 1100 according to an embodiment of the present invention is shown. The electronic device 1100 specifically includes: one or more processors 1102, a control module (chipset) 1104 coupled to at least one of the processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0169] Processor 1102 may include one or more single-core or multi-core processors, and processor 1102 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, electronic device 1100 can serve as a terminal device, server (cluster), or other device as described in the embodiments of the present invention.

[0170] In some embodiments, electronic device 1100 may include one or more computer-readable media (e.g., memory 1106 or non-volatile memory / storage device 1108) having instructions 1114 and one or more processors 1102 that are combined with the one or more computer-readable media and configured to execute instructions 1114 to implement modules and thus perform the actions described in this disclosure.

[0171] In one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1102 and / or any suitable device or component communicating with the control module 1104.

[0172] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0173] Memory 1106 may be used, for example, to load and store data and / or instructions 1114 for electronic device 1100. In one embodiment, memory 1106 may include any suitable volatile memory, such as suitable DRAM (Dynamic Random Access Memory). In some embodiments, memory 1106 may include double data rate type quad synchronous dynamic random access memory.

[0174] In one embodiment, the control module 1104 may include one or more input / output controllers to provide an interface to the non-volatile memory / storage device 1108 and (one or more) input / output devices 1110.

[0175] For example, non-volatile memory / storage device 1108 may be used to store data and / or instructions 1114. Non-volatile memory / storage device 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives, one or more optical disk drives, and / or one or more digital universal optical disk drives).

[0176] The non-volatile memory / storage device 1108 may include storage resources that are physically part of a device on which the electronic device 1100 is mounted, or that can be accessed by the device without being part of the device. For example, the non-volatile memory / storage device 1108 may be accessed via a network via one or more input / output devices 1110.

[0177] One or more input / output devices 1110 may provide an interface for electronic device 1100 to communicate with any other suitable device. Input / output devices 1110 may include communication components, audio components, sensor components, etc. Network interface 1112 may provide an interface for electronic device 1100 to communicate via one or more networks. Electronic device 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 2G (2-Generation wireless telephone technology), 3G (3-Generation wireless telephone technology), 4G (4-Generation wireless telephone technology), 5G (5-Generation wireless telephone technology), etc., or combinations thereof.

[0178] In one embodiment, at least one of the processors 1102 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1104. In one embodiment, at least one of the processors 1102 may be logically packaged with one or more controllers of the control module 1104 to form a system-in-package. In one embodiment, at least one of the processors 1102 may be integrated with the logic of one or more controllers of the control module 1104 on the same die. In one embodiment, at least one of the processors 1102 may be integrated with the logic of one or more controllers of the control module 1104 on the same die to form a system-on-a-chip.

[0179] In various embodiments, electronic device 1100 may be, but is not limited to, a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, touchscreen device, netbook, etc.). In various embodiments, electronic device 1100 may have more or fewer components and / or different architectures. For example, in some embodiments, electronic device 1100 includes one or more cameras, a keyboard, a liquid crystal display screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0180] This invention provides a machine-readable medium storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments.

[0181] Optionally, the machine-readable medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0182] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0183] It will be readily apparent to those skilled in the art that any combination of the above embodiments is feasible, and therefore any combination of the above embodiments constitutes an implementation of the present invention. However, due to space limitations, each embodiment will not be described in detail here. Although preferred embodiments of the present invention have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0184] The above provides a detailed description of the target detection method, apparatus, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A training method for an object detection model, characterized in that, include: Obtain the training sample set; The training sample set includes: multiple image samples, and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; Using an object detection model, the object detection result corresponding to the image sample is determined. Specifically, during feature extraction of the image sample, the object detection model downsamples a first image feature to obtain a corresponding second image feature; performs a split convolution on the second image feature to obtain a third image feature; segments the third image feature into a first branch feature and a second branch feature according to the channel dimension; performs a bi-branch feature interaction process on the first branch feature to obtain an interactively processed feature; the bi-branch feature interaction process is used to segment the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and performs interactive processing on the first branch feature map and the second branch feature map; concatenates the interactively processed feature with the second branch feature to obtain a concatenated feature; performs a second split convolution on the concatenated feature to obtain an enhanced image feature; and the object detection model determines the object detection result corresponding to the image sample based on the enhanced image feature. Based on the target detection results and classification labels corresponding to the image samples, the loss information is determined; Based on the loss information, the model update parameters of the target detection model are obtained.

2. The method according to claim 1, characterized in that, The step of performing bi-branch feature interaction processing on the first branch features includes: The first branch features are convolved to obtain a feature map; The feature map is segmented into a third branch feature map and a fourth branch feature map according to the channel dimension; Convolution transformations are performed on the feature maps of the third and fourth branches respectively to obtain two transformed feature maps. Element-wise multiplication is performed on the two transformed feature maps to obtain a fused feature map; The fused feature map is subjected to dimensionality reduction processing, and the dimensionality-reduced fused feature map is residually connected with the first branch feature to obtain the interactive processing feature.

3. The method according to claim 1, characterized in that, The enhanced image features include: the first-level enhanced image features, the second-level image features, ..., the Nth-level enhanced image features arranged in chronological order of generation time, where N is a positive integer; The target detection model further fuses the enhanced image features of multiple levels in the order from level N to level 2 to obtain fused intermediate features, and fuses the fused intermediate features and the enhanced image features of multiple levels in the order from level 1 to level N to obtain fused features of multiple levels. In the process of fusing enhanced image features from multiple levels in the order from level N to level 2, the enhanced image features of two adjacent levels are fused to obtain an intermediate feature map, and the intermediate feature map is then subjected to coordinate attention weighting processing.

4. The method according to claim 3, characterized in that, The target detection model includes a fusion unit, which is used to fuse multiple input features during the fusion of enhanced image features at multiple levels. The fusion unit configures corresponding weight parameters for each input feature. The input features for each path include: transformed image features obtained by performing image transformation on enhanced image features of one level, or transformed image features obtained by performing image transformation on the fusion result of enhanced image features of multiple levels.

5. The method according to claim 4, characterized in that, The step of obtaining the model update parameters of the target detection model based on the loss information includes: Based on the loss information, the weight parameters corresponding to each input feature are updated.

6. A target detection model, characterized in that, include: Feature extraction network, feature fusion network, and detection network; The feature extraction network is used to extract features from the image to be detected, resulting in enhanced image features at multiple levels. The feature fusion network is used to fuse the enhanced image features at multiple levels to obtain fused features at multiple levels. The detection network is used to obtain target detection results based on the fusion features of the multiple levels; The feature extraction network includes multiple layers of feature extraction units; each layer of feature extraction unit includes a downsampling unit and a feature enhancement unit; the output features of the downsampling unit are input to the feature enhancement unit; the output features of the feature enhancement unit are used as the enhanced image features of the corresponding layer. The feature enhancement unit includes: a first depthwise separable convolutional unit, a channel splitting unit, multiple cascaded bi-branch interactive processing units, a channel fusion unit, and a second depthwise separable convolutional unit; wherein, the channel splitting unit is used to segment the output features of the first depthwise separable convolutional unit into a first branch and a second branch; the first branch is input to at least one bi-branch interactive processing unit, which performs bi-branch feature interaction processing on the corresponding input features; the bi-branch feature interaction processing is used to segment the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and perform interactive processing on the first branch feature map and the second branch feature map; the second branch is input to the channel fusion unit; the channel fusion unit is used to concatenate the output features of the at least one bi-branch interactive processing unit with the second branch, and output the concatenated features to the second depthwise separable convolutional unit.

7. The model according to claim 6, characterized in that, The enhanced image features at multiple levels include: the first-level enhanced image features, the second-level enhanced image features, ..., the Nth-level enhanced image features arranged in chronological order of generation time, where N is a positive integer; The feature fusion network includes: top-down fusion links and bottom-up fusion links; The top-down fusion link is used to fuse enhanced image features from multiple levels in order from level N to level 2 to obtain intermediate fusion features; the bottom-up fusion link is used to fuse the intermediate fusion features and enhanced image features from multiple levels in order from level 1 to level N to obtain fusion features from multiple levels. The top-down fusion link includes: a coordinate attention unit, which is used to perform coordinate attention weighting processing on the intermediate feature map; the intermediate feature map is obtained by fusing enhanced image features from two adjacent layers.

8. The model according to claim 7, characterized in that, The top-down fusion link includes: a first convolutional unit, a first upsampling unit, a first fusion unit, a second convolutional unit, a first feature enhancement unit, a coordinate attention unit, a second upsampling unit, a third convolutional unit, a second fusion unit, and a second feature enhancement unit; The first convolutional unit performs convolution processing on the enhanced image features of the fourth level; the first upsampling unit upsamples the output features of the first convolutional unit; the second convolutional unit performs convolution processing on the enhanced image features of the third level; the first fusion unit concatenates the output features of the first upsampling unit and the output features of the second convolutional unit; the first feature enhancement unit enhances the output features of the first fusion unit; the coordinate attention unit performs coordinate attention weighting processing on the intermediate feature map output by the first feature enhancement unit; the second upsampling unit performs convolution processing on the output features of the coordinate attention unit; the third convolutional unit performs convolution processing on the enhanced image features of the second level; the second fusion unit concatenates the output features of the second upsampling unit and the output features of the third convolutional unit; and the second feature enhancement unit enhances the output features of the second fusion unit to obtain fused intermediate features.

9. The model according to claim 8, characterized in that, The bottom-up fusion link includes: a fourth convolutional unit, a third fusion unit, a third feature enhancement unit, a first downsampling unit, a fourth fusion unit, a fourth feature enhancement unit, a second downsampling unit, a fifth fusion unit, and a fifth feature enhancement unit; The fourth convolutional unit is used to perform convolution processing on the first-level enhanced image features; the third fusion unit is used to concatenate the fused intermediate features with the output features of the fourth convolutional unit; the third feature enhancement unit is used to perform feature enhancement processing on the output features of the third fusion unit; the first downsampling unit is used to downsample the output features of the third feature enhancement unit; the fourth fusion unit is used to concatenate the output features of the first downsampling unit with the output features of the first feature enhancement unit; the fourth feature enhancement unit is used to perform feature enhancement processing on the output features of the fourth fusion unit; the second downsampling unit is used to downsample the output features of the fourth feature enhancement unit; the fifth fusion unit is used to concatenate the output features of the second downsampling unit with the output features of the first convolutional unit; the fifth feature enhancement unit is used to perform feature enhancement processing on the output features of the fifth fusion unit. The fusion features of the multiple levels include: the output features of the third feature enhancement unit, the output features of the fourth feature enhancement unit, and the output features of the fifth feature enhancement unit.

10. The model according to any one of claims 6 to 9, characterized in that, The detection network includes: a channel adaptation unit, a target feature extraction unit, a classification branch unit, a bounding box regression unit, and a prediction output unit; The channel adaptation unit includes a first pointwise convolutional unit, a first batch normalization unit, and a first activation unit, used to perform channel adaptation on the fusion features of the corresponding level. The target feature extraction unit includes: a third depthwise separable convolutional unit, a second batch normalization unit, and a second activation unit, used to determine semantic features and spatial features based on the output features of the channel adaptation unit; The classification branch unit includes: a fourth depthwise separable convolutional unit, a channel attention unit, and a second pointwise convolutional unit, used to extract semantic features of the target category and output the category prediction result based on the output features of the target feature extraction unit; The bounding box regression unit includes a fifth depthwise separable convolutional unit and a third pointwise convolutional unit, which are used to output the bounding box regression result of the target category based on the output features of the target feature extraction unit. The prediction output unit is used to output the target detection result based on the output features of the classification branch unit and the output features of the bounding box regression unit.

11. The model according to any one of claims 6 to 9, characterized in that, The feature extraction network includes four levels of feature extraction units, and the output features of the first three levels of feature enhancement units are used as the enhanced image features of the corresponding levels. The feature extraction network further includes a spatial pyramid pooling unit connected after the feature extraction unit of the fourth level. The spatial pyramid pooling unit is used to perform multi-resolution feature fusion processing on the output features of the feature enhancement unit of the fourth level, and the resulting output features are used as the enhanced image features of the corresponding level.

12. A target detection method, characterized in that, The method includes: Acquire the image to be detected; Using the target detection model trained according to any one of claims 1 to 5, target detection is performed on the image to be detected to obtain the target detection result corresponding to the target object in the image to be detected.

13. A training device for an object detection model, characterized in that, The device includes: The acquisition module is used to acquire a training sample set; the training sample set includes: multiple image samples, and a classification label corresponding to each image sample; the classification label is used to characterize the object category contained in the image sample; A detection module is used to determine the target detection result corresponding to the image sample using a target detection model. Specifically, during feature extraction of the image sample, the target detection model downsamples a first image feature to obtain a corresponding second image feature; performs a split convolution on the second image feature to obtain a third image feature; segments the third image feature into a first branch feature and a second branch feature; performs a bi-branch feature interaction process on the first branch feature to obtain an interactively processed feature; the bi-branch feature interaction process is used to segment the feature map corresponding to the first branch feature into a first branch feature map and a second branch feature map according to the channel dimension, and performs interactive processing on the first branch feature map and the second branch feature map; concatenates the interactively processed feature with the second branch feature to obtain a concatenated feature; performs a second split convolution on the concatenated feature to obtain an enhanced image feature; and the target detection model determines the target detection result corresponding to the image sample based on the enhanced image feature. The loss determination module is used to determine loss information based on the target detection results and classification labels corresponding to the image samples; The parameter update module is used to obtain the model update parameters of the target detection model based on the loss information.

14. A target detection device, characterized in that, The device includes: The acquisition module is used to acquire the image to be detected; The target detection module is used to perform target detection on the image to be detected using a target detection model trained according to any one of the methods in claims 1 to 5, and obtain the target detection result corresponding to the target object in the image to be detected.

15. An electronic device comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 5 and claim 12.

16. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1-5 and claim 12.

Images