A target detection method, an electronic device, and a computer-readable storage medium

Abstract

This application discloses a target detection method, an electronic device, and a computer-readable storage medium. The target detection method includes: dividing the image to be processed according to the region size of the target region where the target object is located in the acquired image to be processed, obtaining at least one image block corresponding to the target region, wherein the image block includes at least one sampling point; performing feature alignment processing on each image block according to the correlation between each sampling point in each image block and its corresponding neighboring pixels, obtaining feature-aligned image blocks; adjusting the feature-aligned image blocks according to the region size of the target region, obtaining an adjusted target region; and performing target detection processing on the adjusted target region to obtain a target detection result. This can improve the accuracy of the target detection result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a target detection method, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of computer vision and deep learning technologies, object detection methods have been widely applied in various fields and have achieved significant breakthroughs. Among them, small-sized object detection is one of the most challenging tasks in the field of object detection. In practical applications, a large amount of key information is presented in the form of small-sized objects, such as distant vehicles or traffic signs in traffic flow detection scenarios, surface defects or tiny foreign objects in industrial inspection scenarios, and early lesions in medical imaging.

[0003] Traditional target detection is usually geared towards larger targets, and it lacks effective attention to smaller targets, resulting in the loss of key information.

[0004] Therefore, there is an urgent need for a target detection method for small-sized targets. Summary of the Invention

[0005] The main technical problem addressed by this application is to provide a target detection method, electronic device, and computer-readable storage medium that can improve the detection accuracy of small-sized targets.

[0006] To solve the above-mentioned technical problems, one technical solution adopted in this application is: to provide a target detection method, the target detection method comprising: The image to be processed is divided into at least one image block corresponding to the target region based on the region size of the target object in the acquired image to be processed. Each image block includes at least one sampling point. Feature alignment is performed on each image block according to the correlation between each sampling point and its corresponding neighboring pixels to obtain feature-aligned image blocks. The feature-aligned image blocks are then adjusted according to the region size of the target region to obtain an adjusted target region. Target detection is performed on the adjusted target region to obtain a target detection result.

[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above-mentioned target detection method.

[0008] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium including program data, which is used to implement the above-mentioned target detection method when executed by a processor.

[0009] The target detection method of this application divides the image into blocks based on the size of the target region where the target object is located in the image to be processed. The size of the image blocks can be adaptively adjusted according to the size of the target object to ensure that each image block contains sufficient image information. Based on the correlation between each sampling point in each image block and its corresponding neighboring pixels, feature alignment processing is performed on each image block to obtain feature-aligned image blocks, ensuring that the boundaries of the image blocks are strictly aligned with the pixel boundaries on the image to be processed and preventing spatial misalignment. The feature-aligned image blocks are then adjusted according to the size of the target region to obtain the adjusted target region. The feature-aligned image blocks are then adjusted again by adjusting the size of the region to further enhance the perception of small-sized targets and avoid over-enhancing large-sized targets, thereby improving the feature representation of the target region and improving the accuracy of target detection. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating an exemplary embodiment of the target detection method shown in this application; Figure 2 yes Figure 1 A flowchart illustrating an exemplary embodiment of step S110 in the target detection method is shown. Figure 3 yes Figure 1 A flowchart illustrating an exemplary embodiment of step S120 in the target detection method is shown. Figure 4 yes Figure 1 A flowchart illustrating an exemplary embodiment of step S130 in the target detection method is shown. Figure 5 yes Figure 1 A flowchart illustrating an exemplary embodiment of step S140 in the target detection method is shown. Figure 6 This is a schematic diagram of an exemplary embodiment of the target detection device shown in this application; Figure 7 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 8 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] First, it's important to clarify that small targets typically refer to objects in the image being processed that occupy a small pixel area. When aligning the target region, the traditional method usually involves pooling. However, when the target region is small, such as covering only a few pixels, the quantization and rounding methods used in pooling can lead to feature misalignment and loss, thus affecting the accuracy of target detection.

[0013] Based on this, embodiments of this application propose a target detection method, electronic device, and computer-readable storage medium. By adaptively adjusting the size of the image blocks in the image to be processed by the region size of the target area, it ensures that target objects of different sizes can obtain high-quality feature representations, thereby improving the accuracy of target detection. For details, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the target detection method shown in this application.

[0014] The execution entity of the target detection method can be a terminal device, a server, or other processing device. The terminal device can be a user equipment (UE), computer, mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The execution entity of the target detection method can also be a target detection device. In some possible implementations, the target detection method can be implemented by a processor calling computer-readable instructions stored in memory.

[0015] Specifically, the target detection method in this embodiment includes the following steps: S110: The image to be processed is divided according to the size of the target region where the target object is located in the acquired image to be processed, so as to obtain at least one image block corresponding to the target region, and the image block includes at least one sampling point.

[0016] The image to be processed refers to the image awaiting target detection. It can be an image selected from an image set, an image acquired by an image acquisition device, a video frame extracted from a video stream acquired by a video acquisition device, or an image submitted by a user. Exemplarily, it can be any image in the image set, or the image with the highest quality in the image set; this application embodiment does not limit this. In some embodiments, the acquired original image can be used as the image to be processed. In other embodiments, the acquired original image can also be preprocessed to obtain the image to be processed. Preprocessing includes, but is not limited to, aligning, cropping, scaling, etc., of the original image to make the image to be processed meet network input requirements.

[0017] The target object in the image to be processed refers to an instance of an object or a region of interest in the image. For example, the target object could be an animal, a vehicle, or a person. The target object can be obtained by detecting the image to be processed using an object detection algorithm. This object detection algorithm can include, but is not limited to, YOLO-V5 (You Only Look Once), YOLO-V4, YOLO-V7, PP-YOLOv2, etc., to accurately identify and locate target objects from images or videos.

[0018] The target region can refer to the image area where the target object is located in the image to be processed. In some embodiments, the target region can be the bounding box obtained by the target detection algorithm detecting the target object in the image to be processed, or the detected bounding box can be optimized to obtain the target region. In other embodiments, the target region can also be determined from the image to be processed by manual calibration. The shape of the target region is generally rectangular, but in other embodiments, the target region can also be pentagonal, circular, or other irregular shapes.

[0019] Region size is used to measure the size of a target region in the image to be processed. In some embodiments, the region size can be determined by statistically analyzing the area occupied by pixels in the target region. In other embodiments, the region size can also be determined by the side lengths of the target region, for example, by taking the maximum side length or the average side length. In still other embodiments, the region size can also be determined by the coordinates of each vertex in the target region, for example, by the distance between vertices. When aligning target regions, if the target region is divided using a standard 7×7 sampling grid, for a target region smaller than 7×7, each sampling grid in the target region covers less than one pixel, resulting in severely insufficient feature extraction accuracy and affecting subsequent processing. Therefore, this application divides the image to be processed by region size to solve the problem of insufficient sampling and low feature extraction accuracy for small targets.

[0020] An image patch is a small block into which a target region is divided. For example, image patches can be obtained using a grid, sliding window, or panning window. Each image patch can be processed and analyzed independently. The size of an image patch is determined by the region size, ensuring that each image patch covers at least one pixel. In some embodiments, a mapping relationship between a preset region size and a preset image patch density can be established. The image patch density corresponding to the region size is determined from the mapping relationship, and the target region containing the target object is divided according to the image patch density to obtain at least one image patch corresponding to the target region. Specifically, feature extraction is performed on the image to be processed to obtain the image features of the image to be processed; the target region containing the target object is determined from the image to be processed based on the image features; the size of the target region is measured to obtain the region size of the target region containing the target object in the image to be processed; then, the image to be processed is divided according to the region size of the target region to obtain at least one image patch corresponding to the target region.

[0021] A sampling point refers to a data point created within an image patch. For example, after dividing the target region into multiple image patches, a fixed number of sampling points are selected within each image patch, such as four sampling points per image patch. The coordinates of the image patch boundaries can be floating-point numbers, and the coordinates of the sampling points can also be floating-point numbers.

[0022] S120: Based on the correlation between each sampling point in each image block and its corresponding neighboring pixels, feature alignment processing is performed on each image block to obtain each image block after feature alignment.

[0023] Neighboring pixels refer to pixels in the image to be processed that are adjacent to the sampling point. It is understood that the image to be processed consists of several pixels. The boundary of the target region extracted from the image to be processed may be located in the middle region of a single pixel and is not aligned with the pixel boundary. Therefore, it is impossible to directly determine the features of each image block in the target region based on the features of each pixel in the image to be processed. Thus, feature alignment processing is required for each image block in the target region to obtain feature-aligned image blocks. In some embodiments, feature alignment is achieved by determining at least one sampling point in each image block and using the correlation between the sampling point and its corresponding neighboring pixels to determine the features corresponding to each sampling point; thereby, the features of the image block are determined based on the features corresponding to each sampling point in the image block, resulting in feature-aligned image blocks. In other embodiments, feature alignment can also be achieved by calculating the features corresponding to each sampling point using bilinear interpolation. The number of neighboring pixels can be one or more, and can be selected according to actual needs. For example, pixels adjacent to the sampling point in any direction can be freely selected as neighboring pixels, or at least one neighboring pixel can be selected according to the distance between each pixel and the sampling point from shortest to farthest.

[0024] The correlation relationship refers to the relationship between a sampling point and its neighboring pixels in an image to be processed. In some embodiments, the correlation relationship is used to characterize the degree of correlation between neighboring pixels and the sampling point. The correlation relationship between the sampling point and its corresponding neighboring pixels is determined based on the degree of feature correlation between the sampling features of the sampling point and the neighborhood features of the corresponding neighboring pixels. For example, if the sampling point is located in the pixel region of the target object and the neighboring pixels are located in the background region, the feature correlation between the two is relatively low. In other embodiments, the correlation relationship includes the spatial correlation relationship between the sampling point and its neighboring pixels. If there are two or more neighboring pixels, the feature of the sampling point is determined based on the distance between the sampling point and each neighboring pixel. The closer the neighboring pixels are, the higher their weight and the greater their influence on the feature of the sampling point.

[0025] Feature-aligned image blocks refer to image blocks that have completed target region alignment. The feature values ​​of feature-aligned image blocks are closer to the true image feature values. In some embodiments, the features of each sampling point in each image block are determined based on the correlation between each sampling point and its corresponding neighboring pixels. Then, the features of the image block are determined based on the features of each sampling point in each image block, resulting in feature-aligned image blocks. For example, max pooling, average pooling, or other methods can be used to process the features of each sampling point in each image block to obtain feature-aligned image blocks.

[0026] S130: Adjust each image block after feature alignment according to the size of the target region to obtain the adjusted target region.

[0027] After obtaining the feature-aligned image blocks, the feature-aligned image blocks can be further adjusted based on the region size of the target region to obtain the adjusted target region. For example, the features of the feature-aligned image blocks can be adjusted based on the region size to obtain the adjusted target region. In some embodiments, a standard region size can be set. When the region size of the target region is greater than or equal to the standard region size, the features of the feature-aligned image blocks are reduced; when the region size of the target region is smaller than the standard region size, the features of the feature-aligned image blocks are increased. In other embodiments, a scale adjustment parameter can be determined based on the region size, and the feature-aligned image blocks can be adjusted based on the scale adjustment parameter to obtain the adjusted target region.

[0028] S140: Perform target detection processing on the adjusted target area to obtain the target detection result.

[0029] The target detection results include, but are not limited to, at least one of the following: information such as the target object's category, action, and clothing. For example, the adjusted target region can be input into a target detection network to obtain the target detection results output by the network.

[0030] As can be seen, the target detection method in this application divides the image into blocks based on the size of the target region where the target object is located in the image to be processed. The density of the image blocks can be adaptively adjusted according to the size of the target object to ensure that target objects of different sizes can obtain high-quality feature representations. Based on the correlation between each sampling point in each image block and its corresponding neighboring pixels, feature alignment processing is performed on each image block to obtain feature-aligned image blocks, so as to ensure that the boundaries of the image blocks are strictly aligned with the pixel boundaries on the image to be processed and to prevent spatial misalignment. The feature-aligned image blocks are adjusted according to the size of the target region to obtain the adjusted target region. The feature-aligned image blocks are adjusted again by adjusting the size of the region to further enhance the perception of small-sized targets and avoid over-enhancing large-sized targets, thereby improving the feature representation of the target region and improving the accuracy of target detection.

[0031] Based on the above embodiments, the embodiments of this application adopt... Figure 2 The flowchart details how the image to be processed is processed to obtain the region size of the target area. Please refer to [link / reference]. Figure 2 , Figure 2 yes Figure 1The illustrated flowchart shows an exemplary embodiment of step S110 in the target detection method. Specifically, step S110, which divides the image to be processed according to the region size of the target region where the target object is located in the acquired image to be processed, to obtain at least one image block corresponding to the target region, further includes the following steps: S210: Perform multi-scale feature extraction on the image to be processed to obtain at least two initial feature maps with different resolutions.

[0032] An initial feature map refers to information extracted from the image to be processed that characterizes its content, such as attributes like texture, shape, edges, and color. For example, an image feature extraction network can be used to extract features from the image to be processed, obtaining at least two initial feature maps. The image feature extraction network may include a shallow feature enhancement backbone network for extracting initial feature maps of different resolutions. As an example, the image feature extraction network may include output features from multiple stages, each with a different downsampling step size, resulting in different resolutions for the output features at each stage. The output features from multiple stages are used as initial feature maps; for example, the total number of stages can be 3 or 5. Furthermore, this embodiment can improve the downsampling rhythm of each stage to better suit the detection of small targets. For example, the resolution of the first N stages can be increased to ensure that small targets retain sufficient response areas on the feature map. N can be 1, 2, etc. For example, the output step size of stage 1 and stage 2 can be adjusted from the standard 8 to 4, and the output feature size of stage 2 can be 1 / 4 of the input feature size, thus retaining an additional layer of high-resolution output features for small target detection.

[0033] In other embodiments, a DetailAwareness Module (DAM) can be introduced into the image feature extraction network. This DAM employs a depthwise separable convolution and residual connection design, enhancing the ability to capture detailed features such as target edges and textures while maintaining a lightweight architecture. Specifically, initial multi-scale feature extraction is performed on the image to be processed to obtain image features at different resolutions. Texture enhancement processing is then applied to the image features at different resolutions to obtain texture-enhanced image features. Finally, residual connections are made between each image feature and its corresponding texture-enhanced image feature to obtain at least two initial feature maps corresponding to the image to be processed.

[0034] Image features of the image to be processed at different resolutions can be extracted by the shallow feature enhancement backbone network in the image feature extraction network. Then, the image features at different resolutions are processed by the detail-aware module to obtain the corresponding initial feature maps.

[0035] The detail-aware module may include convolutional layers and residual connection layers. The convolutional layers are used to enhance the texture information of image features at different resolutions, obtaining image features with enhanced texture information. As an example, the convolutional layers may include pointwise convolution and depthwise separable convolution. For each image feature, pointwise convolution is performed to obtain the pointwise convolutional image feature; the activation function value of the pointwise convolutional image feature is obtained; and depthwise separable convolution is performed on the activation function value of the pointwise convolutional image feature to obtain the image feature with enhanced texture information. The residual connection layer is used to perform residual enhancement on the image features with enhanced texture information and the image features themselves. For example, it can obtain the sum of the image features and the corresponding image features with enhanced texture details to obtain the final initial feature map.

[0036] For example, the formula for the detail-aware module is as follows:

[0037] in, Represents the initial feature map. Represents image features at any resolution. This represents depthwise separable convolution. This represents the activation function. This represents pointwise convolution, where the detail-aware module has only about 1 / 9 the number of parameters of a standard convolution.

[0038] S220: Perform feature recombination processing on each initial feature map to obtain the recombined feature map corresponding to each initial feature map.

[0039] The reconstructed feature map is the feature map obtained by mapping each initial feature map to a unified feature space. For each initial feature map, feature mapping is performed on the other initial feature maps according to the resolution of the initial feature map, resulting in other feature maps after feature mapping; the reconstructed feature map is obtained by fusing the initial feature map with the other feature maps after feature mapping. As an example, assume that at least two initial feature maps include the initial feature map of stage 1. Initial feature map of stage 2 and the initial feature map of stage 3 The feature recombination formula corresponding to the initial feature map is as follows:

[0040] in, Indicates the first The reorganization feature map corresponding to the stage, Represents a 1×1 convolution. This indicates concatenation along the channel dimension. Indicates the first The initial feature map of the stage. Indicates the first The initial feature map of the stage. Indicates the first The initial feature map of the stage. This represents bilinear upsampling. Thus, by upsampling layer by layer, high-level semantic information is fused into low-level high-resolution features.

[0041] S230: Perform texture information enhancement processing on each recombined feature map to obtain each enhanced feature map.

[0042] An enhanced feature map is a feature map obtained by enhancing the texture of a reconstructed feature map. In some embodiments, the reconstructed feature map can be processed using residual enhancement to obtain an enhanced feature map. In other embodiments, the reconstructed feature map can be processed using semantic enhancement to obtain an enhanced feature map. In still other embodiments, residual enhancement can be performed on the reconstructed feature map to obtain a residual feature map; semantic supplementary feature maps can be selected from each initial feature map based on the semantic information of each initial feature map; and feature fusion processing can be performed on the semantic supplementary feature map and the residual feature map to obtain the enhanced feature map. This allows high-resolution but semantically weak feature maps and strong semantic but low-resolution feature maps to complement each other, combining their respective advantages.

[0043] The residual feature map can be determined using the reconstructed feature map and the corresponding initial feature map. By enhancing the residual components in the reconstructed feature map, the edges and textures of small-sized targets can be further improved. As one example, the sum of the reconstructed feature map and the initial feature map can be used as the residual feature map. As another example, edge enhancement can be performed on the reconstructed feature map, and the sum of the reconstructed feature map and the edge-enhanced reconstructed feature map can be used as the residual feature map. As yet another example, edge enhancement and compression excitation processing can be performed on the reconstructed feature map separately to obtain an edge-enhanced reconstructed feature map and a compression-excitation-processed reconstructed feature map; the reconstructed feature product between the edge-enhanced and compression-excitation-processed reconstructed feature maps can be obtained; the reconstructed feature product can be balanced using a balancing coefficient to obtain a balanced reconstructed feature product; and the sum of the reconstructed feature map and the balanced reconstructed feature product can be used as the residual feature map. Specifically, the formula for obtaining the residual feature map from the edge-enhanced and compression-excitation-processed reconstructed feature maps is as follows:

[0044] in, Indicates the first The residual feature map corresponding to the stage, Indicates the first The reorganization feature map corresponding to the stage, Represents the balance coefficient. This indicates that the Squeeze-and-Excitation network is used for feature recalibration along the channel dimension. This represents an edge enhancement operator that enhances edge information by calculating the gradient response of the reconstructed feature map.

[0045] Edge enhancement operator The formula is as follows:

[0046] in, Indicates the first Recombination feature map corresponding to the stage.

[0047] Semantic supplementary features are feature maps selected from initial feature maps at different resolutions. For example, semantically rich initial feature maps can be selected from initial feature maps at different resolutions as semantic supplementary features to semantically complete the residual feature maps and supplement missing semantic information. It should be noted that after the image to be processed is input into the image feature extraction network, the shallow feature enhancement backbone network in the image feature extraction network includes multi-stage feature extraction. The initial feature maps are ordered from smallest to largest stage. The lower the resolution of the obtained initial feature map, the richer the semantic information. Therefore, semantic supplementary features can be selected according to the stage corresponding to the initial feature map. For example, the initial feature map with the largest stage can be used as the semantic supplementary feature, or the initial feature map with a stage greater than a preset stage threshold can be used as the semantic supplementary feature.

[0048] After selecting the semantic supplementary features, the semantic supplementary features and the residual feature map are fused to obtain the enhanced feature map. In some embodiments, the semantic supplementary features and the residual feature map can be fused by concatenation. In other embodiments, a semantically incomplete network can be constructed using a multilayer perceptron, and the semantic supplementary features can be pooled to obtain pooled semantic supplementary features. The pooled semantic supplementary features are then modulated using a multilayer perceptron to obtain modulated semantic supplementary features. The product of the adjusted semantic supplementary features and the residual feature map is used as the enhanced feature map, thereby enabling lower-level features to obtain higher-level semantic information. For example, the formula for the enhanced feature map is as follows:

[0049] in, Represents the enhanced feature map, This represents a multilayer perceptron. Indicates global average pooling. Indicates semantic supplementary features, This represents the residual feature map.

[0050] S240: Extract the region size of the target region where the target object is located from each enhanced feature map, and divide the target region based on the region size of the target region to obtain at least one image block corresponding to the target region.

[0051] After obtaining each enhanced feature map, initial target detection can be performed directly on each enhanced feature map to obtain the target region where the target object is located in the image to be processed; the grid density of the target region is determined according to the region size of the target region; the image to be processed is divided according to the grid density to obtain at least one image block corresponding to the target region.

[0052] In some embodiments, the size of the target region can be determined based on the coordinates of each vertex in the target region. For example, determining the maximum and minimum x-coordinates, as well as the maximum and minimum y-coordinates, from the coordinates of each vertex in the target region can be expressed as follows: The size of the target region can be calculated using the following formula:

[0053] in, Indicates the area size. This indicates taking the maximum value. Indicates the maximum x-coordinate. Represents the minimum x-coordinate. Represents the maximum ordinate. This represents the minimum ordinate.

[0054] After obtaining the region size of the target region, the grid density of the target region is determined based on the region size. The grid density represents the size of each image block after the target region is divided.

[0055] In some embodiments, a preset mapping table can be established, which includes the correspondence between preset region size ranges and preset grid density. After obtaining the region size of the target region, the preset region size range to which the region size of the target region belongs is determined, and the preset grid density corresponding to the preset region size range to which it belongs is determined as the grid density of the target region.

[0056] In other embodiments, the mesh density of the target region can be determined based on a first preset density parameter and the region size in response to the target region's size being smaller than a first preset size; the mesh density of the target region can be determined based on a second preset density parameter in response to the target region's size being greater than or equal to the first preset size and less than or equal to a second preset size; and the mesh density of the target region can be determined based on a third preset density parameter and the region size in response to the target region's size being greater than the second preset size. Wherein, the first preset density parameter is greater than the second preset density parameter, and the second preset density parameter is greater than the third preset density parameter. Thus, different mesh densities are calculated using different methods for different region sizes. Compared to the traditional method of using a uniform mesh size, this method ensures that target regions of different sizes can obtain high-quality feature representations.

[0057] The first preset size and the second preset size can be preset. For example, the first preset size and the second preset size can be set based on experience, whereby the first preset size is used to define small-sized targets and the second preset size is used to define large-sized targets.

[0058] The first, second, and third preset density parameters can be set based on experience. The first preset density parameter is greater than the second preset density parameter, and the second preset density parameter is greater than the third preset density parameter.

[0059] When the target area's size is smaller than a first preset size, it indicates that the target area is a small-sized target. The grid density of the target area is then determined based on the first preset density parameter and the area size. For example, initial adjustment parameters can be determined based on the area size and the standard area size. The first preset density parameter is then adjusted based on these initial adjustment parameters to obtain the grid density of the target area.

[0060] When the size of the target region is greater than or equal to a first preset size and less than or equal to a second preset size, it indicates that the target region is a medium-sized target. In this case, the grid density of the target region is determined based on the second preset density parameter. For example, the second preset density parameter can be used as the grid density of the target region.

[0061] When the target region's size is larger than the second preset size, it indicates that the target region is a large-sized target. In this case, the grid density of the target region is determined based on the third preset density parameter and the region size. For example, initial adjustment parameters can be determined based on the region size and the standard region size. The third preset density parameter is then adjusted based on these initial adjustment parameters to obtain the grid density of the target region. The calculation process for the initial adjustment parameters includes: obtaining the ratio between the standard region size and the target region size; using a preset adjustment hyperparameter as an exponent, calculating the power value of the ratio to obtain the initial adjustment parameters.

[0062] For example, the formula for calculating the grid density of the target region is as follows:

[0063] in, Indicates the grid density. This represents the first preset density parameter. This represents the second preset density parameter. This represents the third preset density parameter. Indicates the standard area size. Indicates the area size of the target region. This indicates the adjustment of hyperparameters. Indicates the first preset size. This indicates the second preset size. As an example, the above values ​​could be: , , , , , , .

[0064] It can be seen that this embodiment will relatively increase the grid density for small target areas, adopt the standard sampling grid density for standard target areas, and relatively decrease the grid density for large target areas.

[0065] In other embodiments, feature fusion processing can be performed on each enhanced feature map to obtain a fused feature map at at least one resolution; from the fused feature maps at different resolutions, a fused feature map with a resolution greater than a preset resolution threshold is selected as the feature map to be segmented; the region size of the target area where the target object is located is extracted from the feature map to be segmented, and the target area is divided based on the region size of the target area to obtain at least one image block corresponding to the target area. The method of dividing the feature map to be segmented can refer to the method of determining the grid density based on the region size described above, and will not be repeated here.

[0066] In this embodiment, the fused feature map is a feature map obtained by fusing enhanced feature maps at different resolutions; alternatively, the feature map obtained by fusing image features of the image to be processed at different resolutions can also be used as the fused feature map. In some embodiments, a traditional feature pyramid network can be used to perform feature fusion processing on each enhanced feature map to obtain each fused feature map.

[0067] Traditional feature pyramid networks suffer from semantic dilution in their top-down path, where the strong semantics of top-level features gradually weaken as they propagate downwards. Therefore, in some embodiments, feature pyramid networks can be improved to obtain multi-scale progressive feature pyramid networks. These networks introduce bidirectional propagation mechanisms and cross-scale attention to achieve more efficient information interaction. Multi-scale progressive feature pyramid networks employ a multi-round iterative approach to fuse enhanced feature maps at various resolutions. Each iteration of the enhanced feature map performs the following steps: Obtain each feature map to be updated in the current iteration; when the current iteration is the first iteration, each feature map to be updated is the enhanced feature map; or, when the current iteration is not the first iteration, the feature map to be updated is the updated feature map corresponding to the previous iteration; for each feature map to be updated, determine the cross-scale attention weight map between the feature map to be updated and each adjacent feature map to be updated, where the absolute value of the difference between the order of adjacent feature maps to be updated and the order of the feature maps to be updated is less than or equal to a preset order difference threshold; perform resolution alignment processing on each adjacent feature map to be updated according to the resolution of the feature map to be updated, obtaining each aligned adjacent feature map to be updated; determine the updated feature map of the current iteration based on each cross-scale attention weight map and each aligned adjacent feature map to be updated; wherein, the updated feature map of the last iteration is used as the fused feature map.

[0068] The feature map to be updated refers to the feature map waiting to be fused in each iteration round. In this embodiment, iterative fusion processing can be performed on enhanced feature maps at different resolutions to obtain fused feature maps at different resolutions. In other embodiments, iterative fusion processing can also be performed on image features of the image to be processed at different resolutions to obtain fused feature maps at different resolutions. The image features of the image to be processed at different resolutions refer to multiple features obtained through initial multi-scale feature extraction processing, that is, image features at various resolutions directly output by the shallow feature enhancement backbone network. When the features to be fused are image features at different resolutions, the feature map to be updated in the first iteration round is the image features at each resolution. The iterative fusion of enhanced feature maps is used as an example below.

[0069] The enhanced feature maps are arranged sequentially, and can be sorted from highest to lowest resolution. In each subsequent iteration, the order of the feature maps to be updated corresponds one-to-one with the order of the enhanced feature maps. When inputting the multi-scale progressive feature pyramid, the input is arranged in descending order of resolution, starting from the first layer. The first layer inputs the enhanced feature map with the highest resolution, the second layer inputs the enhanced feature map with the second highest resolution, and so on. Therefore, the order of the enhanced feature maps can also be determined according to the number of layers in the multi-scale progressive feature pyramid.

[0070] In the sequentially arranged feature maps to be updated, the absolute value of the difference between the order of adjacent feature maps and the order of the current feature map to be updated is less than or equal to a preset order difference threshold. For example, the preset order difference threshold can be 1, then assuming the feature map to be updated belongs to the _th ... When the layer is reached, the first layer will be... Layer, and / or, the first Layer, and / or, the first The feature map to be updated in a layer is used as an adjacent feature map to be updated. When the feature map to be updated belongs to the boundary layer, only the existing neighbors are taken as the feature map to be updated. For example, when the feature map to be updated is in layer 1, the feature maps to be updated in layer 1 and layer 2 are taken as adjacent feature maps to be updated. The updated feature map can be obtained by fusing the adjacent feature maps to be updated with the feature map to be updated.

[0071] Cross-scale attention weight maps are used to measure the importance of each neighboring feature map to be updated. For example, a self-attention mechanism can be used to calculate the cross-scale attention weight map between the feature map to be updated and its neighboring feature maps. Specifically, neighboring feature maps to be updated are resolution-aligned according to their resolution, resulting in aligned neighboring feature maps; a first product is obtained between the key vector weights and the aligned neighboring feature maps; a second product is obtained between the query vector weights and the feature maps to be updated; and self-attention features are calculated based on the first and second products to obtain the cross-scale attention weight map between the feature map to be updated and its neighboring feature maps. This allows for the evaluation of the correlation between each neighboring feature map and the feature map to be updated, thus obtaining the cross-scale attention weight map.

[0072] Resolution alignment refers to aligning the spatial dimensions of adjacent feature maps to be updated to match the spatial dimensions of the original feature maps. For example, resolution alignment can be achieved through upsampling or downsampling, resulting in aligned adjacent feature maps. The spatial dimensions of the aligned adjacent feature maps are the same as those of the original feature maps, meaning they have the same resolution.

[0073] Key vector weights and query vector weights can be obtained through pre-tuning. For example, the model can be pre-trained using a large-scale corpus to obtain key vector weights and query vector weights.

[0074] After obtaining the aligned neighboring feature maps to be updated, key vector weights, and query vector weights, a cross-scale attention weight map between the feature map to be updated and its neighboring feature maps is calculated based on these aligned neighboring feature maps, key vector weights, and query vector weights. For example, the formula for the cross-scale attention weight map is as follows:

[0075] Among them, the feature map to be updated belongs to the first... Layer, adjacent feature maps to be updated belong to the first layer. layer, Indicates the feature map to be updated and the first Cross-scale attention weight map between adjacent feature maps to be updated in a layer. This represents the activation function. Indicates the query vector weight. Indicates the first After the nth iteration Feature map of layer to be updated Represents the key vector weights. This represents the adjacent feature maps to be updated after alignment. Indicates the first After the nth iteration The adjacent feature maps of the layer that need to be updated.

[0076] After obtaining the cross-scale attention weight map between the feature map to be updated and each adjacent feature map to be updated, the updated feature map for the current iteration is determined based on the cross-scale attention weight map and each aligned adjacent feature map to be updated. For example, for each adjacent feature map to be updated, the cross-scale attention weight map corresponding to the adjacent feature map to be updated is fused with the corresponding aligned adjacent feature map to be updated to obtain the cross-scale fused feature map corresponding to that adjacent feature map to be updated; the cross-scale fused features corresponding to each adjacent feature map to be updated are summed, and then the sum is convolved to obtain the updated feature map for the current iteration. For example, the formula for calculating the updated feature map for the current iteration is as follows:

[0077] in, Indicates the first After the nth iteration The updated feature map of layer 1, the first layer The next iteration is the current iteration round. This represents a 3×3 convolution; This represents the neighboring feature maps of the feature map to be updated. Here, the one closest to the first feature map is selected. The feature maps to be updated in a layer are arranged in a sequence that is less than or equal to 1, and these are considered as adjacent feature maps to be updated. Indicates the first The feature map to be updated in layer and the first layer The cross-scale attention weight map between adjacent feature maps to be updated in the layer, and the sum of multiple cross-scale attention maps corresponding to each feature map to be updated is 1; This represents the adjacent feature maps to be updated after alignment. Indicates the first After the nth iteration The adjacent feature maps of the layer to be updated; This indicates element-wise multiplication.

[0078] The last iteration round refers to the last iteration round. In some embodiments, the last iteration round can be based on whether the updated feature map in that iteration round meets the requirements, such as whether the semantic information of the updated feature map meets the requirements. In other embodiments, the last iteration round can be a round number greater than or equal to a preset round number threshold.

[0079] As can be seen, the target detection method in this embodiment performs multi-scale feature extraction on the image to be processed, obtaining at least two initial feature maps with different resolutions; performs feature recombination on each initial feature map to obtain a recombined feature map; performs texture information enhancement on the recombined feature map to obtain an enhanced feature map; extracts the region size of the target area where the target object is located from the enhanced feature map, and divides the target area based on the region size to obtain at least one image patch corresponding to the target area. In the above scheme, the high-resolution initial feature map has weak semantics, while the semantically strong initial feature map has low resolution. Therefore, recombining initial feature maps of different resolutions can combine the advantages of both, achieving high resolution while possessing rich semantic information; subsequently, texture information enhancement can further improve the response of the target area edges and textures, thereby enhancing the feature representation of the target area.

[0080] Based on the above embodiments, the embodiments of this application adopt... Figure 3 The flowchart details how feature alignment is performed on image patches within the target region to obtain feature-aligned image patches. Please refer to [link / reference]. Figure 3 , Figure 3 yes Figure 1 The illustrated flowchart shows an exemplary embodiment of step S120 in the target detection method. Specifically, step S120 performs feature alignment processing on each image block based on the correlation between each sampling point and its corresponding neighboring pixels to obtain the feature-aligned image blocks, and further includes the following steps: S310: For each sampling point in the target region, obtain the concatenation feature between the sampling feature of the sampling point and the neighborhood feature of the corresponding neighboring pixel.

[0081] The sampling features of a sampling point represent the feature intensity of that sampling point, and may include information such as color or brightness. The neighborhood features of a neighboring pixel represent the feature intensity of its neighboring pixels, and may include information such as color or brightness of its neighboring pixels. The neighborhood features of a neighboring pixel can be the features of a single neighboring pixel, or a weighted sum of the features of multiple neighboring pixels.

[0082] Concatenated features refer to features obtained by concatenating sampled features with neighborhood features. For example, sampled features and neighborhood features can be concatenated along the channel dimension to obtain concatenated features.

[0083] S320: Obtain the activation function value of the splicing feature to get the first attention weight of the neighboring pixels. The first attention weight represents the correlation between the neighboring pixels and the sampling point.

[0084] The activation function values ​​are obtained by processing the concatenated features using an activation function, and these activation function values ​​are used as the first attention weights for the neighboring pixels. The activation function can be... Function. For example, the formula for calculating the first attention weight is as follows:

[0085] in, Indicates the first attention weight. This represents the activation function. Indicates learnable parameters, This represents the sampling characteristics of the sampling points. Represents the neighborhood features of neighboring pixels; This indicates concatenation of channel dimensions.

[0086] S330: Determine the second attention weight corresponding to the sampling point based on the preset value and the first attention weight.

[0087] The preset value can be 1, so that the first attention weight and the second attention weight are added together by 1. The difference between the preset value and the first attention weight is determined as the second attention weight. The first attention weight is used to weight the neighborhood features of neighboring pixels, and the second attention weight is used to weight the sampled features of the sampled points.

[0088] S340: The neighborhood features of the neighboring pixels and the sampling features of the sampling points are weighted and summed according to the first attention weight and the second attention weight to obtain each image block after feature alignment.

[0089] For example, the neighborhood features of neighboring pixels can be weighted according to the first attention weight to obtain the weighted neighborhood features; the sampled features can be weighted according to the second attention weight to obtain the weighted sampled features; the weighted neighborhood features and the weighted sampled features are used as the weighted fused neighborhood features corresponding to the sampled points; if an image block includes multiple sampled points (multiple means two or more), the weighted fused neighborhood features corresponding to each sampled point in the image block are fused to obtain the feature-aligned image block; if an image block includes one sampled point, the feature-aligned image block is directly determined according to the weighted fused neighborhood features corresponding to the sampled point.

[0090] The formula for calculating the weighted fusion neighborhood features corresponding to the sampling points is as follows:

[0091] Indicates sampling point The corresponding weighted fusion neighborhood features, Indicates the first attention weight. Representing neighborhood features, This represents the second attention weight. This indicates the sampling characteristics.

[0092] As can be seen, the target detection method in this embodiment of the application, for each sampling point in the target region, obtains the concatenated features between the sampling features of the sampling point and the neighborhood features of the corresponding neighboring pixels; obtains the activation function value of the concatenated features to obtain the first attention weight of the neighboring pixels, which characterizes the correlation between the neighboring pixels and the sampling point; determines the second attention weight corresponding to the sampling point based on a preset value and the first attention weight; and performs a weighted summation of the neighborhood features of the neighboring pixels and the sampling features of the sampling point based on the first attention weight and the second attention weight to obtain each image patch after feature alignment. Thus, the sampling features of the sampling point are adjusted by the neighborhood features of the neighboring pixels to make the sampling features of the sampling point approach the true value, thereby further ensuring the feature authenticity of the image patch.

[0093] Based on the above embodiments, the embodiments of this application adopt... Figure 4 The flowchart details how to further adjust the feature-aligned image patches using region size to obtain the adjusted target region, thus yielding the feature-aligned image patches. Please refer to [link / reference needed]. Figure 4 , Figure 4 yes Figure 1The illustrated flowchart shows an exemplary embodiment of step S130 in the target detection method. Specifically, step S130 adjusts the feature-aligned image blocks according to the region size of the target region to obtain the adjusted target region, and further includes the following steps: S410: Determine the scale adjustment parameters based on the area size of the target area. The area size and the scale adjustment parameters are positively correlated.

[0094] The scaling parameter is used to adjust the features of the feature-aligned image patch to prevent the features of the feature-aligned image patch from being too weak or too strong. For example, the scaling parameter can be determined based on the region size. In some embodiments, it can be determined that when the region size is smaller than the standard region size, the scaling parameter is used to enlarge the features of the feature-aligned image patch; when the region size is larger than the standard region size, the scaling parameter is used to reduce the features of the feature-aligned image patch. As an example, the formula for the scaling parameter is as follows:

[0095] in, Indicates the scaling adjustment parameters. Indicates learnable parameters, Indicates the standard area size. This represents the size of the target area. It can be seen that when the area size is smaller than the standard area size, the scale adjustment parameter is greater than 1; when the area size is larger than the standard area size, the scale adjustment parameter is less than 1.

[0096] S420: Adjust the image blocks after feature alignment according to the scale adjustment parameters to obtain the adjusted target region.

[0097] After obtaining the scale adjustment parameters, the feature-aligned image blocks are adjusted according to the scale adjustment parameters to obtain the adjusted image blocks; the adjusted target region is determined based on the adjusted image blocks.

[0098] In some embodiments, adjusting each image block after feature alignment according to the scale adjustment parameter is actually adjusting the features corresponding to the image blocks after feature alignment according to the scale adjustment parameter.

[0099] In other embodiments, after obtaining the weighted fused neighborhood features of each sampling point in each feature-aligned image block, the weighted fused neighborhood features of each sampling point can be adjusted according to a scale adjustment parameter to obtain the adjusted weighted fused neighborhood features of each sampling point; the feature-aligned image block is then determined based on the adjusted weighted fused neighborhood features of each sampling point in each feature-aligned image block. For example, the adjustment formula for the adjusted weighted fused neighborhood features of each sampling point is as follows:

[0100] in, This represents the weighted fusion neighborhood features after adjustment for each sampling point. Indicates learnable parameters, Indicates the standard area size. Indicates the area size of the target region. This represents the weighted fusion neighborhood features of each sampling point. The weighted fusion neighborhood features of each sampling point can be determined by referring to the above-described embodiment regarding obtaining the weighted fusion neighborhood features of each sampling point.

[0101] As can be seen, the target detection method in this embodiment determines the scale adjustment parameter based on the region size of the target area, and the region size and the scale adjustment parameter are positively correlated. The adjusted image blocks, after feature alignment, are adjusted according to the scale adjustment parameter to obtain the adjusted target area. This can prevent the features of the target area from being too weak or too strong, thus deviating from the true features.

[0102] Based on the above embodiments, the embodiments of this application adopt... Figure 5 The flowchart details how to obtain target detection results through adjusted target region detection. Please refer to [link / reference]. Figure 5 , Figure 5 yes Figure 1 The illustrated flowchart shows an exemplary embodiment of step S140 in the target detection method. Specifically, step S140, which performs target detection processing on the adjusted target region to obtain the target detection result, further includes the following steps: S510: The adjusted target region is downsampled using at least two downsampling rates to obtain at least two downsampled feature maps, each with a different resolution.

[0103] The downsampling rate refers to the proportion of original data samples retained during the downsampling process of the adjusted target region. For example, the downsampling rate can be set empirically to ensure that different downsampled feature maps include the corresponding features of interest.

[0104] A downsampled feature map is a feature map obtained by downsampling the adjusted target region. There can be two or more downsampled feature maps. The following example uses three downsampling rates to downsample the adjusted target region, resulting in three downsampled feature maps. The three different downsampling rates increase sequentially, and the resulting three downsampled feature maps can be represented as follows:

[0105] in, , , These represent three downsampled feature maps. This represents the adjusted target area. Where the downsampling rate... Less than the downsampling rate ,but The highest resolution allows for the downsampling of the corresponding feature map. As a fine-grained branch; As a standard branch; The lowest resolution allows us to use its corresponding downsampled feature map. As a coarse-grained branch. For example, downsampling rate. =1 / 2, downsampling rate =2.

[0106] S520: For each downsampled feature map, obtain the attention interaction features between the target downsampled feature map and other downsampled feature maps. The other downsampled feature maps are the downsampled feature maps other than the target downsampled feature map in each downsampled feature map.

[0107] The target downsampled feature map refers to any downsampled feature map selected from the various downsampled feature maps, while other downsampled feature maps refer to all downsampled feature maps except the target downsampled feature map. In some embodiments, each downsampled feature map can be traversed, with the currently traversed downsampled feature map serving as the target downsampled feature map, and the remaining downsampled feature maps serving as other downsampled feature maps. In other embodiments, the downsampled feature maps can be processed in parallel, where each processing thread targets the target downsampled feature map, and downsampled feature maps not targeted by that processing thread are considered other downsampled feature maps.

[0108] Attention interaction features refer to the features obtained by fusing the target downsampled feature map with other downsampled feature maps. In some embodiments, the target downsampled feature map and other downsampled feature maps can be concatenated to obtain attention interaction features. In other embodiments, the target self-attention feature map of the target downsampled feature map and other self-attention feature maps of the other downsampled feature maps can also be obtained; the resolution of each other self-attention feature map is aligned according to the resolution of the target self-attention feature map to obtain each aligned other attention feature map; the sum of the target self-attention feature map and each aligned other attention feature map is determined as the attention interaction feature corresponding to the target downsampled feature map. This allows the high-resolution branch to guide other lower-resolution branches to focus on finer positions, improving the accuracy of feature representation.

[0109] The target self-attention feature map and other self-attention feature maps can be calculated using a self-attention mechanism. For example, the calculation of the target self-attention feature map and other self-attention feature maps satisfies the following equation:

[0110] in, This represents the target self-attention feature map or other self-attention feature maps. This represents the activation function. Represents the query matrix. Represents the key matrix, Represents a value matrix, These represent the branches corresponding to the three different downsampled feature maps mentioned above, when When it is a fine-grained branch, when When it is a standard branch, This is a coarse-grained branch.

[0111] The calculation of the query matrix, key matrix, and value matrix satisfies the following formula:

[0112]

[0113]

[0114] in, Represents the query matrix. Represents the downsampled feature map. , Indicates the query vector weight. Represents the key matrix, Represents the key vector weights. Represents a value matrix, Represents the weights of the value vector.

[0115] To prevent scale misalignment, the resolution of each other self-attention feature map is aligned with that of the target self-attention feature map. Upsampling or downsampling can be used to align the other self-attention feature maps. Then, the target self-attention feature map is fused with each aligned self-attention feature map to obtain the attention interaction features corresponding to the target downsampled feature map. For example, the sum of the target self-attention feature map and each aligned self-attention feature map can be used to determine the attention interaction features corresponding to the target downsampled feature map. As an example, when the target downsampled feature map is a downsampled feature map of a fine-grained branch, the formula for calculating the attention interaction features of the target downsampled feature map is as follows:

[0116] in, This represents the attention interaction features of the target downsampled feature map. Represents the target self-attention feature map. Other self-attention feature maps representing the standard branch, Other self-attention feature maps representing coarse-grained branches, This represents a learnable scale alignment function used to adjust attention feature maps at different scales to the corresponding resolution. This indicates upsampling.

[0117] A learnable scale alignment function can be expressed as:

[0118] in, This represents a learnable scale alignment function. This represents a 1×1 convolution.

[0119] S530: The attention interaction features corresponding to each downsampled feature map are fused to obtain the target feature map.

[0120] After obtaining the attention interaction features corresponding to each downsampled feature map, the attention interaction features corresponding to each downsampled feature map need to be fused to obtain the target feature map. In some embodiments, the attention interaction features corresponding to each downsampled feature map can be weighted according to the resolution of each downsampled feature map to obtain weighted attention interaction features. In other embodiments, the adjusted target region can be input into a weight prediction network to obtain the fusion weights corresponding to each downsampled feature map; the fusion weights corresponding to each downsampled feature map can be normalized to obtain normalized weights corresponding to each downsampled feature map; and the attention interaction features corresponding to each downsampled feature map can be weighted and summed according to the normalized weights to obtain the target feature map.

[0121] The weight prediction network can be a lightweight meta-network. By inputting the adjusted target region into the weight prediction network, the contribution ratio of the three branches in the final feature fusion can be dynamically adjusted based on the content of the adjusted target region. For example, the prediction formula of the weight prediction network can be expressed as:

[0122] in, This represents the fusion weights corresponding to the three downsampled feature maps. This represents the sigmoid function. Indicates global average pooling. This indicates the adjusted target area. , and This represents the learnable parameters.

[0123] To ensure that the sum of the fusion weights corresponding to the three downsampled feature maps is 1, the fusion weights corresponding to each sampled feature map need to be normalized to obtain normalized weights. The Softmax function can be used to achieve this normalization.

[0124] Before performing a weighted summation of the attention interaction features corresponding to each downsampled feature map, resolution alignment processing is required to unify the resolution of each attention interaction feature. Specifically, the attention interaction features of the fine-grained branch are upsampled to obtain the first feature map to be fused; the attention interaction features of the standard branch are determined as the second feature map to be fused; the attention interaction features of the coarse-grained branch are downsampled to obtain the third feature map to be fused; and the first, second, and third feature maps to be fused are weighted and summed according to the normalized weights corresponding to each sampled feature map to obtain the target feature map.

[0125] For example, the formula for calculating the target feature map is as follows:

[0126] in, Represents the target feature map. Represents an exponential function. This represents the fusion weights of the fine-grained branches. This represents the sum of the exponential function values ​​of the fusion weights under each branch. , Indicates upsampling, This represents the attention interaction features of fine-grained branches. This represents the fusion weight of the standard branch. This represents the attention interaction features of the standard branch. This represents the fusion weights of the coarse-grained branches. This represents the attention interaction features of coarse-grained branches.

[0127] S540: Decode the target feature map to obtain the target detection result.

[0128] The target feature map is input into the decoder to obtain the target detection result output by the decoder.

[0129] As can be seen, the target detection method in this embodiment of the application performs downsampling processing on the adjusted target region using at least two downsampling rates to obtain at least two downsampling feature maps, each with a different resolution. For each downsampling feature map, attention interaction features between the target downsampling feature map and other downsampling feature maps are obtained. The other downsampling feature maps are downsampling feature maps other than the target downsampling feature map. The attention interaction features corresponding to each downsampling feature map are fused to obtain the target feature map. The target feature map is then decoded to obtain the target detection result. This multi-scale branching enhances the multi-scale interaction of small-sized targets. The high-resolution downsampling feature map can obtain more detailed target information, the medium-resolution downsampling feature map balances semantic and spatial information, and the low-resolution downsampling feature map can provide global contextual information. Combining multi-scale information can improve the accuracy of the feature representation of the target feature map.

[0130] It should be noted that the object detection method in this embodiment adopts an encoder-decoder architecture object detection model, such as DEIM (Detection with Improved Matching). The main modules of the object detection model include a backbone network, a fusion network, an encoder, and a decoder. In addition, there are additional modules to improve the model performance, including a Detail Awareness Module (DAM), a multi-scale progressive feature pyramid network, a progressive feature enhancement module (PFEM), a small object-aware ROI alignment module (SOAR), and a multi-scale spatial-temporal attention module (MS-STA). The overall process is as follows: The image to be processed is input into the backbone network to extract image features at different resolutions; the image features at different resolutions are input into the detail-aware module for texture enhancement to obtain at least two initial feature maps; the at least two initial feature maps are input into the progressive feature enhancement module for feature recombination, residual enhancement, and semantic supplementation enhancement to obtain enhanced feature maps; the enhanced feature maps are input into the multi-scale progressive feature pyramid network for feature fusion to obtain fused features at different resolutions; the fused features at different resolutions are input into the small target perception region of interest alignment module for feature alignment to obtain the adjusted target region; the adjusted target region is input into the encoder to obtain the encoded target region; the encoded target region is input into the multi-scale spatial-temporal attention module for multi-resolution branch fusion to obtain the target feature map; the target feature map is input into the decoder for target detection to obtain the target detection result.

[0131] The detail-aware module is used to perform initial multi-scale feature extraction on the image to be processed, obtaining image features of the image to be processed at different resolutions; texture information enhancement processing is performed on the image features at different resolutions respectively, obtaining image features after texture information enhancement; and residual connection is performed between each image feature and the corresponding image features after texture information enhancement to obtain at least two initial feature maps corresponding to the image to be processed.

[0132] The multi-scale progressive feature pyramid uses a multi-round iterative approach to fuse enhanced feature maps at various resolutions. Each iteration of the enhanced feature map performs the following steps: Obtain the feature maps to be updated in the current iteration round; if the current iteration round is the first iteration round, the feature maps to be updated are the enhanced feature maps; or, if the current iteration round is not the first iteration round, the feature maps to be updated are the updated feature maps corresponding to the previous iteration round; for each feature map to be updated, determine the cross-scale attention weight map between the feature map to be updated and each adjacent feature map to be updated, where the absolute value of the difference between the order of adjacent feature maps to be updated and the order of the feature maps to be updated is less than or equal to a preset order difference threshold; perform resolution alignment processing on each adjacent feature map to be updated according to the resolution of the feature map to be updated, obtaining each aligned adjacent feature map to be updated; determine the updated feature map of the current iteration round based on each cross-scale attention weight map and each aligned adjacent feature map to be updated; wherein, the updated feature map of the last iteration round is used as the fused feature map.

[0133] The progressive feature enhancement module is used to perform multi-scale feature extraction processing on the image to be processed, to obtain at least two initial feature maps with different resolutions; to perform feature recombination processing on each initial feature map, to obtain recombined feature maps corresponding to each initial feature map; to perform texture information enhancement processing on each recombined feature map, to obtain enhanced feature maps; to extract the region size of the target region where the target object is located from each enhanced feature map, and to divide the target region based on the region size of the target region, to obtain at least one image patch corresponding to the target region.

[0134] The small target perception region of interest alignment module is used to divide the image to be processed based on the region size of the target area where the target object is located, to obtain at least one image block corresponding to the target area, and the image block includes at least one sampling point; according to the correlation between each sampling point in each image block and the corresponding neighboring pixels, feature alignment processing is performed on each image block to obtain each feature-aligned image block; according to the region size of the target area, each feature-aligned image block is adjusted to obtain the adjusted target area.

[0135] A multi-scale spatial-temporal attention module is used to downsample the adjusted target region at at least two downsampling rates to obtain at least two downsampled feature maps with different resolutions. For each downsampled feature map, attention interaction features between the target downsampled feature map and other downsampled feature maps are obtained. The other downsampled feature maps are the downsampled feature maps other than the target downsampled feature map. The attention interaction features corresponding to each downsampled feature map are fused to obtain the target feature map. The target feature map is then decoded to obtain the target detection result.

[0136] In practical applications, one or more modules can be selected to enhance model performance based on different needs and the capabilities of the adapter. Specifically, the detail-aware module can be introduced after the backbone network and before the fusion network; the multi-scale progressive feature pyramid can replace the fusion network; the progressive feature enhancement module can be introduced after the backbone network and before the fusion network. When both the detail-aware module and the progressive feature enhancement module are introduced simultaneously, their order is not restricted; the detail-aware module can be introduced first, followed by the progressive feature enhancement module. The small target perception region of interest alignment module can be introduced after the fusion network and before the encoder; the multi-scale spatial-temporal attention module can be introduced after the encoder and before the decoder.

[0137] As an example, assume that three model configurations are provided, as shown in Table 1: Table 1

[0138] The three model specifications share the same technical framework. The differences lie in the selection of the backbone network and the degree of module activation. DEIM-SOD-N is for scenarios with extremely limited resources, such as mobile scenarios, and only DAM and basic MS-STA are enabled. DEIM-SOD-S is for scenarios with moderate resources, such as edge computing scenarios, and full MS-STA is enabled. DEIM-SOD-M is for scenarios with relatively good resources, such as high-precision server scenarios, and DAM, MS-STA, PFEM and SOAR are enabled.

[0139] In addition, a multi-stage strategy can be adopted for training object detection models, including three stages: pre-training, joint training, and fine-tuning.

[0140] During the pre-training phase, the backbone network is pre-trained using large-scale detection datasets (such as Objects365 and OpenImages). Pre-training employs a standard detection task, with a batch size of 256 and 100 epochs. The learning rate during pre-training uses a cosine annealing strategy, with an initial learning rate of 0.1 and weight decay of 0.0001.

[0141] The joint training phase jointly optimizes the backbone network, decoder, progressive feature enhancement module, small object perception region of interest alignment module, and multi-scale spatial-temporal attention module. This phase uses all training images from the COCO dataset, with a batch size of 16 (4 images per GPU) and 50 training epochs. A two-stage learning rate strategy is employed: the learning rate of the backbone network is 0.1 times that of the decoder, and the learning rate warmup period is 3 training epochs.

[0142] The fine-tuning phase is specifically optimized for the small object detection task. The fine-tuning dataset includes all images containing small objects from the COCO training set, as well as specially collected small object datasets (such as AI-TOD and SKU110K). The fine-tuning phase uses a small learning rate (0.001), trains for 20 training epochs, and enables small object augmentation strategies in data augmentation.

[0143] Training sample enhancement strategies include at least one of the following: stitching multiple sample images into one, which can significantly increase the number and diversity of small targets. During stitching, ensure that each image contains at least one small target to prevent an excessive number of negative samples; for example, four sample images can be stitched into one. Copying the target region of a small target from a sample image and randomly pasting it to other locations within the same sample image, performing scale transformations and color jittering during copying and pasting to avoid a harsh pasting effect; cropping a sub-image containing a small target from the sample image, with the sub-image size being 0.3-0.6 times that of the sample image. The cropping location is determined based on the spatial distribution of small targets, ensuring that the cropped area contains as many small targets as possible.

[0144] Please see Figure 6 , Figure 6This is a schematic diagram of an exemplary embodiment of the target detection device shown in this application. The target detection device 600 includes a segmentation module 610, a feature alignment module 620, an adjustment module 630, and a target detection module 640. The segmentation module 610 is used to segment the image to be processed according to the region size of the target region where the target object is located in the acquired image to be processed, to obtain at least one image block corresponding to the target region, and the image block includes at least one sampling point. The feature alignment module 620 is used to perform feature alignment processing on each image block according to the correlation relationship between each sampling point in each image block and the corresponding neighboring pixels, to obtain each image block after feature alignment. The adjustment module 630 is used to adjust each image block after feature alignment according to the region size of the target region, to obtain an adjusted target region. The target detection module 640 is used to perform target detection processing on the adjusted target region to obtain a target detection result.

[0145] The above scheme involves the target detection device dividing the image into blocks based on the size of the target region containing the target object in the image to be processed. The size of each block can be adaptively adjusted according to the size of the target object to ensure that each block contains sufficient image information. Feature alignment is then performed on each block based on the correlation between each sampling point and its corresponding neighboring pixels, resulting in feature-aligned blocks. This ensures that the boundaries of the blocks are strictly aligned with the pixel boundaries in the image to be processed, preventing spatial misalignment. The feature-aligned blocks are then adjusted according to the size of the target region to obtain an adjusted target region. Further adjustments to the feature-aligned blocks based on the region size enhance the perception of small targets and prevent over-enhancement of large targets, thereby improving the feature representation of the target region and increasing the accuracy of target detection.

[0146] The functions of each module can be found in the implementation examples of the target detection method, and will not be repeated here.

[0147] To implement the target detection method of the above embodiments, this application proposes another electronic device, please refer to [link / reference needed]. Figure 7 , Figure 7 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0148] Electronic device 700 includes memory 710 and processor 720, wherein memory 710 and processor 720 are coupled together.

[0149] The memory 710 is used to store program data, and the processor 720 is used to execute the program data to implement the target detection method of the above embodiment.

[0150] In this embodiment, processor 720 can also be referred to as CPU (Central Processing Unit). Processor 720 may be an integrated circuit chip with signal processing capabilities. Processor 720 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 720 can be any conventional processor.

[0151] This application also provides a computer-readable storage medium, such as Figure 8 As shown, the computer-readable storage medium 800 is used to store program data 810, which, when executed by a processor, is used to implement the target detection method as described in the method embodiments of this application.

[0152] The methods involved in the target detection method embodiments of this application, when implemented as software functional units and sold or used as independent products, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0154] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "and / or" is merely a description of the association of related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this document means two or more. In addition, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements, such as including at least one of A, B, and C, and may mean including any one or more elements selected from the set consisting of A, B, and C.

Claims

1. A target detection method, characterized in that, The target detection method includes: The image to be processed is divided according to the size of the target region where the target object is located in the acquired image to be processed, so as to obtain at least one image block corresponding to the target region, and the image block includes at least one sampling point; Based on the correlation between each sampling point and its corresponding neighboring pixels in each image block, feature alignment is performed on each image block to obtain each image block after feature alignment. The image blocks after feature alignment are adjusted according to the region size of the target region to obtain the adjusted target region. The adjusted target area is then subjected to target detection processing to obtain the target detection result.

2. The target detection method according to claim 1, characterized in that, The step of adjusting each image patch after feature alignment according to the region size of the target region to obtain the adjusted target region includes: The scale adjustment parameters are determined based on the region size of the target region, and the region size and the scale adjustment parameters are positively correlated. The image blocks after feature alignment are adjusted according to the scale adjustment parameters to obtain the adjusted target region.

3. The target detection method according to claim 1, characterized in that, The step of dividing the image to be processed according to the region size of the target region where the target object is located in the acquired image to be processed, to obtain at least one image block corresponding to the target region, includes: Multi-scale feature extraction is performed on the image to be processed to obtain at least two initial feature maps, wherein the at least two initial feature maps have different resolutions; Each initial feature map is recombined to obtain the recombined feature map corresponding to each initial feature map. Texture information enhancement processing is performed on each reconstructed feature map to obtain each enhanced feature map; Extract the region size of the target region where the target object is located from each enhanced feature map, and divide the target region based on the region size of the target region to obtain at least one image block corresponding to the target region.

4. The target detection method according to claim 3, characterized in that, The step of performing multi-scale feature extraction on the image to be processed to obtain at least two initial feature maps includes: The image to be processed is subjected to initial multi-scale feature extraction processing to obtain the image features of the image to be processed at different resolutions; Texture enhancement processing was performed on image features at different resolutions to obtain image features with enhanced texture information. Each image feature is residually concatenated with the corresponding texture information-enhanced image features to obtain at least two initial feature maps corresponding to the image to be processed.

5. The target detection method according to claim 3, characterized in that, The step of performing texture information enhancement processing on the reconstructed feature map to obtain an enhanced feature map includes: The reconstructed feature map is subjected to residual enhancement processing to obtain a residual feature map; Based on the semantic information of each initial feature map, a semantic supplementary feature map is selected from each initial feature map; The semantic supplementary feature map and the residual feature map are subjected to feature fusion processing to obtain the enhanced feature map.

6. The target detection method according to claim 3, characterized in that, The step of extracting the region size of the target region where the target object is located from each enhanced feature map, and dividing the target region based on the region size of the target region to obtain at least one image patch corresponding to the target region includes: Enhanced feature maps at different resolutions are subjected to feature fusion processing to obtain fused feature maps at different resolutions; Select the fused feature map with a resolution greater than a preset resolution threshold from the fused feature maps at different resolutions as the feature map to be segmented; Extract the region size of the target region where the target object is located from the feature map to be segmented, and divide the target region based on the region size of the target region to obtain at least one image block corresponding to the target region.

7. The target detection method according to claim 6, characterized in that, The enhanced feature maps are arranged in order. The step of performing feature fusion processing on the enhanced feature maps at different resolutions to obtain fused feature maps at different resolutions includes: For each iteration of the enhanced feature map, the following steps are performed: Obtain each feature map to be updated in the current iteration round; when the current iteration round is the first iteration round, each feature map to be updated is an enhanced feature map; or, when the current iteration round is not the first iteration round, the feature map to be updated is the updated feature map corresponding to the previous iteration round. For each feature map to be updated, a cross-scale attention weight map of the feature map to be updated and each adjacent feature map to be updated is determined based on the feature map to be updated and each adjacent feature map to be updated. The absolute value of the difference between the arrangement order of the adjacent feature maps to be updated and the arrangement order of the feature map to be updated is less than or equal to a preset order difference threshold. Based on the resolution of the feature map to be updated, each adjacent feature map to be updated is aligned to obtain each aligned adjacent feature map to be updated. The updated feature map for the current iteration is determined based on the attention weight maps for each cross-scale and the adjacent feature maps to be updated after alignment. The updated feature map from the last iteration is used as the fused feature map.

8. The target detection method according to claim 7, characterized in that, The step of determining the cross-scale attention weight map of the feature map to be updated and each adjacent feature map to be updated based on the feature map to be updated and each adjacent feature map to be updated includes: The adjacent feature maps to be updated are aligned according to their resolution to obtain aligned adjacent feature maps. Obtain the first product between the key vector weights and the adjacent feature maps to be updated after alignment. Obtain the second product between the query vector weights and the feature map to be updated; Self-attention features are calculated based on the first product and the second product to obtain a cross-scale attention weight map between the feature map to be updated and the adjacent feature maps to be updated.

9. The target detection method according to claim 3, characterized in that, The step of dividing the target region based on the region size of the target region to obtain at least one image patch corresponding to the target region includes: The grid density of the target region is determined based on the region size of the target region. The image to be processed is divided according to the grid density to obtain at least one image block corresponding to the target region.

10. The target detection method according to claim 9, characterized in that, The step of determining the grid density of the target region based on the region size of the target region includes: In response to the target region's size being smaller than a first preset size, the grid density of the target region is determined based on the first preset density parameter and the region size. In response to the target region's size being greater than or equal to the first preset size and less than or equal to the second preset size, the grid density of the target region is determined according to the second preset density parameter; In response to the target region's size being larger than the second preset size, the grid density of the target region is determined based on the third preset density parameter and the region size. Wherein, the first preset density parameter is greater than the second preset density parameter, and the second preset density parameter is greater than the third preset density parameter.

11. The target detection method according to claim 1, characterized in that, The step of performing feature alignment processing on each image block based on the correlation between each sampling point in each image block and its corresponding neighboring pixels to obtain feature-aligned image blocks includes: For each sampling point in the target region, the concatenation feature between the sampling feature of the sampling point and the neighborhood feature of the corresponding neighboring pixel is obtained; The activation function value of the spliced ​​feature is obtained to obtain the first attention weight of the neighboring pixel, and the first attention weight represents the degree of correlation between the neighboring pixel and the sampling point; The second attention weight corresponding to the sampling point is determined based on the preset value and the first attention weight; The neighborhood features of the neighboring pixels and the sampling features of the sampling points are weighted and summed according to the first attention weight and the second attention weight to obtain the image blocks after feature alignment.

12. The target detection method according to claim 1, characterized in that, The step of performing target detection processing on the adjusted target region to obtain the target detection result includes: The adjusted target region is downsampled using at least two downsampling rates to obtain at least two downsampled feature maps, each with a different resolution. For each downsampled feature map, the attention interaction features between the target downsampled feature map and other downsampled feature maps are obtained, wherein the other downsampled feature maps are downsampled feature maps other than the target downsampled feature map. The attention interaction features corresponding to each downsampled feature map are fused to obtain the target feature map; The target feature map is decoded to obtain the target detection result.

13. The target detection method according to claim 12, characterized in that, The step of obtaining the attention interaction features between the target downsampled feature map and other downsampled feature maps includes: Obtain the target self-attention feature map of the target downsampled feature map and the other self-attention feature maps of the other downsampled feature maps; According to the resolution of the target self-attention feature map, the other self-attention feature maps are aligned to obtain the other attention feature maps after alignment. The sum of the target self-attention feature map and the other attention feature maps after alignment processing is determined as the attention interaction feature corresponding to the target downsampled feature map.

14. The target detection method according to claim 12, characterized in that, The step of fusing the attention interaction features corresponding to each downsampled feature map to obtain the target feature map includes: The adjusted target region is input into the weight prediction network to obtain the fusion weights corresponding to each downsampled feature map; The fusion weights corresponding to each downsampled feature map are normalized to obtain the normalized weights corresponding to each downsampled feature map. The attention interaction features corresponding to each downsampled feature map are weighted and summed according to the normalized weights corresponding to each downsampled feature map to obtain the target feature map.

15. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-14.

16. A computer-readable storage medium, characterized in that, include: The system stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-14.

Images