Target detection method and device, electronic equipment and nonvolatile storage medium

Abstract

The application discloses a target detection method and device, electronic equipment and a nonvolatile storage medium. The method comprises: performing feature extraction on a to-be-recognized image to obtain a plurality of first feature maps of different scales; dividing the first feature maps into a plurality of image blocks, and rearranging the image blocks in the first feature maps to obtain a second feature map; calculating self-attention weights according to the second feature map, and generating a third feature map according to the self-attention weights, wherein the self-attention weights are used to represent the influence degree of each feature in the second feature map on recognizing a target object in the to-be-recognized image; and determining the target object in the to-be-recognized image according to the third feature map. The application solves the technical problem of poor accuracy of target detection positioning in the related art.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a target detection method, apparatus, electronic device, and non-volatile storage medium. Background Technology

[0002] Convolutional Neural Networks (CNNs) are commonly used deep, feedback-based artificial neural networks and a type of deep learning method that has been successfully applied in computer vision. With the resurgence of deep learning technology, object detection methods based on deep learning have developed rapidly in recent years. Region-candidate-based R-CNN, and its improved versions Fast R-CNN and Faster R-CNN, have been proposed and widely used. However, these network models suffer from poor accuracy in actual object detection.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a target detection method, apparatus, electronic device, and non-volatile storage medium to at least solve the technical problem of poor target detection and localization accuracy in neural networks used for target detection in related technologies.

[0005] According to one aspect of the embodiments of this application, a target detection method is provided, comprising: extracting features from an image to be identified to obtain multiple first feature maps of different scales; dividing the first feature maps into multiple image blocks and rearranging the image blocks in the first feature maps to obtain a second feature map; calculating self-attention weights based on the second feature maps and generating a third feature map based on the self-attention weights, wherein the self-attention weights are used to characterize the degree of influence of each feature in the second feature map on identifying a target object in the image to be identified; and determining the target object in the image to be identified based on the third feature map.

[0006] Optionally, the height of the target object in the image to be identified is greater than its width; rearranging the image blocks in the first feature map to obtain the second feature map includes: determining multiple image block sets based on the image blocks in each column of the first feature map, wherein each image block set corresponds to a column in the first feature map; combining the image blocks in the image block sets into a target sub-map, wherein each image block set corresponds to a target sub-map; and determining a second feature map corresponding to the first feature map based on the target sub-map, wherein the first feature map and the second feature map have the same scale, and the arrangement order of the image blocks in the first feature map and the second feature map is different.

[0007] Optionally, calculating self-attention weights and generating a third feature map based on the self-attention weights includes: performing a linear transformation on the second feature map to obtain a query matrix, a key matrix, and a value matrix; determining self-attention weights based on the query matrix and the key matrix; performing a weighted calculation based on the self-attention weights and the value matrix to obtain a fourth feature map, wherein the feature map contains the self-attention scores corresponding to each image patch; rearranging the image patches in the set of image patches corresponding to each target sub-map that makes up the fourth feature map, and combining them into a third feature map, wherein the first feature map and the third feature map have the same scale, and the image patches in the first feature map and the third feature map have the same arrangement order.

[0008] Optionally, feature extraction based on the image to be identified to obtain multiple first feature maps at different scales includes: extracting features sequentially from bottom to top based on multiple convolutional kernels in the feature pyramid network to obtain multiple first candidate maps at different scales, wherein each convolutional kernel corresponds to one first candidate map, and the scale of the first candidate maps corresponding to each convolutional kernel decreases sequentially from bottom to top while the feature information contained in the first candidate maps increases sequentially; performing multiple upsampling operations sequentially from top to bottom on the first candidate map corresponding to the top convolutional kernel to obtain multiple second candidate maps at different scales, wherein each upsampling operation corresponds to one second candidate map; fusing the first and second candidate maps of the same scale to obtain multiple fifth feature maps at different scales; and adjusting the fifth feature maps through a channel attention module and a spatial attention module to obtain the first feature map.

[0009] Optionally, adjusting the fifth feature map using the channel attention module and the spatial attention module to obtain the first feature map includes: performing average pooling on the fifth feature map using the channel attention module to obtain the first channel feature map, and performing max pooling on the fifth feature map to obtain the second channel feature map, wherein the spatial scales of the first and second channel feature maps are preset values; determining the channel attention weights corresponding to the fifth feature map based on the first and second channel feature maps, wherein the channel attention weights are used to characterize the importance of each channel in the fifth feature map; generating a sixth feature map based on the channel attention weights, wherein the sixth feature map contains the channel attention scores corresponding to each channel; and adjusting the sixth feature map using the spatial attention module to obtain the first feature map.

[0010] Optionally, adjusting the sixth feature map to obtain the first feature map includes: performing average pooling on the sixth feature map along the channel dimension to obtain a first spatial feature map, and performing max pooling on the sixth feature map along the channel dimension to obtain a second spatial feature map; concatenating the first spatial feature map and the second spatial feature map to obtain a target spatial feature map; performing a convolution operation on the target spatial feature map and inputting the convolutional target spatial feature map into a target activation function to obtain spatial attention weights, wherein the spatial attention weights are used to characterize the importance of each region in the sixth feature map; and generating the first feature map based on the spatial attention weights, wherein the first feature map contains the spatial attention scores corresponding to each region.

[0011] Optionally, based on the third feature map, the target object in the image to be identified is determined: a candidate box is generated on the third feature map based on the region extraction network; the target category corresponding to the candidate box is determined based on the region selected by the candidate box in the third feature map; if the target category is the target object, the candidate box is adjusted to obtain the target box, wherein the target box includes the target object, and the proportion of the region where the target object is located to the region selected by the candidate box is less than the proportion of the region where the target object is located to the region selected by the target box.

[0012] According to another aspect of the embodiments of this application, a target detection apparatus is also provided, comprising: a feature extraction module, configured to extract features based on an image to be identified to obtain multiple first feature maps of different scales; a spatial mapping module, configured to divide the first feature maps into multiple image blocks and rearrange the image blocks in the first feature maps to obtain a second feature map; a self-attention module, configured to calculate self-attention weights based on the second feature maps and generate a third feature map based on the self-attention weights, wherein the self-attention weights are used to characterize the degree of influence of each feature in the second feature map on identifying a target object in the image to be identified; and a target detection module, configured to determine the target object in the image to be identified based on the third feature map.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a target detection method during runtime.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes a target detection method by running the computer program.

[0015] In this embodiment, feature extraction is performed on the image to be identified to obtain multiple first feature maps at different scales. The first feature maps are divided into multiple image blocks, and the image blocks in the first feature maps are rearranged to obtain second feature maps. Based on the second feature maps, self-attention weights are calculated, and a third feature map is generated based on the self-attention weights. The self-attention weights are used to characterize the degree of influence of each feature in the second feature map on the identification of the target object in the image to be identified. Based on the third feature map, the target object in the image to be identified is determined. By innovatively implementing the BABEL attention algorithm (based on an adaptive weighted attention mechanism) on the basis of the Feature Pyramid Network (FPN), the image is divided into squares based on the feature map of the tower image. The x and y axes are divided with the block size as the factor interval. The calculation path is reordered, and the tower feature map is moved from the vertical to the horizontal block position. Then, the new feature map is divided into blocks, and self-attention calculation is performed in each block. The features of adjacent blocks are incorporated, which achieves the purpose of improving the detection accuracy of multi-scale tower detection. This solves the technical problem of poor accuracy in target detection and localization in neural networks used in related technologies. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a target detection method according to an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of a target detection method flow according to an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of an improved feature pyramid network and attention network according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of a method for optimizing the feature map of a tower image according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram illustrating an example of an application of a tower image feature map optimization method according to an embodiment of this application;

[0022] Figure 6 This is a schematic diagram illustrating how a column space is mapped to a block space, using a ground base station tower image as an example, according to an embodiment of this application.

[0023] Figure 7 This is a schematic diagram illustrating how a column space is mapped to a block space, using a tower feature map as an example, according to an embodiment of this application.

[0024] Figure 8 This is a schematic diagram of a self-attention architecture for the BABEL algorithm according to an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of a system architecture for a target detection algorithm provided according to an embodiment of this application;

[0026] Figure 10 This is a schematic diagram illustrating the effect of applying a target detection algorithm to detect tower targets according to an embodiment of this application;

[0027] Figure 11 This is a schematic diagram illustrating attention visualization of five feature maps at different scales according to an embodiment of this application;

[0028] Figure 12 This is a schematic diagram of the structure of a target detection device provided according to an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] The network models described above in related technologies suffer from poor detection accuracy during actual target detection. To address this issue, this application provides a solution, which is detailed below.

[0032] According to an embodiment of this application, a method embodiment for target detection is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware structure block diagram of a computer terminal (or electronic device) for implementing a target detection method is shown. Figure 1 As shown, the computer terminal 10 (or electronic device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0034] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or electronic device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the target detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned target detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0037] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or electronic device).

[0038] Under the above operating environment, this application provides a target detection method. Figure 2 This is a schematic diagram of a target detection method flow provided according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0039] Step S202: Feature extraction is performed on the image to be identified to obtain multiple first feature maps at different scales;

[0040] In some embodiments of this application, feature extraction based on the image to be identified to obtain multiple first feature maps of different scales includes the following steps: based on the multi-layer convolutional kernels in the feature pyramid network, feature extraction is performed sequentially from bottom to top to obtain multiple first candidate maps of different scales, wherein each convolutional kernel corresponds to a first candidate map, and the scale of the first candidate map corresponding to each convolutional kernel decreases sequentially from bottom to top, while the feature information contained in the first candidate map increases sequentially; for the first candidate map corresponding to the top convolutional kernel, multiple upsampling operations are performed sequentially from top to bottom to obtain multiple second candidate maps of different scales, wherein each upsampling operation corresponds to a second candidate map; the first candidate maps and second candidate maps of the same scale are fused to obtain multiple fifth feature maps of different scales; the fifth feature maps are adjusted through a channel attention module and a spatial attention module to obtain the first feature map.

[0041] Specifically, through the improved Feature Pyramid Network (FPN), feature fusion from bottom to top and from top to bottom, along with lateral connections of feature maps, is achieved. Top-level features are fused with bottom-level features via upsampling to construct a multi-scale image pyramid feature structure. The improved pyramid-shaped target detection algorithm, based on FPN, implements channel attention and spatial attention algorithms for the feature pyramid, consisting of channel attention and spatial attention mechanisms, such as... Figure 3 As shown, attention features are incorporated along two different dimensions, channel and space, to achieve sequential attention from channel to space. Experiments show that the improved and innovative attention algorithm enhances the performance of pyramid-shaped object recognition classification and detection. By implementing an attention mechanism in both channel and space dimensions, the algorithm improves model performance, achieving a mean average precision (mAP) of over 95%.

[0042] In this embodiment, the ResNet Backbone of the Faster R-CNN algorithm can be used to calculate features composed of feature maps at multiple scales. An FPN+Attention feature pyramid hierarchical structure is employed to fuse abstract semantic information from the bottom to the top layers and from the top layers to the bottom layers, constructing a feature pyramid incorporating an attention mechanism. The following section... Figure 3 The network shown will be described in further detail.

[0043] Specifically, feature maps (i.e., the first candidate maps mentioned above) are first constructed from the bottom up using ResNet Backbone: multiple feature maps (first candidate maps) of different scales and different numbers of channels are constructed from the bottom up to form different feature layers. The data (3×608×608) of the pyramid-shaped image (i.e., the image to be identified mentioned above) is input from bottom to top. After passing through the convolution kernel, the output feature map (first candidate image) is calculated. The output of the first layer feature map (first candidate image) is 64×152×152, which serves as the stem layer of the backbone network and does not participate in the FPN feature pyramid calculation. The output of the second layer feature map (first candidate image) is 256×152×152, the output of the third layer feature map (first candidate image) is 512×76×76, the output of the fourth layer feature map (first candidate image) is 1024×38×38, and the output of the fifth layer feature map (first candidate image) is 2048×19×19. The output of each stage of the second, third, fourth, and fifth layers is selected to form the feature set of the backbone network, and a feature pyramid is created in the FPN network.

[0044] Then, the FPN Neck network obtains multiple second candidate maps of different scales through top-down upsampling, and fuses feature maps of different spatial sizes through lateral connections (i.e., fusing the first and second candidate maps of the same scale as described above): upsampling high-level feature maps to incorporate abstract high-level features, and lateral connections to fuse the upsampled feature maps and the backbone network feature maps to locate target information. The output of the second layer feature map (second candidate map) of the Feature Pyramid FPN is represented as 256×152×152, the third layer feature map (second candidate map) as 256×76×76, the fourth layer feature map (second candidate map) as 256×38×38, and the fifth layer feature map (second candidate map) as 256×19×19. The feature set of each level of the Feature Pyramid is fixed in number of channels, and the feature dimension of each convolutional layer output channel is 256 channels.

[0045] By merging feature maps of different spatial sizes through horizontal connection, that is, by merging the first and second candidate maps of the same scale, multiple fifth feature maps of different scales are obtained; the fifth feature maps are then adjusted through the channel attention module and the spatial attention module to obtain the first feature map.

[0046] Specifically, channel attention and spatial attention are applied to the Faster R-CNN algorithm. Based on the Faster R-CNN algorithm, the attention mechanism of the Feature Pyramid Network (FPN) is improved. Through the lightweight network structure of channel and spatial attention networks, attention feature maps (i.e. the first feature map mentioned above) are constructed in the channel dimension and spatial dimension according to the fifth feature map mentioned above, which enhances the feature representation of multi-scale images with different spatial pyramid shapes.

[0047] like Figure 3 As shown, the output of the second layer feature map (first feature map) of channel and spatial attention A2 is 256×152×152, the output of the third layer feature map (first feature map) of channel and spatial attention A3 is 256×76×76, the output of the fourth layer feature map (first feature map) of channel and spatial attention A4 is 256×38×38, the output of the fifth layer feature map (first feature map) of channel and spatial attention A5 is 256×19×19, and the sixth layer of channel and spatial attention A6, based on the fifth layer, uses Maxpooling to output the sixth layer feature map (first feature map), and the output of the feature map is 256×10×10. A total of 5 feature maps (first feature maps) {A2, A3, A4, A5, A6} are constructed.

[0048] To address the issue of varying object sizes in tower images, this application proposes a layered pyramid structure. Different feature map scales are adapted to corresponding feature block segmentation, enabling precise displacement and combination of tower image window blocks.

[0049] The following section further describes the method of adjusting the fifth feature map using the channel attention module and the spatial attention module to obtain the first feature map.

[0050] In some embodiments of this application, adjusting the fifth feature map to obtain the first feature map through a channel attention module and a spatial attention module includes the following steps: performing average pooling on the fifth feature map through the channel attention module to obtain the first channel feature map, and performing max pooling on the fifth feature map to obtain the second channel feature map, wherein the spatial scales of the first channel feature map and the second channel feature map are both preset values; determining the channel attention weights corresponding to the fifth feature map based on the first channel feature map and the second channel feature map, wherein the channel attention weights are used to characterize the importance of each channel of the fifth feature map; generating a sixth feature map based on the channel attention weights, wherein the sixth feature map contains the channel attention scores corresponding to each channel; and adjusting the sixth feature map through the spatial attention module to obtain the first feature map.

[0051] Specifically, for the channel attention module, a tower feature map F (the fifth feature map) is input. Channel attention is calculated using max pooling and average pooling to generate feature matrices in different spaces. Then, the average pooling features (i.e., the first channel feature map mentioned above) and the max pooling features (i.e., the second channel feature map mentioned above) are aggregated to improve the feature representation capability of the network. The aggregated features are then input into a shared multilayer neural network (MLP), and the final channel attention feature output result, i.e., the sixth feature map mentioned above, is obtained by summing point by point and applying the activation function (Sigmoid).

[0052] In some embodiments of this application, adjusting the sixth feature map to obtain the first feature map includes the following steps: performing average pooling on the sixth feature map along the channel dimension to obtain the first spatial feature map, and performing max pooling on the sixth feature map along the channel dimension to obtain the second spatial feature map; concatenating the first spatial feature map and the second spatial feature map to obtain the target spatial feature map; performing a convolution operation on the target spatial feature map and inputting the convolutional target spatial feature map into the target activation function to obtain spatial attention weights, wherein the spatial attention weights are used to characterize the importance of each region in the sixth feature map; and generating the first feature map based on the spatial attention weights, wherein the first feature map contains the spatial attention score corresponding to each region.

[0053] Specifically, for the spatial attention module, an iron tower feature map F (sixth feature map) is input. The spatial attention module applies average pooling and max pooling operations to connect the average pooling and max pooling results (i.e., connecting the first spatial feature map and the second spatial feature map as mentioned above). Then, the final spatial attention feature output result, i.e., the first feature map mentioned above, is obtained through a convolutional neural network and a sigmoid activation layer.

[0054] Traditional convolutional neural networks only focus on channel attention analysis, which is limited to the channel effect of feature maps and constrained by channel attention features. This application proposes a channel attention and spatial attention method, which introduces spatial attention and channel attention from two dimensions: channel and space, to realize a sequential attention structure from channel to space.

[0055] By integrating channel attention and spatial attention modules, tower features are extracted sequentially along the channel and spatial dimensions. The attention mechanism enhances the expressive power of the tower feature map, improves the feature representation of the region of interest, focuses on important features, and suppresses unimportant features. Practical results show that the improved channel attention and spatial attention algorithms have wide applicability in tower recognition and target detection. They can learn the information of the target region well and aggregate features, thereby improving the accuracy of tower target detection.

[0056] Step S204: Divide the first feature map into multiple image blocks and rearrange the image blocks in the first feature map to obtain the second feature map;

[0057] The height of the target object in the image to be identified is greater than its width. In this embodiment, the target object is an iron tower as an example.

[0058] In some embodiments of this application, rearranging the image blocks in the first feature map to obtain the second feature map includes the following steps: determining multiple image block sets based on the image blocks in each column of the first feature map, wherein each image block set corresponds to a column in the first feature map; combining the image blocks in the image block sets into a target sub-map, wherein each image block set corresponds to a target sub-map; and determining a second feature map corresponding to the first feature map based on the target sub-map, wherein the first feature map and the second feature map have the same scale, and the arrangement order of the image blocks in the first feature map and the second feature map is different.

[0059] The following examples illustrate the rearrangement method described above. Figure 4 As shown, the matrix feature map is optimized by dividing the image into blocks and reordering the order through optimization of the loop variable. This converts columns sequentially into rows within the image blocks, providing a user-friendly way to access images for tower services. Figure 5 The image shown is an example of the tower image feature map optimization method implemented in this application. Taking the top left corner of the image as the origin, the horizontal axis as the x-axis and the vertical axis as the y-axis, the x and y axes are divided into 16 blocks with a factor of 4. The calculated path is reordered and each column of image blocks is mapped to the block space. For example, after sampling the image blocks A00, A10, A20, and A30 in the first column, they are filled into the new mapped block space (i.e., the target sub-image mentioned above) from left to right and from top to bottom. Here, it is mapped to the 0th block space.

[0060] like Figure 6 As shown, using a ground base station tower as an example, the feature map optimization method of this application can be illustrated. The tower image can be divided into multiple blocks like tiles, and optimization calculations can be performed piece by piece. Figure 7 As shown, in the actual operation of the feature map code, a new empty feature map is first generated with random variables. Then, after the transformation and recombination of the x and y coordinates, features are obtained from the original feature map by column (i.e., the set of image blocks is determined), and recombined into a feature block (i.e., the target sub-map mentioned above). The block is then filled into the corresponding block position of the new feature map. This process is repeated for each column of the original feature map, and each block cell of the new feature map is filled in turn. The improved code instructions are used to realize the transformation of the tower feature map from column space to block space, and the filling is carried out in turn.

[0061] Building upon the aforementioned feature map optimization methods, this application proposes a novel attention mechanism. It introduces a new image patch local attention mechanism and develops an improved AI attention model. Based on the optimization method for the feature map of the Eiffel Tower image, it maps the Query matrix, Key matrix, and Value matrix to a new space from a fresh perspective, forming the BABEL attention algorithm. Figure 8 This is a schematic diagram of a self-attention architecture for the BABEL algorithm according to an embodiment of this application, as shown below. Figure 8 As shown, the details are as follows:

[0062] First, in the forward propagation algorithm, the input feature map x(dim,height,width) (i.e., the first feature map mentioned above) is optimized based on the feature map optimization method. The correspondence between the block size (block_size), the number of attention heads (n_heads), and the number of channels (dim) of the feature map is adapted. The columns in the feature map are converted into rows in the image block in order, and the order is rearranged to map from the column space to the block space, thus constructing a new feature map (i.e., the second feature map mentioned above).

[0063] Regarding the problem that standard attention mechanisms in related technologies cannot be applied to image applications due to the lack of interrelation between windows and the inability of features to interact between adjacent block windows, this application proposes a method of moving block windows to enable interaction between image features of two adjacent block windows, thereby achieving the ability to model global image features and making it more suitable for object detection tasks.

[0064] After obtaining the second feature map, the steps of the BABEL attention algorithm are explained in step S206 below.

[0065] Step S206: Calculate the self-attention weights based on the second feature map, and generate a third feature map based on the self-attention weights. The self-attention weights are used to characterize the degree of influence of each feature in the second feature map on the recognition of the target object in the image to be recognized.

[0066] In some embodiments of this application, calculating self-attention weights and generating a third feature map based on the self-attention weights includes the following steps: performing a linear transformation on the second feature map to obtain a query matrix, a key matrix, and a value matrix; determining self-attention weights based on the query matrix and the key matrix; performing a weighted calculation based on the self-attention weights and the value matrix to obtain a fourth feature map, wherein the feature map contains the self-attention scores corresponding to each image block; rearranging the image blocks in the set of image blocks corresponding to each target sub-map that makes up the fourth feature map and combining them into a third feature map, wherein the first feature map and the third feature map have the same scale, and the arrangement order of the image blocks in the first feature map and the third feature map is the same.

[0067] Specifically, the second feature map is processed by a weight matrix W. Q A linear transformation is performed to construct a new Query matrix. Similarly, after linear transformations of different weight matrices, key and value matrices are constructed. A series of sampling and attention operations are performed, and improved self-attention is used to capture the spatial interaction between the feature maps of the Eiffel Tower image. This enhances the receptive field of each image feature, enabling the detection of image features of larger target objects. Various model variants can be derived. The column space to block space mapping transformation can be performed on the Query matrix, Key matrix, and Value matrix respectively. The column space and block space transformation can also be performed on the image feature maps of adjacent window blocks of the Key-Value joint matrix.

[0068] Based on the query matrix, key matrix, and value matrix, perform BABEL self-attention calculation on the tower feature map (second feature map). Then, transform the new feature map (fourth feature map) after attention calculation into the dimension of the original input feature map through a linear transformation to obtain the third feature map.

[0069] To address the issue that traditional convolutional neural network pooling layers lose some feature information and ignore the relationship between local and global contexts, this application proposes an AI attention prediction method based on Q, K, and V matrices. The aim is to incorporate the characteristics of the Eiffel Tower image into the attention Q, K, and V matrices by combining the features of the Eiffel Tower image window displacement, thereby capturing the global contextual information of the image during target detection and making target detection and localization more accurate.

[0070] Step S208: Based on the third feature map, determine the target object in the image to be identified.

[0071] In some embodiments of this application, determining the target object in the image to be identified based on the third feature map includes the following steps: generating candidate boxes on the third feature map based on a region extraction network; determining the target category corresponding to the candidate box based on the region selected by the candidate box in the third feature map; and adjusting the candidate box to obtain the target box when the target category is the target object, wherein the target box includes the target object, and the proportion of the region where the target object is located to the region selected by the candidate box is less than the proportion of the region where the target object is located to the region selected by the target box.

[0072] Figure 9 This is a schematic diagram of a system architecture for a target detection algorithm provided in an embodiment of this application, such as... Figure 9 As shown, tower image recognition is a target detection task. Common tower types include simple towers, landscape towers, rooftop towers, ordinary ground towers, and ordinary rooftop towers. Based on the Faster R-CNN network, this application proposes an improved target detection algorithm based on an attention mechanism. The algorithm uses a ResNet backbone network to extract image features and a Feature Pyramid Network (FPN) to extract multi-scale features. It innovatively adopts a channel attention mechanism, a spatial attention mechanism, and a BABEL attention algorithm to achieve feature fusion. Then, it uses an RPN network (i.e., the aforementioned region extraction network) and a Standard RoI Head to predict the tower bounding box category and tower bounding box location, thereby improving the ability to detect multiple targets in tower classification scenarios.

[0073] This application provides a target detection method and system based on an improved AI attention mechanism. It proposes the BABEL attention algorithm, which implements channel attention and spatial attention. Based on an open-source computer vision framework, it embeds the improved attention algorithm and implements the backbone network, neck network, and head network modules of the tower recognition target detection algorithm model. It improves the standard target detection network from the aspects of feature pyramid and attention mechanism.

[0074] In actual production and daily life, the target detection method provided in this application is applied to intelligently identify tower type images, audit the matching degree between the tower type identified in the photo and the tower type of the leased unit, automatically detect errors in billing parameters such as tower type, and update the resource system to ensure that the data of the existing network is consistent with the system data and tower bills, strengthen the basic management of towers, and achieve the effect of improving the efficiency of base station AI intelligent operation through data-driven approach.

[0075] The target detection method in this application can be used to output the category of tower-shaped targets and locate their positions. The mAP (memory accuracy) is used as the evaluation metric for tower-shaped target detection. The mAP value of the target detection method in this application reaches 95.6%. The performance of various target detection algorithms on different datasets was also compared. In the benchmark dataset, the mAP of YoloV3, Faster R-CNN, and the improved Faster R-CNN+Attention was compared. YoloV3's mAP was 88.2%, Faster R-CNN's was 91.6%, and the improved Faster R-CNN+Attention's was 92.6%. In the augmented dataset, using the improved Faster R-CNN+Attention algorithm, the mAP was improved to 95.6%, an improvement of 3%. High-quality image datasets have a significant impact on the model's performance. The use of improved channel and spatial attention feature pyramids allows the model to better learn global and local features, improving the detection performance on multi-scale targets.

[0076] Figure 10 This is a schematic diagram illustrating the effect of applying a target detection algorithm to detect tower targets according to an embodiment of this application. Figure 10 As shown, the improved tower type recognition target detection method can accurately identify the type of pole-mounted tower.

[0077] As an optional implementation, the present application's solution can also visualize the relationship between the internal structure of the feature map and the output structure, making the improved AI attention mechanism more interpretable and thus solving the black-box problem of traditional neural network training parameters. Figure 11 As shown, five feature maps of the improved feature pyramid to be displayed are obtained, and attention visualizations of the five feature maps at different scales are drawn based on the visualization function of MMEngine.

[0078] As an optional implementation method, this application addresses the problem of the limited number of pyramid recognition image samples during network model training by constructing a custom pyramid dataset, including VOC and COCO formats, and implementing Mixup, Mosaic, and Random data augmentation.

[0079] As an alternative implementation method, this application proposes a model compression and acceleration method to address the problem of increased model parameters (increased computation and memory overhead). It applies a teacher-student model to implement knowledge distillation technology, thereby promoting the application and research of model compression in the field of computer vision for tower target detection.

[0080] Traditional convolutional neural networks (CNNs) use convolutional kernels to extract local features of images. Lower-level networks represent detailed local features, while higher-level networks represent more abstract features. In traditional CNNs, the number of parameters is closely related to the range of the convolutional kernel during convolution, but the interaction between the kernel and the feature map is not linked to the content of the input image. To address these issues, this application proposes the BABEL attention algorithm, which has several advantages: Firstly, it fully utilizes the business features of tower images. Starting from the data features of tower images, it better captures the image features of the tower. Based on the vertical rectangular shape of the tower, the feature map is cut into square blocks, and after transformation and recombination of x and y coordinates, feature maps with the same tower content are concentrated in one region. Secondly, it improves the implementation of the Q, K, V matrix for self-attention. The standard Q, K, V matrix is ​​formed by the same input image feature information X through three weight matrices W. Q W K W V It is a linear transformation that expresses the relationship between the same image feature sequence blocks. The improved BABEL attention algorithm has been improved and various attention variant structures have been derived. Inputting a tower image feature, it is transformed from vertical feature to horizontal block feature by dividing the feature blocks, creating a new tower feature map in the new transfer space. Based on the new tower feature map, a new Q matrix or K, V matrix is ​​mapped. From a new perspective, the relationship of the Q, K, V matrices that were originally only located in the ontology space has been transferred to express the attention relationship between the ontology space and the new transfer space, expressing the self-association and self-mapping relationship of the same tower feature map in different spaces. One advantage is that self-attention local feature calculation is performed within each window block. This allows the self-attention mechanism to interact based on the feature map of the tower itself, rather than convolutional interactions unrelated to image features. By constructing a KV joint neighbor window using the K and V matrices, global features between adjacent windows are captured through the transformation of the joint KV matrix. This captures the ability to capture global image features that depend on long distances, thus capturing the overall image features of the tower. In this way, it has both the local features of the tower in each window block and the global features of the entire tower feature map, thus improving and implementing a new self-attention BABEL model.

[0081] Furthermore, this application implements both channel attention and spatial attention algorithms based on the BABEL attention algorithm. Traditional convolutional neural networks have been studied from the perspectives of network depth (LeNet, VGGNet, ResNet), network width (GoogLeNet), and network dimension (Xception, ResNeXt) to enhance the network's feature representation capabilities. This application implements channel attention and spatial attention algorithms from the perspective of network structure attention, further improving the image's feature representation capabilities from the attention level. Given a tower feature map as input, it generates fused attention feature map information sequentially from both channel attention and spatial attention dimensions. Its purpose is to focus the model's attention on the important features of the tower image itself, ignoring unimportant regional image information and irrelevant noise. By combining channel and spatial dimensions for analysis, network performance is further improved. The channel attention and spatial attention algorithms themselves are lightweight modules, which are improved and embedded into the feature pyramid for tower recognition target detection, performing feature fusion of channel attention and spatial attention on feature maps at different scales.

[0082] Based on the BABEL attention algorithm, this application also provides an improved feature pyramid algorithm. The standard feature pyramid network is based on a deep convolutional neural network, constructing feature fusion and lateral connections between feature maps from bottom to top and from top to bottom. It fuses top-level features with bottom-level features through upsampling to construct multi-scale image pyramid features. The improved feature pyramid algorithm in this application incorporates channel attention and spatial attention algorithms, as well as the BABEL attention algorithm, especially the BABEL attention algorithm. This involves the correspondence between the scale of the input tower feature map and the number of input channels, the number of attention heads, and the size of the partitioning window in the BABEL attention algorithm. The parameters of the BABEL attention algorithm are automatically adapted to feature maps of different scales. First, bottom-up feature calculation is performed to obtain relatively low-level semantic feature maps of different scales, forming a hierarchical feature structure. Then, top-down upsampling is performed to construct higher-level, more abstract, and higher-resolution feature maps. Finally, through lateral connections, the bottom-up and top-down feature maps of the same scale are fused, embedding features extracted by the channel attention and spatial attention algorithms and the BABEL attention algorithm. The final output is a series of hierarchical tower feature map results. High-level, deep features are favorable for large targets and rich in semantic information, but easily overlook small targets. Low-level, shallow features extract less semantic information and have a smaller scope of perception, but are better able to focus on small objects.

[0083] Through the above steps, the BABEL attention algorithm (based on an adaptive weighted attention mechanism) was innovatively implemented on the basis of the Feature Pyramid Network (FPN). Based on the feature map of the Eiffel Tower image, the image is divided into squares, and the x and y axes are divided with block size as the interval factor. The calculation path is reordered, and the Eiffel Tower feature map is moved from the vertical to the horizontal block position. Then, the new feature map is divided into blocks, and self-attention calculation is performed in each block, and the features of adjacent blocks are incorporated. This achieves the goal of improving the detection accuracy of multi-scale tower detection, and thus solves the technical problem of poor accuracy in target detection and localization in neural networks used for target detection in related technologies.

[0084] According to an embodiment of this application, an embodiment of a target detection device is also provided. Figure 12 This is a schematic diagram of the structure of a target detection device according to an embodiment of this application. Figure 12 As shown, the device includes:

[0085] The feature extraction module 120 is used to extract features from the image to be recognized and obtain multiple first feature maps at different scales.

[0086] The spatial mapping module 122 is used to divide the first feature map into multiple image blocks and rearrange the image blocks in the first feature map to obtain the second feature map;

[0087] The self-attention module 124 is used to calculate self-attention weights based on the second feature map and generate a third feature map based on the self-attention weights. The self-attention weights are used to characterize the degree of influence of each feature in the second feature map on the recognition of the target object in the image to be recognized.

[0088] The target detection module 126 is used to determine the target object in the image to be identified based on the third feature map.

[0089] It should be noted that each module in the above-mentioned target detection device can be a program module (for example, a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0090] It should be noted that the target detection device provided in this embodiment can be used to perform... Figure 2 The target detection method shown above is also applicable to the embodiments of this application, and will not be repeated here.

[0091] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device containing the non-volatile storage medium executes the following target detection method by running the computer program: extracting features from the image to be identified to obtain multiple first feature maps of different scales; dividing the first feature maps into multiple image blocks and rearranging the image blocks in the first feature maps to obtain second feature maps; calculating self-attention weights based on the second feature maps and generating a third feature map based on the self-attention weights, wherein the self-attention weights are used to characterize the degree of influence of each feature in the second feature map on identifying the target object in the image to be identified; and determining the target object in the image to be identified based on the third feature map.

[0092] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0093] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0097] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0098] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A target detection method characterized by, The method comprises the following steps: feature extraction is performed on the to-be-identified image to obtain a plurality of first feature maps of different scales; the first feature map is divided into a plurality of image blocks, and the image blocks in the first feature map are rearranged to obtain a second feature map, comprising: determining a plurality of image block sets according to the image blocks in each column of the first feature map, wherein each image block set corresponds to a column of the first feature map; combining the image blocks in the image block set into a target subgraph, wherein each image block set corresponds to a target subgraph; determining the second feature map corresponding to the first feature map according to the target subgraph, wherein the first feature map and the second feature map have the same scale, and the arrangement order of the image blocks in the first feature map and the second feature map is different, wherein the height of the target object contained in the to-be-identified image is greater than the width; self-attention weights are calculated according to the second feature map, and a third feature map is generated according to the self-attention weights, wherein the self-attention weights are used to represent the influence degree of each feature in the second feature map on identifying the target object in the to-be-identified image; determining the target object in the to-be-identified image according to the third feature map.

2. The object detection method of claim 1, wherein, calculating self-attention weights and generating a third feature map according to the self-attention weights comprises: performing linear transformation on the second feature map to obtain a query matrix, a key matrix and a value matrix; determining the self-attention weights according to the query matrix and the key matrix; performing weighted calculation according to the self-attention weights and the value matrix to obtain a fourth feature map, wherein the fourth feature map contains self-attention scores corresponding to each image block; rearranging the image blocks in the image block set corresponding to each target subgraph constituting the fourth feature map and combining them into the third feature map, wherein the first feature map and the third feature map have the same scale, and the arrangement order of the image blocks in the first feature map and the third feature map is the same.

3. The object detection method of claim 1, wherein, feature extraction is performed on the to-be-identified image to obtain a plurality of first feature maps of different scales, comprising: feature extraction is performed on the to-be-identified image to obtain a plurality of first feature maps of different scales, comprising: feature extraction is performed on the to-be-identified image to obtain a plurality of first feature maps of different scales, comprising: feature extraction is performed on the to-be-identified image to obtain a plurality of first feature maps of different scales, comprising: feature extraction is performed on the to-be-identified image to obtain a plurality of first feature maps of different scales, comprising: the first candidate graph corresponding to the topmost convolution kernel is sequentially subjected to a plurality of upsampling operations from top to bottom to obtain a plurality of second candidate graphs of different scales, wherein each upsampling operation corresponds to a second candidate graph; the first candidate graph and the second candidate graph with the same scale are fused to obtain a plurality of fifth feature maps of different scales; the fifth feature map is adjusted through a channel attention module and a spatial attention module to obtain the first feature map.

4. The object detection method of claim 3, wherein, The fifth feature map is adjusted through the channel attention module and the spatial attention module to obtain the first feature map, including: The fifth feature map is subjected to an average pooling operation through the channel attention module to obtain a first channel feature map, and is subjected to a maximum pooling operation to obtain a second channel feature map, wherein the spatial scales of the first channel feature map and the second channel feature map are preset values; The channel attention weight corresponding to the fifth feature map is determined according to the first channel feature map and the second channel feature map, wherein the channel attention weight is used to represent the importance of each channel of the fifth feature map; The sixth feature map is generated according to the channel attention weight, wherein the sixth feature map contains the channel attention score corresponding to each channel. The sixth feature map is adjusted through the spatial attention module to obtain the first feature map.

5. The object detection method of claim 4, wherein, The sixth feature map is adjusted to obtain the first feature map, including: The sixth feature map is subjected to an average pooling operation in the channel dimension to obtain a first spatial feature map, and is subjected to a maximum pooling operation in the channel dimension to obtain a second spatial feature map; The first spatial feature map and the second spatial feature map are connected to obtain a target spatial feature map; The target spatial feature map is subjected to a convolution operation, and the target spatial feature map after the convolution operation is input into a target activation function to obtain a spatial attention weight, wherein the spatial attention weight is used to represent the importance of each region in the sixth feature map; The first feature map is generated according to the spatial attention weight, wherein the first feature map contains the spatial attention score corresponding to each region.

6. The object detection method of claim 1, wherein, The target object in the to-be-recognized image is determined according to the third feature map: A candidate box is generated on the third feature map according to a region extraction network; The target class corresponding to the candidate box is determined according to the region framed by the candidate box in the third feature map; In the case that the target class is the target object, the candidate box is adjusted to obtain a target box, wherein the target object is included in the target box, and the proportion of the region where the target object is located in the region framed by the candidate box is smaller than the proportion of the region where the target object is located in the region framed by the target box.

7. A target detection apparatus characterized by comprising: Including: A feature extraction module is configured to perform feature extraction on a to-be-recognized image to obtain a plurality of first feature maps of different scales; The spatial mapping module is configured to divide the first feature map into a plurality of image blocks, and rearrange the image blocks in the first feature map to obtain a second feature map, including: determining a plurality of image block sets according to the image blocks in each column of the first feature map, wherein each image block set corresponds to a column of the first feature map; combining the image blocks in the image block sets into target subgraphs, wherein each image block set corresponds to a target subgraph; determining the second feature map corresponding to the first feature map according to the target subgraphs, wherein the first feature map and the second feature map have the same size, and the arrangement order of the image blocks in the first feature map and the second feature map is different, and the target object contained in the to-be-identified image has a height greater than a width; The self-attention module is configured to calculate a self-attention weight according to the second feature map, and generate a third feature map according to the self-attention weight, wherein the self-attention weight is used to represent the influence degree of each feature in the second feature map on identifying the target object in the to-be-identified image; The target detection module is configured to determine the target object in the to-be-identified image according to the third feature map.

8. An electronic device, comprising: It includes: The memory and the processor are used to run the program stored in the memory, wherein the program executes the target detection method in any one of claims 1 to 6.

9. A non-volatile storage medium, comprising: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the target detection method in any one of claims 1 to 6 by running the computer program.

Images