A recognition method and device based on feature pyramid and attention mechanism

By introducing feature pyramids and attention mechanisms into convolutional neural networks, the problem of convolutional neural networks ignoring shallow features is solved, and the accuracy of data recognition and the generalization ability of the model are improved.

CN117612168BActive Publication Date: 2024-11-05SHENZHEN WANZHIDA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311612636.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-29
Publication Date
2024-11-05
Estimated Expiration
2043-11-29

AI Technical Summary

Technical Problem

Existing convolutional neural networks tend to ignore shallow features during data recognition, resulting in low recognition accuracy.

Method used

Using the recognition method based on feature pyramids and attention mechanisms, deep features are extracted through top-down feature paths, and focused on important areas through bottom-up attention paths, and combined with the average prediction results of the backbone network, the recognition results are generated. .

Benefits of technology

The generalization ability of the model and the ability to extract features of important regions are improved, and the accuracy of data recognition is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117612168B_ABST
    Figure CN117612168B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a recognition method and device based on feature pyramid and attention mechanism, which relates to the technical field of data recognition technology. The method includes: obtaining first data to be recognized; transmitting the first feature of the first data to the second feature through a first feature path to obtain first feature information; focusing on the first feature information through a first attention path to obtain first attention information; determining the recognition result of the first data based on the first attention information and the average prediction result of the first backbone network on the first data. Through the present invention, the problem of inaccurate data recognition accuracy is solved, thereby achieving the effect of improving data recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of smoke recognition technology, and in particular, to a recognition method and device based on feature pyramid and attention mechanism. Background Art

[0002] Existing convolutional neural networks tend to ignore shallow features when processing deep features of data. This is because the later layers of the network pay more attention to high-level and abstract features when extracting features, while ignoring the detailed features of the underlying layers.

[0003] However, this ignored information may be the most important for data recognition, and thus ignoring these underlying detailed features can easily lead to certain errors in the final data recognition results.

[0004] Therefore, a new identification method is needed to identify the data. Summary of the invention

[0005] The embodiments of the present invention provide a recognition method and device based on feature pyramid and attention mechanism to at least solve the problem of low data recognition accuracy in related technologies.

[0006] According to an embodiment of the present invention, a recognition method based on feature pyramid and attention mechanism is provided, comprising:

[0007] Acquire first data to be identified;

[0008] Transmitting the first feature of the first data to the second feature through a first feature path to obtain first feature information; wherein the first feature path is established from top to bottom based on the first backbone network;

[0009] Focusing attention on the first feature information through a first attention path to obtain first attention information, wherein the first attention path is established from bottom to top based on the first backbone network;

[0010] Based on the first attention information and the average prediction result of the first backbone network on the first data, a recognition result of the first data is determined.

[0011] In an exemplary embodiment, before transmitting the first feature of the first data to the second feature through the first feature path to obtain the first feature information, the method includes:

[0012] Constructing a feature pyramid through a first convolution block and a first output feature of the first backbone network, wherein the first convolution block includes a plurality of specified convolution blocks, and the first output feature includes the last output feature of all convolution layers;

[0013] Acquire output feature data of the first data, wherein the output feature data corresponds to data blocks of the same spatial size in the first data;

[0014] The feature pyramid generates a feature hierarchy structure based on the local data of the output feature data, wherein the feature hierarchy structure corresponds to the first output feature;

[0015] The first feature path is determined based on the lateral connection of the feature hierarchy and the output feature data.

[0016] In an exemplary embodiment, before determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data, the method includes:

[0017] Performing a first convolution operation on the output feature data to obtain a first convolution result;

[0018] Performing an average pooling operation on the output feature data to obtain a first operation result, and performing a second convolution operation based on the first operation result to obtain a second convolution result;

[0019] The first convolution result and the second convolution result are horizontally calculated by using an activation function to obtain the horizontal connection.

[0020] In an exemplary embodiment, before focusing attention on the first feature information through the first attention path to obtain first attention information, the method includes:

[0021] generating a spatial attention mask based on the feature data of the local data;

[0022] Based on the spatial attention mask, determining a spatial attention pyramid;

[0023] Performing global average pooling processing on the fully connected layer of the first backbone network and the feature data of the feature pyramid to obtain channel attention information;

[0024] Based on the spatial attention pyramid and the channel attention information, determine the first attention path.

[0025] According to another embodiment of the present invention, a recognition device based on feature pyramid and attention mechanism is provided, comprising:

[0026] A data acquisition module, used for acquiring first data to be identified;

[0027] A feature transmission module, used for transmitting the first feature of the first data to the second feature through a first feature path to obtain first feature information; wherein the first feature path is established from top to bottom based on the first backbone network;

[0028] a data attention module, configured to focus attention on the first feature information through a first attention path to obtain first attention information, wherein the first attention path is established from bottom to top based on the first backbone network;

[0029] A data identification module is used to determine an identification result of the first data based on the first attention information and an average prediction result of the first backbone network on the first data.

[0030] In an exemplary embodiment, the apparatus further comprises:

[0031] a feature pyramid construction module, configured to construct a feature pyramid through a first convolution block and a first output feature of the first backbone network before transmitting the first feature of the first data to the second feature through the first feature path to obtain the first feature information, wherein the first convolution block includes a plurality of specified convolution blocks, and the first output feature includes the last output feature of all convolution layers;

[0032] a feature acquisition module, used for acquiring output feature data of the first data, wherein the output feature data corresponds to data blocks of the same spatial size in the first data;

[0033] A structure generation module, configured to generate a feature hierarchy structure based on the local data of the output feature data, wherein the feature hierarchy structure corresponds to the first output feature;

[0034] A feature path determination module is used to determine the first feature path based on the lateral connection of the feature hierarchy and the output feature data.

[0035] In an exemplary embodiment, the apparatus further comprises:

[0036] A first convolution module, configured to perform a first convolution operation on the output feature data to obtain a first convolution result before determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data;

[0037] A pooling convolution module, used to perform an average pooling operation on the output feature data to obtain a first operation result, and perform a second convolution operation based on the first operation result to obtain a second convolution result;

[0038] A horizontal calculation module is used to perform horizontal calculation on the first convolution result and the second convolution result through an activation function to obtain the horizontal connection.

[0039] In an exemplary embodiment, the apparatus further comprises:

[0040] an attention mask module, configured to generate a spatial attention mask based on the feature data of the local data before focusing the first feature information through the first attention path to obtain the first attention information;

[0041] An attention pyramid module, configured to determine a spatial attention pyramid based on the spatial attention mask;

[0042] an attention information determination module, configured to perform global average pooling processing on the feature data of the feature pyramid and the fully connected layer of the first backbone network to obtain channel attention information;

[0043] An attention path determination module is used to determine the first attention path based on the spatial attention pyramid and the channel attention information.

[0044] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0045] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0046] Through the present invention, the introduction of the first feature path and the first attention path helps to improve the generalization ability of the first backbone network model, so that it can achieve good performance in different data sets and environments, and strengthen the feature extraction of important areas. Therefore, the problem of low data recognition accuracy can be solved, thereby achieving the effect of improving data recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a hardware structure block diagram of a mobile terminal of a recognition method based on a feature pyramid and an attention mechanism according to an embodiment of the present invention;

[0048] Figure 2 is a flow chart of a recognition method based on feature pyramid and attention mechanism according to an embodiment of the present invention;

[0049] Figure 3is a schematic diagram of a spatial attention mask generation process according to a specific embodiment of the present invention;

[0050] Figure 4 is a schematic diagram of a channel attention mask generation process according to a specific embodiment of the present invention;

[0051] Figure 5 It is a structural block diagram of a recognition device based on feature pyramid and attention mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 is a hardware structure block diagram of a mobile terminal of a recognition method based on feature pyramid and attention mechanism according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0055] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a recognition method based on a feature pyramid and an attention mechanism in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0056] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0057] In this embodiment, a recognition method based on feature pyramid and attention mechanism is provided. Figure 2 is a flow chart of recognition based on feature pyramid and attention mechanism according to an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0058] Step S201, obtaining first data to be identified;

[0059] In this embodiment, the first data may be smoke image data to be identified, or laser point cloud data, or other types of data that need to be classified and identified through a neural network model.

[0060] Step S202: transmitting the first feature of the first data to the second feature through a first feature path to obtain first feature information; wherein the first feature path is established from top to bottom based on the first backbone network;

[0061] In this embodiment, a top-down feature path is introduced to extract features at different scales, thereby capturing data features of different sizes and types (such as resolution features) in the data, so that the model can better adapt to data forms and structures of different scales and improve the accuracy of data recognition.

[0062] Among them, the first feature path includes a top-down feature path, the first feature includes deep features of the data, such as shape, part and other feature data, and the second feature includes shallow features, such as edge, texture and other features; the first backbone network can be a VGG19 neural network model; the first feature information includes data after feature extraction through the first feature path.

[0063] Step S203, focusing attention on the first feature information through a first attention path to obtain first attention information, wherein the first attention path is established from bottom to top based on the first backbone network;

[0064] In this embodiment, the attention path focuses on the image region or feature channel of interest, which helps to extract detail information, highlight important targets or features, and reduce sensitivity to noise or interference, thereby improving the overall performance.

[0065] The first attention path includes the attention path, and the first attention information is information obtained by focusing on an important area in the first feature information.

[0066] Step S204: determining a recognition result of the first data based on the first attention information and an average prediction result of the first backbone network on the first data.

[0067] In this embodiment, the recognition result is obtained according to the average prediction result of the attention path and the backbone network, so that the features of the recognition result are more prominent and the noise is less, thereby ensuring the recognition accuracy.

[0068] Through the above steps, the introduction of the first feature path and the first attention path helps to improve the generalization ability of the first backbone network model, so that it can achieve good performance in different data sets and environments, and strengthen the feature extraction of important areas, thereby solving the problem of low data recognition accuracy and improving the effect of data recognition accuracy.

[0069] The execution subject of the above steps may be a base station, a terminal, etc., but is not limited thereto.

[0070] In an optional embodiment, before transmitting the first feature of the first data to the second feature through the first feature path to obtain the first feature information, the method includes:

[0071] Step S20201, constructing a feature pyramid through a first convolution block and a first output feature of the first backbone network, wherein the first convolution block includes a plurality of specified convolution blocks, and the first output feature includes the last output feature of all convolution layers;

[0072] Step S20202, obtaining output feature data of the first data, wherein the output feature data corresponds to data blocks of the same spatial size in the first data;

[0073] Step S20203, the feature pyramid generates a feature hierarchy structure based on the local data of the output feature data, wherein the feature hierarchy structure corresponds to the first output feature;

[0074] Step S20204: determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data.

[0075] In this embodiment, the top-down feature path extracts features of different scales at different levels of the network by a method constructed based on the idea of ​​feature pyramid. This can capture features of different sizes and improve the robustness to scale changes. Through the top-down feature path, high-level features can be passed to low-level features, thereby providing richer contextual information for low-level features. The transmission of this contextual information helps to improve the semantic expression ability of low-level features.

[0076] Among them, the top-down feature path first builds the VGG19 backbone network, and selects the first two convolution blocks, the third convolution block, and the last output features of all convolution layers in VGG19 to establish a pyramid level. The aforementioned first convolution block includes the first two convolution blocks and the third convolution block in VGG19. The feature pyramid can locate samples at different scales to focus on the subtle differences of objects at different scales.

[0077] Then use {B1,B2,…,B l} represents the output feature map of blocks with the same spatial size, where l represents the number of blocks, and the aforementioned output feature data includes the output feature map.

[0078] The feature pyramid selects a portion of these features and generates N corresponding feature hierarchies {F n ,F n+1 ,…,F n+N-1}(1≤n≤n+N-1< <l)。

[0079] By applying in B n+N-1 A top-down path follows, which goes from feature maps at higher pyramid levels to feature maps at lower pyramid levels, and lateral connections B between corresponding features k →B' k (k=n,n+1,…,n+N-1) maintains the information of the backbone network to obtain the first feature path. Specifically, the top-down feature path is obtained by doubling the feature map of the previous layer and adding it to the lateral connection of the layer:

[0080] F n+N-1 =ReLU(conv3×3(B' n+N-1 )) (1)

[0081] F n+1 =ReLU(conv3×3(upsample(B' n+N-1 )+B' n+1 )) (2)

[0082] F n =ReLU(conv3×3(upsample(B' n+1 )+B' n )) (3)

[0083] Among them, ReLU refers to the ReLU activation function, conv3×3 refers to a 3×3 convolution operation with a step size of 1 and a padding value of 1, and upsample is a double upsampling operation.

[0084] In an optional embodiment, before determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data, the method includes:

[0085] Step S2020401, performing a first convolution operation on the output feature data to obtain a first convolution result;

[0086] Step S2020402, performing an average pooling operation on the output feature data to obtain a first operation result, and performing a second convolution operation based on the first operation result to obtain a second convolution result;

[0087] Step S2020403: Performing horizontal calculation on the first convolution result and the second convolution result through an activation function to obtain the horizontal connection.

[0088] In this embodiment, the horizontal connection B k →B' k The calculation process can be expressed as:

[0089] B' k =ReLU(conv1×1(B k ))+ReLU(conv1×1(AvgPool(B k ))) (4)

[0090] Among them, ReLU refers to the ReLU activation function, conv1×1 refers to a 1×1 convolution operation with a step size of 1 and a padding value of 0, and AvgPool is an average pooling operation.

[0091] In the process of constructing the feature pyramid, the AvgPool operation is first applied, and then the 1×1 convolution is applied. This can reduce the size of the feature map, extract the average features, reduce the feature dimension, and enhance the expressiveness of the features. In this way, a more efficient and expressive feature pyramid can be constructed.

[0092] Directly performing 1x1 convolution can introduce nonlinear transformation and feature fusion, while performing AvgPool followed by 1x1 convolution can extract average features and reduce dimensions. The AvgPool operation will reduce the spatial resolution of the feature map and may lose some spatial information.

[0093] At this time, adding the result of direct 1x1 convolution with the result of AvgPool and then 1x1 convolution can make full use of their respective advantages, improve the diversity and richness of features, and further enhance the expressiveness of features.

[0094] Right now:

[0095] By adding the result of direct 1x1 convolution with the result of AvgPool followed by 1x1 convolution, the semantic information of low-resolution features can be fused while retaining high-resolution features, so that both detail information and the integrity of global semantic information can be considered when processing tasks, thereby achieving the effects of feature fusion, enhanced feature representation, spatial information retention, and network design flexibility. This can help the network better understand the input data, improve performance and generalization capabilities, and adapt to task requirements of different scales and semantic levels.

[0096] In an optional embodiment, before focusing attention on the first feature information through the first attention path to obtain the first attention information, the method includes:

[0097] Step S20301, generating a spatial attention mask based on the feature data of the local data;

[0098] Step S20302, determining a spatial attention pyramid based on the spatial attention mask;

[0099] Performing global average pooling processing on the fully connected layer of the first backbone network and the feature data of the feature pyramid to obtain channel attention information;

[0100] Step S20303, determine the first attention path based on the spatial attention pyramid and the channel attention information.

[0101] In this embodiment, the bottom-up attention path is constructed by introducing an additional attention hierarchy {A n ,A n+1 ,…,An+N-1}, further enhancing the feature pyramid structure. Locating more important regions and channel attention paths at different scales By enhancing channel correlation and reducing local information, a bottom-up attention path is constructed. This hierarchical structure is used to selectively emphasize the information features at different scales of the feature pyramid, which can improve the performance of the data recognition task.

[0102] refer to Figure 3 As shown, each block has a corresponding feature map F k As input, generate a spatial attention mask (i.e., the aforementioned spatial attention mask), specifically, each feature map F k First, a 3×3 deconvolution layer and an output channel are used to compress the spatial information. Then the spatial attention mask is converted to Normalized to the interval (0,1), where each pixel value represents the importance of the position in the entire feature map. The closer the value is to 1, the more important the position is; the closer the value is to 0, the less important the position is. This enhances the model's attention to different positions and extracts more useful spatial information.

[0103] Spatial Attention Mask The calculation process can be expressed as:

[0104]

[0105] Here σ is the sigmoid function, × represents deconvolution, and v c Represents the convolution kernel. Thus, we get the spatial attention pyramid based on multi-scale feature maps. (i.e. the aforementioned spatial attention pyramid).

[0106] refer to Figure 4 As shown, channel attention By combining two fully connected layer (FC) operations and global average pooling (GAP), the channel attention mask is obtained from the corresponding feature map in the feature pyramid. It can be expressed as:

[0107]

[0108] Among them, GAP is the global average pooling function:

[0109]

[0110] Here σ and ReLU refer to the sigmoid function and ReLU function respectively; dot product represents element-wise multiplication; W and H represent F kThe spatial dimensions of ; W1 and W2 are the weight matrices of the two FC layers. In this application, channel attention plays a different role from the spatial attention pyramid, because they provide low-level details from lower pyramid levels to higher pyramid levels in a bottom-up path.

[0111] Finally, use the calculated attention weighted feature F k , and get F′ k To classify:

[0112]

[0113] in Represents an addition operation using broadcast semantics. A classifier with one GAP layer and two FC layers makes the final prediction.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0115] In this embodiment, a recognition device based on a feature pyramid and an attention mechanism is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0116] Figure 5 is a structural block diagram of a recognition device based on feature pyramid and attention mechanism according to an embodiment of the present invention, such as Figure 5 As shown, the device comprises:

[0117] A data acquisition module 51 is used to acquire first data to be identified;

[0118] A feature transmission module 52, configured to transmit the first feature of the first data to the second feature through a first feature path to obtain first feature information; wherein the first feature path is established from top to bottom based on the first backbone network;

[0119] A data attention module 53, configured to focus attention on the first feature information through a first attention path to obtain first attention information, wherein the first attention path is established from bottom to top based on the first backbone network;

[0120] The data identification module 54 is used to determine the identification result of the first data based on the first attention information and the average prediction result of the first backbone network on the first data.

[0121] In an optional embodiment, the device further comprises:

[0122] a feature pyramid construction module, configured to construct a feature pyramid through a first convolution block and a first output feature of the first backbone network before transmitting the first feature of the first data to the second feature through the first feature path to obtain the first feature information, wherein the first convolution block includes a plurality of specified convolution blocks, and the first output feature includes the last output feature of all convolution layers;

[0123] a feature acquisition module, used for acquiring output feature data of the first data, wherein the output feature data corresponds to data blocks of the same spatial size in the first data;

[0124] A structure generation module, configured to generate a feature hierarchy structure based on the local data of the output feature data, wherein the feature hierarchy structure corresponds to the first output feature;

[0125] A feature path determination module is used to determine the first feature path based on the lateral connection of the feature hierarchy and the output feature data.

[0126] In an optional embodiment, the device further comprises:

[0127] A first convolution module, configured to perform a first convolution operation on the output feature data to obtain a first convolution result before determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data;

[0128] A pooling convolution module, used to perform an average pooling operation on the output feature data to obtain a first operation result, and perform a second convolution operation based on the first operation result to obtain a second convolution result;

[0129] A horizontal calculation module is used to perform horizontal calculation on the first convolution result and the second convolution result through an activation function to obtain the horizontal connection.

[0130] In an optional embodiment, the device further comprises:

[0131] an attention mask module, configured to generate a spatial attention mask based on the feature data of the local data before focusing the first feature information through the first attention path to obtain the first attention information;

[0132] An attention pyramid module, configured to determine a spatial attention pyramid based on the spatial attention mask;

[0133] an attention information determination module, configured to perform global average pooling processing on the feature data of the feature pyramid and the fully connected layer of the first backbone network to obtain channel attention information;

[0134] An attention path determination module is used to determine the first attention path based on the spatial attention pyramid and the channel attention information.

[0135] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0136] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0137] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0138] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0139] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0140] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0141] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A recognition method based on feature pyramid and attention mechanism, characterized in that: include: Acquire first data to be identified, wherein the first data includes smoke image data to be identified; Transmitting the first feature of the first data to the second feature through a first feature path to obtain first feature information; wherein the first feature path is established from top to bottom based on the first backbone network; Focusing the first feature information through a first attention path to obtain first attention information, wherein the first attention path is established from bottom to top based on the first backbone network; Determine a recognition result of the first data based on the first attention information and an average prediction result of the first backbone network on the first data; Before transmitting the first feature of the first data to the second feature through the first feature path to obtain the first feature information, the method includes: Constructing a feature pyramid through a first convolution block and a first output feature of the first backbone network, wherein the first convolution block includes a plurality of specified convolution blocks, and the first output feature includes the last output feature of all convolution layers; Acquire output feature data of the first data, wherein the output feature data corresponds to data blocks of the same spatial size in the first data; The feature pyramid generates a feature hierarchy structure based on the local data of the output feature data, wherein the feature hierarchy structure corresponds to the first output feature; determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data; Before focusing attention on the first feature information through the first attention path to obtain first attention information, the method includes: generating a spatial attention mask based on the feature data of the local data; Based on the spatial attention mask, determining a spatial attention pyramid; Performing global average pooling processing on the fully connected layer of the first backbone network and the feature data of the feature pyramid to obtain channel attention information; Based on the spatial attention pyramid and the channel attention information, determine the first attention path.

2. The method according to claim 1, characterized in that: Before determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data, the method includes: Performing a first convolution operation on the output feature data to obtain a first convolution result; Performing an average pooling operation on the output feature data to obtain a first operation result, and performing a second convolution operation based on the first operation result to obtain a second convolution result; The first convolution result and the second convolution result are horizontally calculated through an activation function to obtain the horizontal connection.

3. A recognition device based on feature pyramid and attention mechanism, characterized in that: include: A data acquisition module, used for acquiring first data to be identified, wherein the first data includes smoke image data to be identified; A feature transmission module, used for transmitting the first feature of the first data to the second feature through a first feature path to obtain first feature information; wherein the first feature path is established from top to bottom based on the first backbone network; a data attention module, configured to focus attention on the first feature information through a first attention path to obtain first attention information, wherein the first attention path is established from bottom to top based on the first backbone network; a data identification module, configured to determine an identification result of the first data based on the first attention information and an average prediction result of the first backbone network on the first data; Wherein, the device further comprises: a feature pyramid construction module, configured to construct a feature pyramid through a first convolution block and a first output feature of the first backbone network before transmitting the first feature of the first data to the second feature through the first feature path to obtain the first feature information, wherein the first convolution block includes a plurality of specified convolution blocks, and the first output feature includes the last output feature of all convolution layers; a feature acquisition module, used for acquiring output feature data of the first data, wherein the output feature data corresponds to data blocks of the same spatial size in the first data; A structure generation module, configured to generate a feature hierarchy structure based on the local data of the output feature data, wherein the feature hierarchy structure corresponds to the first output feature; a feature path determination module, configured to determine the first feature path based on the lateral connection of the feature hierarchy and the output feature data; The device also includes: an attention mask module, configured to generate a spatial attention mask based on the feature data of the local data before focusing the first feature information through the first attention path to obtain the first attention information; An attention pyramid module, configured to determine a spatial attention pyramid based on the spatial attention mask; an attention information determination module, configured to perform global average pooling processing on the feature data of the feature pyramid and the fully connected layer of the first backbone network to obtain channel attention information; An attention path determination module is used to determine the first attention path based on the spatial attention pyramid and the channel attention information.

4. The device according to claim 3, characterized in that The device also includes: A first convolution module, configured to perform a first convolution operation on the output feature data to obtain a first convolution result before determining the first feature path based on the lateral connection of the feature hierarchy and the output feature data; A pooling convolution module, used to perform an average pooling operation on the output feature data to obtain a first operation result, and perform a second convolution operation based on the first operation result to obtain a second convolution result; A horizontal calculation module is used to perform horizontal calculation on the first convolution result and the second convolution result through an activation function to obtain the horizontal connection.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 2 when executed.

6. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 2.