Feature extraction method, device, electronic device and computer readable storage medium

Abstract

The present application provides a feature extraction method, device, electronic device and computer-readable storage medium, which are applied to the field of target detection technology, wherein the method comprises: fusing a low-resolution deep feature map into a high-resolution shallow feature map through an upsampling information fusion channel, and gradually fusing the high-resolution shallow feature map into a low-resolution high-level feature map through a downsampling information fusion channel, and then further fusing the fused features of different resolutions obtained through the two channels to obtain target features, so that the target features contain features of feature maps of all resolutions, thereby improving the accuracy of target detection.

Description

Technical Field

[0001] The present application relates to the technical field of target detection, and in particular to a feature extraction method, device, electronic device and computer-readable storage medium. Background Art

[0002] Objects in images often show diversity in posture, scale, aspect ratio, etc., making it difficult to detect objects of multiple categories and different sizes. This is especially difficult in complex scenes when the image background is changeable and the target scale is relatively small.

[0003] At present, deep convolutional neural networks are usually used for target detection. For example, the commonly used model structures include yolov3, Faster R-CNN, retinanet, centernet, etc. The basic process is: an image is passed through a deep convolutional neural network to obtain feature maps of different sizes (shallow features and deep features), and these different layers of features are used to predict categories and boxes. However, the feature maps of different sizes obtained by the deep convolutional neural network contain limited information, resulting in poor accuracy in target detection and recognition. Summary of the invention

[0004] The present application provides a feature extraction method, device, electronic device and computer-readable storage medium, which are used to provide a feature extraction method so that the extracted target features contain more semantic information, thereby improving the accuracy of target detection. The technical solutions adopted in the present application are as follows:

[0005] In a first aspect, a feature extraction method is provided for a dual-channel feature fusion network, wherein the dual-channel feature fusion network includes: an upsampling information fusion channel, a downsampling information fusion channel, and a feature fusion channel; the method includes:

[0006] Feature maps of n different resolutions of the target image based on upsampling information fusion channel Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2, l≥1;

[0007] Based on the down-sampling information fusion channel, feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps

[0008] Based on the feature fusion channel, n first fusion feature maps p tdWith n second fusion feature maps Perform feature fusion processing to obtain n target feature maps

[0009] Optionally, n feature maps with different resolutions are arranged in ascending order of resolution.

[0010] Optionally, the feature maps of n different resolutions of the target image are fused based on the upsampling information channel Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps include:

[0011] Based on n feature maps of different resolutions Through the following formula:

[0012]

[0013] Determine n first fusion features in

[0014] Optionally, the feature maps of n different resolutions of the target image are fused based on the down-sampling information channel Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps include:

[0015] Based on n feature maps of different resolutions Through the following formula:

[0016]

[0017] Get n first fusion features in,

[0018] Optionally, based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps include:

[0019] Based on n first fusion feature maps p td With n second fusion feature maps Through the following formula:

[0020]

[0021] Get n target feature maps in,

[0022] Optionally, based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps include:

[0023] The n first fusion feature maps p are fused by weighted feature fusion method. td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps The first fusion feature map p td The weight value and the second fusion feature map The weight values ​​are learned through deep neural network.

[0024] Optionally, the method further comprises:

[0025] n target feature maps Input into the target detection network for target detection and recognition.

[0026] In a second aspect, a feature extraction device is provided, the device comprising:

[0027] Upsampling fusion module, used to fuse n feature maps of different resolutions of the target image based on the upsampling information channel Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2,

[0028] Downsampling fusion module, used to fuse n feature maps of different resolutions of the target image based on the downsampling information channel Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps

[0029] Fusion module, used to fusion n first fusion feature maps p based on feature fusion channel td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps

[0030] Optionally, n feature maps with different resolutions are arranged in ascending order of resolution.

[0031] Optionally, an upsampling fusion module is used to generate feature maps based on n different resolutions. Through the following formula:

[0032]

[0033] Determine n first fusion features in

[0034] Optionally, a downsampling module is used to generate feature maps based on n different resolutions. Through the following formula:

[0035]

[0036] Get n first fusion features in,

[0037] Optionally, a fusion module is specifically configured to: td With n second fusion feature maps Through the following formula:

[0038]

[0039] Get n target feature maps in,

[0040] Optionally, a fusion module is specifically used to fusion the n first fusion feature maps p by a weighted feature fusion method. td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps The first fusion feature map p td The weight value and the second fusion feature map The weight values ​​are learned through deep neural network.

[0041] Optionally, the device further comprises:

[0042] Detection and recognition module, used to transform n target feature maps Input into the target detection network for target detection and recognition.

[0043] In a third aspect, an electronic device is provided, the electronic device comprising:

[0044] one or more processors;

[0045] Memory;

[0046] One or more applications, wherein the one or more applications are stored in a memory and configured to be executed by one or more processors, and the one or more programs are configured to: execute the feature extraction method shown in the first aspect.

[0047] In a fourth aspect, a computer-readable storage medium is provided, which is used to store computer instructions. When the computer-readable storage medium is run on a computer, the computer can execute the feature extraction method shown in the first aspect.

[0048] The present application provides a feature extraction method, device, electronic device and computer-readable storage medium. Compared with the prior art that performs target detection based on shallow features or deep features extracted through convolutional neural networks, the present application uses an upsampling information fusion channel to extract n feature maps of different resolutions of the target image. Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2, Based on the down-sampling information fusion channel, feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps Based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps That is, the low-resolution deep feature map is fused into the high-resolution shallow feature map through the upsampling information fusion channel, and the high-resolution shallow feature map is gradually fused into the low-resolution high-level feature map through the downsampling information fusion channel. Then, the fused features of different resolutions obtained through the two channels are further fused to obtain the target features, so that the target features contain the features of feature maps of all resolutions, thereby improving the accuracy of target detection.

[0049] Additional aspects and advantages of the present application will be partially given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0051] Figure 1 A schematic diagram of a feature extraction method according to an embodiment of the present application;

[0052] Figure 2 A structural schematic diagram of a feature extraction device according to an embodiment of the present application;

[0053] Figure 3A schematic diagram of the DualFPN network structure of an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0055] Figure 5 Schematic diagram of the Stack-DualFPN network structure of an embodiment of the present application;

[0056] Figure 6 This is an example diagram of the target detection network structure;

[0057] Figure 7 This is another example diagram of target detection network structure;

[0058] Figure 8 Schematic diagram of FPN network structure. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0060] It will be understood by those skilled in the art that, unless otherwise stated, the singular forms "a", "an" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0062] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0063] The embodiment of the present application provides a feature extraction method for a dual-channel feature fusion network, the dual-channel feature fusion network comprising: an upsampling information fusion channel, a downsampling information fusion channel, and a feature fusion channel; Figure 1As shown, the method may include the following steps:

[0064] Step S101, based on the upsampling information fusion channel, the feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2,

[0065] Step S102: Fusing the n feature maps of different resolutions of the target image through the downsampling information fusion channel Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps

[0066] Step S103: Based on the feature fusion channel, the n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps

[0067] Among them, n feature maps with different resolutions are arranged from small to large in terms of resolution.

[0068] in, Figure 8 The schematic diagram of the FPN structure is shown in Figure 1. The basic principle of the FPN structure is to upsample the high-level features and fuse them with the shallow network so that the features of each resolution contain the detailed information of the layer and the global semantic information. Figure 3 As shown in Figure 1, a dual-channel feature fusion method is provided. On the basis of FPN, a downsampling information fusion channel is added to fuse shallow features into high-level features, thereby improving the detection accuracy. Among them, multiple DualFPNs can be cascaded to obtain a Stack-DualFPN structure, as shown in the following figure. Figure 5 shown.

[0069] Exemplarily, the process of the feature extraction method based on dual-channel fusion can be as follows:

[0070] After the input image passes through the CNN backbone network, a multi-resolution feature map is output Represents the i-th level feature in the CNN network. The resolution of this level feature is 1 / 2 of the input image resolution. i For target detection and segmentation, features after level 2 are usually used. As input feature maps, these feature maps are used as input to the feature fusion module after the 1*1 convolution network unifies the number of channels.

[0071] like Figure 3 As shown, l = 3, n = 7, Take a total of 5 layers of features as an example. The following introduces the structure of the DualFPN network. For the input features The output of DualFPN is recorded as:

[0072]

[0073] The structure consists of the following three steps:

[0074] a. Top-down upsampling feature fusion channel, such as Figure 3 As shown in the line labeled 1, this step is calculated using formula (1):

[0075]

[0076]

[0077] Conv stands for convolution module, which usually includes three sub-operations: convolution, batch normalization, and activation function. The convolution operation uses 3*3 convolution. Optionally, depthwise separable convolution can be used for acceleration. Upsample is an upsampling operation, which usually uses interpolation (such as nearest neighbor, bilinear interpolation, etc.) or deconvolution operation. The purpose is to make The resolution is doubled, The resolution is consistent.

[0078] That is, the low-resolution deep feature map is gradually integrated into the high-resolution shallow feature map.

[0079] b. Bottom-up downsampling feature fusion channel, such as Figure 3 As shown in the line labeled 2, this step is calculated using formula (2):

[0080]

[0081] Downsample is a downsampling operation, which is usually completed using pooling (max pooling, average pooling) or roialign operations. The resolution is halved, The resolution is consistent.

[0082] That is, the high-resolution shallow feature map is gradually integrated into the low-resolution high-level feature map.

[0083] c. Dual-channel feature fusion, such as Figure 3 As shown in the line numbered 3, the features of a and b are fused

[0084]

[0085] After these three steps, all resolution feature layers are fused, and each output resolution feature is a mixture of all input resolution features.

[0086] The embodiment of the present application provides a possible implementation method. For example, each fusion node in the DualFPN structure has two inputs, and the contribution weights of the two inputs to the output of the node are the same. In order to further improve the expression ability of the fusion feature, the embodiment of the present application uses a weighted feature fusion method to increase the weights of the two inputs, and the corresponding formulas (1), (2), and (3) become the following forms:

[0087]

[0088]

[0089]

[0090] Among them, ω1+ω2=1 and ω1≥0, ω2≥0. In order to ensure these two conditions, two trainable variables ω′1 and ω′2 are defined here so that

[0091] ω1=max(ω′1,0) / (max(ω′1,0)+max(ω′2,0)+ε)

[0092] ω2=max(ω′2,0) / (max(ω′1,0)+max(ω′2,0)+ε)

[0093] Two weight values ​​can be obtained based on deep neural network training.

[0094] The embodiment of the present application provides a feature extraction method. Compared with the prior art that performs target detection based on shallow features or deep features extracted by convolutional neural networks, the present application uses an upsampling information fusion channel to extract n feature maps of different resolutions of the target image. Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2, Based on the down-sampling information fusion channel, feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps Based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps That is, the low-resolution deep feature map is fused into the high-resolution shallow feature map through the upsampling information fusion channel, and the high-resolution shallow feature map is gradually fused into the low-resolution high-level feature map through the downsampling information fusion channel. Then, the fused features of different resolutions obtained through the two channels are further fused to obtain the target features, so that the target features contain the features of feature maps of all resolutions, thereby improving the accuracy of target detection.

[0095] This embodiment of the application provides a possible implementation method, and the method further includes:

[0096] n target feature maps Input into the target detection network for target detection and recognition.

[0097] Specifically, increasing the depth of the neural network can improve the performance of the model. For the feature fusion module, increasing the depth can also improve the feature representation ability, thereby improving the accuracy of detection and segmentation tasks. The number of layers and resolution are consistent with the input, so it is very convenient to overlay. Figure 5 As shown in the figure, multiple DualFPNs are connected in series to form a Stack DualFPN structure. The number of DualFPN stacking modules is selected based on the user's trade-off between timeliness and accuracy of their detection or segmentation tasks. When the number of stacking modules is 1, it is DualFPN.

[0098] Among them, based on the output of DualFPN Target detection can also be performed based on the features output by the Stack DualFPN structure, that is, the features output by the DualFPN are passed through one or more DualFPNs, and the output features are used for target detection.

[0099] For example, Figure 7 The RetinaNet target detection network based on Stack DualFPN is shown:

[0100] DualFPN can replace FPN and be applied to current detection and segmentation algorithms, which will greatly improve the accuracy. The following describes the application of DualFPN using the one-stage detection algorithm RetinaNet.

[0101] Regretting commonly used CNN backbone networks include ResNet, Mobilet, etc. Figure 7The pre-trained model structure of ResNet50 shown contains Five resolution feature maps, RetinaNet uses 3 to 5 level feature maps The number of channels is 512, 1024, and 2048 respectively. First, use 1*1 convolution to unify the number of channels to 256, and get Then use Stack DualFPN to fuse the features The figure shows that two layers of DualFPN are used. According to the details of RetinaNet, C5 is convolved with stride 2 to obtain After convolution with stride 2, we get Will Input into the target category prediction network and the target prediction network to obtain the results.

[0102] Optionally, you can As input Input to DualFPN for feature fusion, such as Figure 6 As shown, the detection effect will also be improved to a certain extent.

[0103] Figure 2 A feature extraction device provided in an embodiment of the present application includes: an upsampling fusion module 201, a downsampling fusion module 202, and a fusion module 203, wherein:

[0104] Upsampling fusion module 201, used to fuse n feature maps of different resolutions of the target image based on upsampling information channels Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2,

[0105] Downsampling fusion module 202, used to fuse n feature maps of different resolutions of the target image based on the downsampling information channel Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps

[0106] Fusion module 203, used for fusing n first fusion feature maps p based on feature fusion channels td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps

[0107] The embodiment of the present application provides a feature extraction device. Compared with the prior art that performs target detection based on shallow features or deep features extracted by convolutional neural networks, the present application uses an upsampling information fusion channel to extract n feature maps of different resolutions of the target image. Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2, Based on the down-sampling information fusion channel, feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps Based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps That is, the low-resolution deep feature map is fused into the high-resolution shallow feature map through the upsampling information fusion channel, and the high-resolution shallow feature map is gradually fused into the low-resolution high-level feature map through the downsampling information fusion channel. Then, the fused features of different resolutions obtained through the two channels are further fused to obtain the target features, so that the target features contain the features of feature maps of all resolutions, thereby improving the accuracy of target detection.

[0108] Optionally, n feature maps with different resolutions are arranged in ascending order of resolution.

[0109] Optionally, an upsampling fusion module is used to generate feature maps based on n different resolutions. Through the following formula:

[0110]

[0111] Determine n first fusion features in

[0112] Optionally, a downsampling module is used to generate feature maps based on n different resolutions. Through the following formula:

[0113]

[0114] Get n first fusion features in,

[0115] Optionally, a fusion module is specifically configured to: td With n second fusion feature maps Through the following formula:

[0116]

[0117] Get n target feature maps in,

[0118] Optionally, a fusion module is specifically used to fusion the n first fusion feature maps p by a weighted feature fusion method. td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps The first fusion feature map p td The weight value and the second fusion feature map The weight values ​​are learned through deep neural network.

[0119] Optionally, the device further comprises:

[0120] Detection and recognition module, used to transform n target feature maps Input into the target detection network for target detection and recognition.

[0121] The embodiment of the present application provides a feature extraction device, which is applicable to the method shown in the above embodiment and will not be described in detail here.

[0122] The present application embodiment provides an electronic device, such as Figure 4 As shown, Figure 4 The electronic device 40 shown includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as through a bus 402. Further, the electronic device 40 may also include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one, and the structure of the electronic device 40 does not constitute a limitation on the embodiments of the present application. The processor 401 is used in the embodiments of the present application to implement Figure 2 Functions of the modules shown. The transceiver 404 includes a receiver and a transmitter.

[0123] Processor 401 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0124] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI bus or an EISA bus, etc. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0125] The memory 403 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0126] The memory 403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement Figure 2 The illustrated embodiment provides the functionality of the feature extraction device.

[0127] The embodiment of the present application provides an electronic device. Compared with the prior art that performs target detection based on shallow features or deep features extracted by convolutional neural networks, the present application uses an upsampling information fusion channel to fusion n feature maps of different resolutions of the target image. Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2, Based on the down-sampling information fusion channel, feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps Based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps That is, the low-resolution deep feature map is fused into the high-resolution shallow feature map through the upsampling information fusion channel, and the high-resolution shallow feature map is gradually fused into the low-resolution high-level feature map through the downsampling information fusion channel. Then, the fused features of different resolutions obtained through the two channels are further fused to obtain the target features, so that the target features contain the features of feature maps of all resolutions, thereby improving the accuracy of target detection.

[0128] The present application embodiment provides an electronic device applicable to the above method embodiment, which will not be described in detail here.

[0129] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method shown in the above embodiment is implemented.

[0130] The embodiment of the present application provides a computer-readable storage medium. Compared with the prior art that performs target detection based on shallow features or deep features extracted by convolutional neural networks, the present application uses an upsampling information fusion channel to perform target detection on n feature maps of different resolutions of the target image. Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain n first fused feature maps Where n≥2, Based on the down-sampling information fusion channel, feature maps of n different resolutions of the target image are Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain n second fused feature maps Based on the feature fusion channel, n first fusion feature maps p td With n second fusion feature maps Perform feature fusion processing to obtain n target feature maps That is, the low-resolution deep feature map is fused into the high-resolution shallow feature map through the upsampling information fusion channel, and the high-resolution shallow feature map is gradually fused into the low-resolution high-level feature map through the downsampling information fusion channel. Then, the fused features of different resolutions obtained through the two channels are further fused to obtain the target features, so that the target features contain the features of feature maps of all resolutions, thereby improving the accuracy of target detection.

[0131] The present application embodiment provides a computer-readable storage medium applicable to the above method embodiment, which will not be described in detail here.

[0132] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0133] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A feature extraction method, characterized in that: Used in a dual-channel feature fusion network, the dual-channel feature fusion network includes: an upsampling information fusion channel, a downsampling information fusion channel, and a feature fusion channel; including: Feature maps of different resolutions of the target image based on upsampling information fusion channels Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain the first fused feature map in l≥1, n≥2, including: Feature maps based on different resolutions Through the following formula: Determine the first fusion feature map in Feature maps of different resolutions of the target image based on down-sampling information fusion channels Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain a second fused feature map include: Feature maps based on different resolutions Through the following formula: Get the second fusion feature map in, Based on the feature fusion channel, the first fusion feature map With the second fusion feature map Perform feature fusion processing to obtain the target feature map The method further comprises: Multiple dual-channel feature fusion networks are connected in series to obtain the Stack DualFPN structure; The Stack DualFPN structure is combined with the RetinaNet one-stage detection algorithm, and the CNN backbone network of RetinaNet is used to extract feature maps of different resolutions of the target image. The number of channels of the feature maps of different resolutions is unified through convolution, and the feature maps with the unified number of channels are input into the Stack DualFPN structure to obtain the target feature map. The target feature map is input into the target category prediction network and the target prediction network of RetinaNet to obtain the result.

2. The method according to claim 1, characterized in that The feature maps with different resolutions are arranged from small to large in terms of resolution.

3. The method according to claim 1, characterized in that Based on the feature fusion channel, the first fusion feature map With the second fusion feature map Perform feature fusion processing to obtain the target feature map include: Based on the first fusion feature map With the second fusion feature map Through the following formula: Get the target feature map in, 4. The method according to claim 1, characterized in that: Based on the feature fusion channel, the first fusion feature map With the second fusion feature map Perform feature fusion processing to obtain the target feature map include: The first fusion feature map is fused by a weighted feature fusion method. With the second fusion feature map Perform feature fusion processing to obtain the target feature map The first fusion feature map The weight value and the second fusion feature map The weight values ​​are learned through deep neural network.

5. The method according to any one of claims 1 to 4, characterized in that: The method further includes: The target feature map Input into the target detection network for target detection and recognition.

6. A feature extraction device, characterized in that: include: Upsampling fusion module, used to fuse the feature maps of different resolutions of the target image based on the upsampling information Perform feature fusion processing to fuse the features of the low-resolution feature map into the features of the high-resolution feature map to obtain the first fused feature map in l≥1, n≥2, including: Feature maps based on different resolutions Through the following formula: Determine the first fusion feature map in Downsampling fusion module, used to fuse the feature maps of different resolutions of the target image based on the downsampling information Perform feature fusion processing to fuse the features of the high-resolution feature map into the features of the low-resolution feature map to obtain a second fused feature map include: Feature maps based on different resolutions Through the following formula: Get the second fusion feature map in, A fusion module is used to combine the first fusion feature map based on the feature fusion channel With the second fusion feature map Perform feature fusion processing to obtain the target feature map The device also includes: Multiple dual-channel feature fusion networks are connected in series to obtain the Stack DualFPN structure; The Stack DualFPN structure is combined with the RetinaNet one-stage detection algorithm, and the CNN backbone network of RetinaNet is used to extract feature maps of different resolutions of the target image. The number of channels of the feature maps of different resolutions is unified through convolution, and the feature maps with the unified number of channels are input into the Stack DualFPN structure to obtain the target feature map. The target feature map is input into the target category prediction network and the target prediction network of RetinaNet to obtain the result.

7. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the feature extraction method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer instructions, and when the computer-readable storage medium is run on a computer, the computer can execute the feature extraction method described in any one of claims 1 to 5.

Images