Feature map processing method and apparatus, non-transitory storage medium, and electronic device

Abstract

The application discloses a feature map processing method and device, a nonvolatile storage medium and an electronic device. The method comprises the following steps: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result, and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map. The application solves the technical problem that, in the related art, when a feature map is processed, it is difficult to meet both a higher average accuracy and a lower calculation cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of feature map processing, in particular to a feature map processing method and device, a nonvolatile storage medium and an electronic device. BACKGROUND

[0002] In the related art, the detection algorithm in the visual intelligent open platform needs to meet the detection needs of customers at various consumption levels, so when processing the feature map, it is required to have a high average accuracy rate and not to increase or reduce the calculation amount, so as to reduce the cost of algorithm calling.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The embodiments of the present application provide a feature map processing method and device, a nonvolatile storage medium and an electronic device, to at least solve the technical problem in the related art that when processing the feature map, it is difficult to meet both a high average accuracy rate and a low calculation cost.

[0005] According to an aspect of the embodiments of the present application, a feature map processing method is provided, including: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0006] According to another aspect of the embodiments of the present application, a feature map processing method is also provided, including: obtaining an original feature map; performing coding processing on the original feature map by using a first network model to obtain a first coding result and performing coding processing on the original feature map by using a second network model to obtain a second coding result, wherein the first network model adopts a bottleneck network structure in the feature dimension, and the second network model adopts a bottleneck network structure in the feature space; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0007] According to another aspect of the embodiments of the present application, a feature map processing method is also provided, including: receiving a current input original feature map; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is obtained by the server through feature space encoding processing on the original feature map.

[0008] According to another aspect of the embodiments of the present application, a feature map processing method is also provided, including: receiving a current input original feature map; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is obtained by the server through feature space encoding processing on the original feature map.

[0009] According to another aspect of the embodiments of the present application, a feature map processing method is also provided, including: receiving a current input original feature map; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is obtained by the server through feature space encoding processing on the original feature map.

[0010] According to another aspect of the embodiments of the present application, a feature map processing method is also provided, including: receiving a current input original feature map; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is obtained by the server through feature space encoding processing on the original feature map.

[0011] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, including a stored program, wherein when the program runs, the non-volatile storage medium controls a device where the non-volatile storage medium is located to execute any one of the feature map processing methods.

[0012] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising: a processor; and a memory connected with the processor, configured to provide the processor with instructions to process the following processing steps: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0013] In the embodiments of the present application, the original feature map is obtained, feature dimension coding processing is performed on the original feature map to obtain a first coding result, feature space coding processing is performed on the original feature map to obtain a second coding result, and feature fusion processing is performed on the first coding result and the second coding result to obtain a target feature map.

[0014] It is easy to note that, in the embodiments of the present application, the parallel feature coding mode of the bottleneck network structure in the feature dimension and the bottleneck network structure in the feature space is adopted to perform feature dimension coding processing and feature space coding processing on the original feature map, that is, the embodiments of the present application simultaneously perform coding processing in the low-dimensional and low-resolution feature space and the feature dimension, and the low dimension and low resolution mean that less calculation amount is used; the receptive field of the convolutional neural network model can be expanded in the low resolution, and the expression ability of the detection model is enhanced, so that the average accuracy is higher.

[0015] Therefore, the embodiments of the present application achieve the purpose of considering the higher average processing accuracy and the lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers of various consumption levels, and further solving the technical problem that the higher average accuracy and the lower calculation cost cannot be considered when processing the feature map in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0017] Figure 1 Fig. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing the feature map processing method;

[0018] Figure 2 Fig. 2 is a flowchart of a feature map processing method according to the embodiments of the present application;

[0019] Figure 3 Fig. 3 is a scene schematic diagram of a feature map processing method according to the embodiments of the present application;

[0020] Figure 4 is a scene schematic diagram of a feature map processing method according to an embodiment of the application;

[0021] Figure 5 is a flowchart of another feature map processing method according to an embodiment of the application;

[0022] Figure 6 is a flowchart of another feature map processing method according to an embodiment of the application;

[0023] Figure 7 is a flowchart of another feature map processing method according to an embodiment of the application;

[0024] Figure 8 is a flowchart of another feature map processing method according to an embodiment of the application;

[0025] Figure 9 is a structural schematic diagram of a feature map processing device according to an embodiment of the application;

[0026] Figure 10 is a structural schematic diagram of an electronic device according to an embodiment of the application;

[0027] Figure 11 is a structural block diagram of another computer terminal according to an embodiment of the application. DETAILED DESCRIPTION

[0028] In order to make the personnel in the art better understand the scheme of the application, the technical scheme in the embodiments of the application will be clearly and completely described below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the application.

[0029] It should be noted that the terms "first", "second", and the like in the specification and claims of the application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some of the nouns or terms that appear in the description of the embodiments of the present application are applicable to the following explanations:

[0031] Bottleneck network structure with double-link heterogeneous: dimension reduction through a convolutional layer with a kernel size of 1x1, followed by a 3x3 convolutional layer, and finally dimension increase by a 1x1 convolutional layer. Since the feature dimension of the intermediate layer looks thinner than the input and output feature dimensions, it is like a bottleneck, so it is also called a bottleneck network structure (Bottleneck). One of the great benefits of using a Bottleneck structure model is that it can greatly reduce the amount of calculation.

[0032] Bottleneck in feature dimension: traditional Bottleneck structure.

[0033] Bottleneck in feature space: down-sampling the feature space to a low-resolution feature space through a 3x3 convolutional layer with a stride of 2, then encoding features in the low-resolution feature space through a 3x3 convolutional layer with a stride of 1, and finally mapping the low-resolution feature space to a high-resolution feature through an up-sampling layer. Since the feature space is down-sampled and up-sampled, the intermediate feature layer looks lower in resolution than the input and output feature resolutions, like a bottleneck structure, so it is called a Bottleneck in feature space.

[0034] Bottleneck in feature space with N-level resolution: down-sampling the feature space to a low-resolution feature space through N 3x3 convolutional layers with a stride of 2, then encoding features in the low-resolution feature space through a 3x3 convolutional layer with a stride of 1, and finally mapping the low-resolution feature space to a high-resolution feature through an up-sampling layer.

[0035] One-level resolution heterogeneous Bottleneck sub-module structure, i.e., double-link heterogeneous Bottleneck sub-module structure, refers to the module structure that combines the Bottleneck in feature dimension and the Bottleneck in feature space together to encode the input features in parallel, which is called a double-link heterogeneous Bottleneck sub-module structure.

[0036] Two-level resolution heterogeneous Bottleneck sub-module structure: refers to further reducing the resolution by 2 times on the basis of the one-level resolution heterogeneous Bottleneck sub-module structure to enhance the expression ability of the traditional convolutional model in a larger receptive field, which is called a two-level resolution heterogeneous Bottleneck sub-module structure.

[0037] N-level resolution heterogeneous bottleneck sub-module structure: refers to further reducing the resolution by 2 times on the basis of N-1 level resolution, and enhancing the expression ability of the traditional convolution model under a larger receptive field.

[0038] Receptive field of the convolution layer: intuitively, the receptive field is the size of the visual receptive area. In the convolutional neural network, the definition of the receptive field is the size of the region mapped on the original image by the pixel points on the feature map output by each layer of the convolutional neural network.

[0039] Feature map: the features corresponding to the image obtained by mapping in the convolutional neural network are called feature maps.

[0040] Target detection model: an artificial intelligence model that integrates object recognition and object positioning. The output is the category of the object and the position information of the object.

[0041] Embodiment 1

[0042] According to the embodiments of the present application, an embodiment of a feature map processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] The method embodiment provided in Embodiment 1 of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the feature map processing method is shown. As shown in Figure 1 The computer terminal 10 (or mobile device 10) can include one or more processors 102 (the processor 102 can include but is not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in Figure 1 or have different structures from Figure 1different configurations.

[0044] It should be noted that the one or more processors 102 and / or other data processing circuitry described above can be generally referred to herein as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any of the other elements of the computer terminal 10 (or mobile device). As referred to in embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0045] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the feature map processing method in embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the feature map processing method described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

[0047] The display can be, for example, a touch screen type liquid crystal display (LCD), which can enable a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0048] In the above operating environment, the present application provides a feature map processing method as shown in Figure 2 Figure 2 is a flowchart of a feature map processing method according to embodiments of the present application, as shown in Figure 2 the feature map processing method includes the following method steps:​

[0049] In step S202, an original feature map is acquired.

[0050] In step S204, the original feature map is subjected to feature dimension coding processing to obtain a first coding result, and the original feature map is subjected to feature space coding processing to obtain a second coding result.

[0051] In step S206, the first coding result and the second coding result are subjected to feature fusion processing to obtain a target feature map.

[0052] In the embodiments of the present application, the original feature map is acquired, the original feature map is subjected to feature dimension coding processing to obtain a first coding result, and the original feature map is subjected to feature space coding processing to obtain a second coding result, and the first coding result and the second coding result are subjected to feature fusion processing to obtain a target feature map.

[0053] It is easy to note that the embodiments of the present application adopt a parallel feature coding manner of bottleneck network structure in feature dimension and bottleneck network structure in feature space, and the original feature map is subjected to feature dimension coding processing and feature space coding processing, that is, the embodiments of the present application simultaneously perform coding processing in a low-dimensional, low-resolution feature space and feature dimension, and the low-dimensional, low-resolution means that less calculation amount is used; the low resolution can expand the receptive field of the convolutional neural network model and enhance the expression ability of the detection model, and therefore has a higher average accuracy.

[0054] Therefore, the embodiments of the present application achieve the purpose of considering both higher average processing accuracy and lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers at various consumption levels, and further solving the technical problem that in the related art, when processing the feature map, both higher average accuracy and lower calculation cost cannot be considered.

[0055] It should be noted that the feature map processing method can be applied to a visual intelligent open platform, for example, applied to a detection algorithm or a network model opened to the outside in the visual intelligent open platform.

[0056] In an optional embodiment, the feature map processing method is applied to at least one of the following network models: a target detection model, a target recognition model, and an image segmentation model.

[0057] The double-link heterogeneous bottleneck network structure provided by the embodiments of the present application can be applied not only to the target detection model, but also to the target recognition model, the image segmentation model, and other types of convolutional neural networks, and has wide applicability.

[0058] In an optional embodiment, the original feature map is processed by parallel encoding in feature dimension to obtain the first encoding result and by parallel encoding in feature space to obtain the second encoding result.

[0059] As an optional embodiment, the application provides a dual-link heterogeneous BottleNeck structure. The dual-link heterogeneous BottleNeck structure encodes input features in parallel by a BottleNeck in feature dimension and a BottleNeck in feature space to form a new feature space. The BottleNeck in feature dimension maintains the expression ability of a traditional detection model, and the BottleNeck in feature space enhances the expression ability of the traditional model under a larger receptive field, thereby improving the average accuracy of the detection model. Furthermore, based on the dual-link heterogeneous BottleNeck structure, the expression ability of the target detection model is enhanced and the detection index of the model is improved under the same amount of calculation.

[0060] In an optional embodiment, the original feature map is processed by parallel encoding in feature dimension to obtain the first encoding result, including:

[0061] In step S302, the feature dimension of the original feature map is reduced from a first dimension to a second dimension by a first convolution layer to obtain a dimension reduction result, wherein the first convolution layer uses a first convolution kernel and a first span.

[0062] In step S304, the dimension reduction result is encoded by a second convolution layer to obtain the first encoding result, wherein the second convolution layer uses a second convolution kernel and the first span.

[0063] Optionally, the dual-link heterogeneous BottleNeck structure provided by the application uses a BottleNeck structure in feature dimension of link 1 and a BottleNeck structure in feature space of link 2 to encode features in parallel, as shown in FIG. 1. Figure 3 As shown in FIG. 1, an original feature map with a size of CxHxW is encoded by the BottleNeck structure of two links in parallel, wherein C is the feature dimension of the original feature map, and HxW is the height and length of the original feature map.

[0064] The bottleneck network structure in the feature dimension in the bottleneck network structure of link 1 encodes the original feature map. First, the input feature is reduced from the feature dimension of C (i.e., the first dimension) to the feature dimension of 0.5C (i.e., the second dimension) by a first convolutional layer with a stride of 1 (i.e., the first stride) and a span of 1x1 (i.e., the first convolutional kernel), and then the feature is encoded by a second convolutional layer with a stride of 1 (i.e., the first stride) and a span of 3x3 (i.e., the second convolutional kernel) on the low-dimensional feature, to obtain the first encoding result.

[0065] In an alternative embodiment, the feature space encoding processing of the original feature map to obtain the second encoding result includes:

[0066] In step S402, the feature resolution of the original feature map is mapped from the first resolution to the second resolution by a third convolutional layer to obtain a mapping processing result, wherein the third convolutional layer uses the second convolutional kernel and the second stride.

[0067] In step S404, the mapping processing result is encoded by a fourth convolutional layer to obtain a third encoding result, wherein the fourth convolutional layer uses the second convolutional kernel and the first stride.

[0068] In step S406, the third encoding result is up-sampled by an up-sampling layer to obtain the second encoding result.

[0069] In the alternative embodiment, the bottleneck network structure BottleNeck in the feature space of link 2 encodes the original feature map. The input feature is mapped from the feature resolution of HxW to a low-resolution feature (i.e., the third encoding result) of 0.5Hx0.5W by a third convolutional layer with a stride of 2 (i.e., the second stride) and a span of 3x3 (i.e., the second convolutional kernel), and then the feature is encoded by a fourth convolutional layer with a stride of 1 (i.e., the first stride) and a span of 3x3 (i.e., the second convolutional kernel) on the low-resolution feature space, and finally the low-resolution feature is mapped to a high-resolution feature (i.e., the second encoding result) by an up-sampling layer.

[0070] In an alternative embodiment, the feature space encoding processing of the original feature map to obtain the second encoding result includes:

[0071] In step S412, the original feature map is processed by a first-level feature space encoding to obtain a first intermediate result.

[0072] In step S414, the original feature map is processed by a second-level feature space encoding to obtain a second intermediate result.

[0073] Step S416: The first intermediate result and the second intermediate result are spliced ​​together to obtain the third intermediate result;

[0074] Step S418: Upsample the third intermediate result using an upsampling layer to obtain the second encoding result.

[0075] In the embodiments of this application, when the convolutional neural network has a large convolutional receptive field, it can acquire information over a larger area, and thus the corresponding expressive power of the convolutional neural network will be stronger. For example, BottleNeck on the feature space of N-level resolution encodes N resolution features on N low-resolution feature spaces. Each resolution feature has a large convolutional field of view, which can effectively enhance the expressive power of the detection model. Moreover, encoding on low-resolution features also has the advantage of less computation.

[0076] like Figure 4 As shown, using Link 1, the original feature map is processed by feature space encoding through a Bottleneck in the feature dimension. First, the original feature map is passed through a 1x1 fifth convolutional layer with a stride of 1 to reduce the feature dimension from C to 0.5C. Then, a 3x3 sixth convolutional layer with a stride of 1 is used for the first level of feature space encoding on the low-dimensional feature, resulting in the first intermediate result. Using Link 2, a Bottleneck structure composed of 2N layers of feature space resolution is used. After feature space encoding through a Bottleneck in the feature space, the original feature map is passed through a 3x3 seventh convolutional layer with a stride of 2 to map the feature from HxW resolution to a low-resolution feature of 0.5Hx0.5W. Then, a 3x3 eighth convolutional layer with a stride of 1 is used for the second level of feature space encoding on the low-resolution feature space, resulting in the second intermediate result.

[0077] In this embodiment, the features of link 1 and link 2 can be fused: the features encoded by link 1 and link 2 are concatenated together through a concatenation layer, and then fused through a 1x1 convolutional layer. Finally, an output feature map of size CxHxW is obtained.

[0078] For example, feature fusion using modules with k-level resolution and modules with k-1-level resolution: The features encoded in the feature space of k-level resolution and the features encoded in the feature space of k+1-level resolution are concatenated together through a concatenation layer, and then information is fused through a 1x1 convolutional layer. Finally, an output feature map with the same size as the input feature map of the sub-module with k-level resolution is obtained.

[0079] In an alternative embodiment, the first-level feature space encoding processing is performed on the original feature map to obtain the first intermediate result, including:

[0080] In step S420, a fifth convolutional layer is used to map the feature resolution of the original feature map from the first resolution to a second resolution to obtain a first mapping processing result, wherein the fifth convolutional layer uses the second convolutional kernel and the second stride;

[0081] In step S422, a sixth convolutional layer is used to perform feature encoding on the mapping processing result to obtain the first intermediate result, wherein the sixth convolutional layer uses the second convolutional kernel and the first stride.

[0082] In the embodiment of the present application, link 1 is used to perform feature space encoding processing on the original feature map through BottleNeck in the feature dimension. First, the original feature map is reduced from C dimensions to 0.5C dimensions through a fifth convolutional layer with a 1x1 stride of 1, and then the low-dimensional feature is processed through a sixth convolutional layer with a 3x3 stride of 1 to obtain the first intermediate result.

[0083] In an alternative embodiment, the second-level feature space encoding processing is performed on the original feature map to obtain the second intermediate result, including:

[0084] In step S430, a seventh convolutional layer is used to map the feature resolution of the original feature map from the second resolution to a third resolution to obtain a second mapping processing result, wherein the seventh convolutional layer uses the second convolutional kernel and the second stride;

[0085] In step S432, an eighth convolutional layer is used to perform feature encoding on the mapping processing result to obtain a fourth intermediate result, wherein the eighth convolutional layer uses the second convolutional kernel and the first stride;

[0086] In step S434, the fourth intermediate result is up-sampled by the up-sampling layer to obtain the second intermediate result.

[0087] Through the embodiment of the present application, link 2 is used to form a BottleNeck structure in the feature space of 2N levels of resolution, and the original feature map is processed through BottleNeck in the feature space of one level to perform feature space encoding processing. Then, the original feature map is mapped from HxW resolution to 0.5Hx0.5W low-resolution feature through a seventh convolutional layer with a 3x3 stride of 2, and then the low-resolution feature space is processed through an eighth convolutional layer with a 3x3 stride of 1 to perform second-level feature space encoding processing to obtain the second intermediate result.

[0088] After feature encoding via a two-level Bottleneck in the feature space, the input features are mapped from a resolution of 0.5Hx0.5W to a low-resolution feature of 0.25Hx0.25W by a 3x3 convolutional layer with a stride of 2. Then, feature encoding is performed in the low-resolution feature space by a 3x3 convolutional layer with a stride of 1. Similarly, after feature encoding via a two-level Bottleneck in the feature space, the input features are further reduced by a factor of 2 by a 3x3 convolutional layer with a stride of 2. Then, feature encoding is performed in the low-resolution feature space by a 3x3 convolutional layer with a stride of 1.

[0089] In such Figure 4 In the scenario shown, the two-layer resolution heterogeneous BottleNeck has a rank of 2, and the feature map resolution is downsampled to 2. 2 =4 times.

[0090] In an optional embodiment, feature fusion processing is performed on the first encoding result and the second encoding result to obtain the target feature map, including:

[0091] Step S502: The first encoding result and the second encoding result are concatenated in the feature dimension using the concatenation layer to obtain the concatenation result.

[0092] Step S504: The above splicing results are added together in the feature dimension using the feature addition layer to obtain the target feature map.

[0093] In this embodiment of the application, the first encoding result and the second encoding result obtained by encoding link 1 and link 2 respectively are concatenated by a concatenation layer (Cocat) to obtain a concatenated result. Then, information is fused by a 1x1 convolutional layer, that is, the concatenation result is added in the feature dimension by a feature addition layer, and finally a target feature map of size CxHxW is obtained.

[0094] The parallel feature coding mode of the BottleNeck in the feature dimension of link 1 and the BottleNeck in the feature space of link 2 has the following advantages: the bottleneck mode can encode features in a low-dimensional and low-resolution feature space at the same time, and the low dimension and low resolution mean less calculation, so the calculation amount can be kept at the same level as that of yolov5; the use of two bottleneck modes can enhance the expression ability of the model, on the one hand, the structure of link 1 is similar to the bottleneck module structure of yolov5, so the model expression ability similar to yolov5 can be retained; and on the other hand, the structure of link 2 is added, which can expand the receptive field of the convolutional neural network model in low resolution and enhance the expression ability of the detection model, so the model of the present application has a higher average accuracy than yolov5.

[0095] In the embodiment of the present application, by adopting a new double-link heterogeneous bottleneck network structure, the BottleNeck input features in the feature space are mapped from HxW resolution to 0.5Hx0.5W low-resolution features through a 3x3 convolution layer with a stride of 2, and then the features are encoded in the low-resolution feature space through a 3x3 convolution layer with a stride of 1, and finally the low-resolution features are mapped to high-resolution features through an up-sampling layer.

[0096] The double-link heterogeneous bottleneck network structure in the embodiment of the present application encodes image features by combining the BottleNeck in the feature dimension and the BottleNeck in the feature space, the BottleNeck in the feature dimension retains the similar expression ability of the model as the traditional detection model, and the BottleNeck in the feature space effectively enhances the expression ability of the detection model by expanding the receptive field of the convolutional neural network model.

[0097] According to the embodiment of the present application, based on the sub-module of the double-link heterogeneous bottleneck network structure (i.e. the BottleNeck structure), the average accuracy of the detection model can be higher than that of the detection model using only the BottleNeck structure in the feature dimension under the same calculation amount, which has a higher performance-price ratio for consumer customers and can better meet the detection needs of consumer customers.

[0098] As an optional embodiment, in terms of software open source, the embodiment of the present application can but is not limited to modifying the detection model structure on the yolov5 open source code.

[0099] According to the embodiment of the present application, another feature map processing method is provided as shown in Figure 5 Figure 5 ​is a flowchart of another feature map processing method according to an embodiment of the present application, as shown in Figure 5 The feature map processing method comprises the following steps.

[0100] In step S602, an original feature map is obtained.

[0101] In step S604, the original feature map is encoded by using a first network model to obtain a first encoding result, and the original feature map is encoded by using a second network model to obtain a second encoding result, wherein the first network model adopts a bottleneck network structure in feature dimension, and the second network model adopts a bottleneck network structure in feature space.

[0102] In step S606, the first encoding result and the second encoding result are fused to obtain a target feature map.

[0103] In the embodiment of the present application, the original feature map is obtained, the original feature map is encoded by using a first network model to obtain a first encoding result, and the original feature map is encoded by using a second network model to obtain a second encoding result, wherein the first network model adopts a bottleneck network structure in feature dimension, and the second network model adopts a bottleneck network structure in feature space; the first encoding result and the second encoding result are fused to obtain a target feature map.

[0104] It is easy to note that the bottleneck network structure in feature dimension and the bottleneck network structure in feature space are used in parallel feature encoding mode in the embodiment of the present application, and the original feature map is encoded in feature dimension and feature space, that is, the embodiment of the present application encodes in low dimension and low resolution feature space and feature dimension at the same time, which means that less calculation is used; the receptive field of the convolutional neural network model can be expanded in low resolution, and the expression ability of the detection model is enhanced, so that the average accuracy is higher.

[0105] Therefore, the embodiment of the present application achieves the purpose of considering higher average processing accuracy and lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers at various consumption levels, and further solving the technical problem that higher average accuracy and lower calculation cost cannot be considered when processing the feature map in the related art.

[0106] According to the embodiment of the present application, another feature map processing method is provided, as shown in Figure 6 The feature map processing method comprises the following steps. Figure 6 is a flowchart of another feature map processing method according to an embodiment of the present application, as shown in Figure 6 The feature map processing method comprises the following steps.

[0107] Step S702, receiving a current input original feature map;

[0108] Step S704, sending the original feature map to a server;

[0109] Step S706, receiving a target feature map from the server, wherein the target feature map is obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is obtained by the server through feature space encoding processing on the original feature map.

[0110] In the embodiment of the application, the original feature map input by the client is received, the original feature map is sent to the server, and the target feature map from the server is received, wherein the target feature map is obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is obtained by the server through feature space encoding processing on the original feature map.

[0111] It is easy to note that the bottleneck network structure in the feature dimension and the bottleneck network structure in the feature space are used in parallel feature encoding manner in the embodiment of the application, the original feature map is processed through feature dimension encoding and feature space encoding, that is, the embodiment of the application encodes in the low-dimensional and low-resolution feature space and feature dimension at the same time, which means less calculation amount; the receptive field of the convolutional neural network model can be expanded in low resolution, and the expression ability of the detection model is enhanced, so that the average accuracy is higher.

[0112] Therefore, the embodiment of the application achieves the purpose of considering higher average processing accuracy and lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers at various consumption levels, and further solving the technical problem that higher average accuracy and lower calculation cost cannot be considered when processing the feature map in the related art.

[0113] According to the embodiment of the application, another feature map processing method is provided as shown in Figure 7 . Figure 7 is a flowchart of another feature map processing method according to the embodiment of the application, as shown in Figure 7 , the feature map processing method comprises:

[0114] Step S802, receiving the original feature map from the client;

[0115] Step S804, performing feature dimension coding processing on the original feature map to obtain a first coding result, and performing feature space coding processing on the original feature map to obtain a second coding result;

[0116] Step S806, performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map;

[0117] Step S808, returning the target feature map to the client.

[0118] In the embodiment of the present application, the original feature map is received from the client by the server; the original feature map is processed by feature dimension coding to obtain a first coding result, and the original feature map is processed by feature space coding to obtain a second coding result; the first coding result and the second coding result are processed by feature fusion to obtain a target feature map; and the target feature map is returned to the client.

[0119] It is easy to note that the bottleneck network structure in the feature dimension and the bottleneck network structure in the feature space are used in parallel feature coding manner in the embodiment of the present application, and the original feature map is processed by feature dimension coding and feature space coding, that is, the embodiment of the present application encodes in low dimension and low resolution feature space and feature dimension at the same time, which means that less calculation is used; the receptive field of the convolutional neural network model can be expanded in low resolution, and the expression ability of the detection model is enhanced, so that the average accuracy is higher.

[0120] Therefore, the embodiment of the present application achieves the purpose of considering higher average processing accuracy and lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers of various consumption levels, and further solving the technical problem that higher average accuracy and lower calculation cost cannot be considered when processing the feature map in the related art.

[0121] According to the embodiment of the present application, another feature map processing method is provided as shown in Figure 8 . Figure 8 is a flowchart of another feature map processing method according to the embodiment of the present application, as shown in Figure 8 , the feature map processing method comprises:

[0122] Step S902, obtaining an original feature map corresponding to an original input image;

[0123] In step S904, the original feature map is subjected to feature dimension coding processing to obtain a first coding result, and is subjected to feature space coding processing to obtain a second coding result.

[0124] In step S906, the first coding result and the second coding result are subjected to feature fusion processing to obtain a target feature map.

[0125] In step S908, the category information and the position information of the target object in the original input image are detected based on the target feature map.

[0126] In the embodiments of the present application, in the actual application scenario, the original feature map corresponding to the original input image is obtained, the original feature map is subjected to feature dimension coding processing to obtain a first coding result, and is subjected to feature space coding processing to obtain a second coding result, the first coding result and the second coding result are subjected to feature fusion processing to obtain a target feature map, and the category information and the position information of the target object in the original input image are detected based on the target feature map.

[0127] It is easy to note that the embodiments of the present application adopt a parallel feature coding mode of bottleneck network structure in feature dimension and bottleneck network structure in feature space, the original feature map is subjected to feature dimension coding processing and feature space coding processing to obtain a target feature map, and the category information and the position information of the target object in the original input image are detected based on the target feature map. The embodiments of the present application simultaneously perform coding processing in a low-dimensional, low-resolution feature space and feature dimension, and the low dimension and low resolution mean that less calculation is used; in the low resolution, the receptive field of the convolutional neural network model can be expanded, and the expression ability of the detection model is enhanced, so that a higher average accuracy is achieved.

[0128] Therefore, the embodiments of the present application achieve the purpose of considering higher average processing accuracy and lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers at various consumption levels, and further solving the technical problem that higher average accuracy and lower calculation cost cannot be considered when processing the feature map in the related art.

[0129] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action order described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a general hardware platform, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a non-volatile storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method described in the above embodiments.

[0131] Embodiment 2

[0132] According to the embodiments of the present application, an embodiment of a device for implementing the above feature map processing method is also provided, Figure 9 is a structural schematic diagram of a feature map processing device according to an embodiment of the present application, as Figure 9 shown, the device comprises an acquisition module 700, an encoding processing module 702, and a fusion processing module 704, wherein:

[0133] The acquisition module 700 is configured to acquire an original feature map; the encoding processing module 702 is configured to perform feature dimension encoding processing on the original feature map to obtain a first encoding result, and perform feature space encoding processing on the original feature map to obtain a second encoding result; and the fusion processing module 704 is configured to perform feature fusion processing on the first encoding result and the second encoding result to obtain a target feature map.

[0134] It should be noted that the acquisition module 700, the encoding processing module 702, and the fusion processing module 704 correspond to steps S202 to S206 in Embodiment 1, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0135] It should be noted that the preferred embodiments of the present embodiment can refer to the related description in Embodiment 1, which will not be repeated here.

[0136] Embodiment 3

[0137] According to the embodiments of the present application, an embodiment of an electronic device is also provided, which can be any one of the computing devices in the computing device group. Figure 10 is a structural schematic diagram of an electronic device according to an embodiment of the present application, as Figure 10As shown, the electronic device comprises a processor 800 and a memory 802, wherein:

[0138] The processor 800; and the memory 802, connected with the processor 800, is used to provide the processor with instructions to process the following processing steps: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0139] In the embodiment of the present application, the original feature map is obtained; the original feature map is processed by feature dimension coding to obtain a first coding result, and the original feature map is processed by feature space coding to obtain a second coding result; and the first coding result and the second coding result are processed by feature fusion to obtain a target feature map.

[0140] It is easy to note that the bottleneck network structure in the feature dimension and the bottleneck network structure in the feature space are used in parallel feature coding mode in the embodiment of the present application, and the original feature map is processed by feature dimension coding and feature space coding, that is, the embodiment of the present application encodes in low dimension and low resolution feature space and feature dimension at the same time, which means that less calculation is used; the receptive field of the convolutional neural network model can be expanded in low resolution, and the expression ability of the detection model is enhanced, so that the average accuracy is higher.

[0141] Therefore, the embodiment of the present application achieves the purpose of considering higher average processing accuracy and lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers at various consumption levels, and further solving the technical problem that higher average accuracy and lower calculation cost cannot be considered when processing the feature map in the related art.

[0142] It should be noted that the preferred embodiments of the present embodiment can refer to the related description in Embodiment 1, which will not be repeated here.

[0143] Embodiment 4

[0144] According to the embodiments of the present application, an embodiment of a computer terminal is also provided, which can be any one of the computer terminal devices in the computer terminal group. Optionally, in the present embodiment, the computer terminal can be replaced by a terminal device such as a mobile terminal.

[0145] Optionally, in the present embodiment, the computer terminal can be located in at least one network device of the plurality of network devices of the computer network.

[0146] In the embodiment, the computer terminal can execute program codes of the following steps in the feature map processing method: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0147] Optionally, Figure 11 is another structural block diagram of a computer terminal according to an embodiment of the present application, as Figure 11 indicated, the computer terminal can include one or more (only one is shown in the figure) processors 902, a memory 904, and a peripheral interface 906.

[0148] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the feature map processing method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the feature map processing method described above. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0149] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0150] Optionally, the processor can further execute program codes of the following steps: reducing the feature dimension of the original feature map from a first dimension to a second dimension by using a first convolutional layer to obtain a dimension reduction processing result, wherein the first convolutional layer uses a first convolutional kernel and a first span; and performing feature coding on the dimension reduction processing result by using a second convolutional layer to obtain the first coding result, wherein the second convolutional layer uses a second convolutional kernel and the first span.

[0151] Optionally, the processor can further execute program codes of the following steps: mapping a feature resolution of the original feature map from a first resolution to a second resolution by using a third convolutional layer to obtain a mapping processing result, wherein the third convolutional layer uses a second convolutional kernel and a second stride; performing feature encoding on the mapping processing result by using a fourth convolutional layer to obtain a third encoding result, wherein the fourth convolutional layer uses the second convolutional kernel and a first stride; and performing up-sampling on the third encoding result by using an up-sampling layer to obtain the second encoding result.

[0152] Optionally, the processor can further execute program codes of the following steps: performing splicing processing on the first encoding result and the second encoding result in a feature dimension by using a splicing layer to obtain a splicing processing result; and performing addition processing on the splicing processing result in the feature dimension by using a feature addition layer to obtain the target feature map.

[0153] Optionally, the processor can further execute program codes of the following steps: obtaining an original feature map; performing encoding processing on the original feature map by using a first network model to obtain a first encoding result and performing encoding processing on the original feature map by using a second network model to obtain a second encoding result, wherein the first network model uses a bottleneck network structure in a feature dimension and the second network model uses a bottleneck network structure in a feature space; and performing feature fusion processing on the first encoding result and the second encoding result to obtain a target feature map.

[0154] Optionally, the processor can further execute program codes of the following steps: receiving a currently input original feature map; sending the original feature map to a server; and receiving a target feature map from the server, wherein the target feature map is a feature map obtained by the server by performing feature fusion processing on a first encoding result and a second encoding result, the first encoding result is an encoding result obtained by the server by performing feature dimension encoding processing on the original feature map, and the second encoding result is an encoding result obtained by the server by performing feature space encoding processing on the original feature map.

[0155] Optionally, the processor can further execute program codes of the following steps: receiving an original feature map from a client; performing feature dimension encoding processing on the original feature map to obtain a first encoding result and performing feature space encoding processing on the original feature map to obtain a second encoding result; performing feature fusion processing on the first encoding result and the second encoding result to obtain a target feature map; and returning the target feature map to the client.

[0156] The embodiment of the present application provides a feature map processing scheme. The original feature map is acquired; the original feature map is subjected to feature dimension coding processing to obtain a first coding result and the original feature map is subjected to feature space coding processing to obtain a second coding result; the first coding result and the second coding result are subjected to feature fusion processing to obtain a target feature map.

[0157] It is easily noticed that the embodiment of the present application adopts a parallel feature coding mode of the bottleneck network structure in the feature dimension and the bottleneck network structure in the feature space, and subjects the original feature map to feature dimension coding processing and feature space coding processing, that is, the embodiment of the present application simultaneously encodes in the low-dimensional and low-resolution feature space and the feature dimension, and the low-dimensional and low-resolution means that less calculation amount is adopted; the low resolution can expand the receptive field of the convolutional neural network model and enhance the expression ability of the detection model, and therefore has a higher average accuracy.

[0158] Therefore, the embodiment of the present application achieves the purpose of considering the higher average processing accuracy and the lower calculation cost when processing the feature map, thereby realizing the technical effect of meeting the feature map processing needs of customers of various consumption levels, and further solving the technical problem that the higher average accuracy and the lower calculation cost cannot be considered when processing the feature map in the related art.

[0159] Those skilled in the art can understand that the above-mentioned embodiments can be implemented by one or more of the following technical solutions. Figure 11 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Figure 11 It does not limit the structure of the electronic device. For example, the computer terminal can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 11 The computer terminal can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 11 The computer terminal can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0160] Those skilled in the art can understand that all or part of the steps of the various methods of the above-mentioned embodiments can be completed by a program instructing the related hardware of the terminal device, and the program can be stored in a computer-readable nonvolatile storage medium, which can include a flash disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.

[0161] Embodiment 5

[0162] According to the embodiments of the present application, an embodiment of a non-volatile storage medium is also provided. Optionally, in the present embodiment, the non-volatile storage medium can be used to save the program code executed by the feature map processing method provided in the above embodiment 1.

[0163] Optionally, in the present embodiment, the non-volatile storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0164] Optionally, in the present embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining an original feature map; performing feature dimension coding processing on the original feature map to obtain a first coding result and performing feature space coding processing on the original feature map to obtain a second coding result; and performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map.

[0165] Optionally, in the present embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: reducing the feature dimension of the original feature map from a first dimension to a second dimension by using a first convolutional layer to obtain a dimension reduction processing result, wherein the first convolutional layer uses a first convolutional kernel and a first span; and performing feature coding on the dimension reduction processing result by using a second convolutional layer to obtain the first coding result, wherein the second convolutional layer uses a second convolutional kernel and the first span.

[0166] Optionally, in the present embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: mapping the feature resolution of the original feature map from a first resolution to a second resolution by using a third convolutional layer to obtain a mapping processing result, wherein the third convolutional layer uses a second convolutional kernel and a second span; performing feature coding on the mapping processing result by using a fourth convolutional layer to obtain a third coding result, wherein the fourth convolutional layer uses the second convolutional kernel and the first span; and performing up-sampling on the third coding result by using an up-sampling layer to obtain the second coding result.

[0167] Optionally, in the present embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: performing first-level feature space coding processing on the original feature map to obtain a first intermediate result; performing second-level feature space coding processing on the original feature map to obtain a second intermediate result; performing splicing processing on the first intermediate result and the second intermediate result to obtain a third intermediate result; and performing up-sampling on the third intermediate result by using an up-sampling layer to obtain the second coding result.

[0168] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: mapping, by a fifth convolutional layer, a feature resolution of the original feature map from the first resolution to a second resolution to obtain a first mapping processing result, wherein the fifth convolutional layer uses a second convolutional kernel and a second stride; and performing feature encoding on the mapping processing result by a sixth convolutional layer to obtain the first intermediate result, wherein the sixth convolutional layer uses the second convolutional kernel and a first stride.

[0169] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: mapping, by a seventh convolutional layer, a feature resolution of the original feature map from the second resolution to a third resolution to obtain a second mapping processing result, wherein the seventh convolutional layer uses the second convolutional kernel and the second stride; performing feature encoding on the mapping processing result by an eighth convolutional layer to obtain a fourth intermediate result, wherein the eighth convolutional layer uses the second convolutional kernel and the first stride; and upsampling the fourth intermediate result by the upsampling layer to obtain the second intermediate result.

[0170] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: performing splicing processing on the first encoding result and the second encoding result in a feature dimension by a splicing layer to obtain a splicing processing result; and performing addition processing on the splicing processing result in the feature dimension by a feature addition layer to obtain the target feature map.

[0171] Optionally, in the embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining an original feature map; performing encoding processing on the original feature map by a first network model to obtain a first encoding result and performing encoding processing on the original feature map by a second network model to obtain a second encoding result, wherein the first network model uses a bottleneck network structure in a feature dimension, and the second network model uses a bottleneck network structure in a feature space; and performing feature fusion processing on the first encoding result and the second encoding result to obtain a target feature map.

[0172] Optionally, in the embodiment, the nonvolatile storage medium is configured to store program code for performing the following steps: receiving a current input original feature map; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is a feature map obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is an encoding result obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is an encoding result obtained by the server through feature space encoding processing on the original feature map.

[0173] Optionally, in the embodiment, the nonvolatile storage medium is configured to store program code for performing the following steps: receiving a current input original feature map; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is a feature map obtained by the server through feature fusion processing on a first encoding result and a second encoding result, the first encoding result is an encoding result obtained by the server through feature dimension encoding processing on the original feature map, and the second encoding result is an encoding result obtained by the server through feature space encoding processing on the original feature map.

[0174] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0175] In the above embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0176] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit embodiment described above is only schematic. For example, the division of the units is only a logical function division. There can be another division for actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, units or modules, and can be electrical or other forms.

[0177] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.

[0178] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0179] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable nonvolatile storage medium. Based on this understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a nonvolatile storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned nonvolatile storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0180] The above is only the preferred embodiment of the present application. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A feature map processing method, characterized by, The method comprises the following steps: obtaining an original feature map; performing feature dimension coding processing on the original feature map in parallel to obtain a first coding result, and performing feature space coding processing on the original feature map to obtain a second coding result; performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map; wherein the feature dimension coding processing on the original feature map to obtain the first coding result and the feature space coding processing on the original feature map to obtain the second coding result comprise: performing feature dimension coding processing on the original feature map in parallel using a first double-link network structure to obtain the first coding result, and performing feature space coding processing on the original feature map to obtain the second coding result; the feature dimension coding processing on the original feature map in parallel using the first double-link network structure to obtain the first coding result comprises: using the first double-link network structure to reduce the feature dimension of the original feature map from a first dimension to a second dimension to obtain a dimension reduction processing result; and performing feature coding on the dimension reduction processing result to obtain the first coding result; the feature space coding processing on the original feature map to obtain the second coding result comprises: performing coding processing on the feature space of the first level and the second level of the original feature map in parallel using a second double-link network structure to obtain a first intermediate result and a second intermediate result, respectively; obtaining the second coding result based on the first intermediate result and the second intermediate result.

2. The feature map processing method of claim 1, wherein, the feature dimension coding processing on the original feature map to obtain the first coding result comprises: using a first convolutional layer to reduce the feature dimension of the original feature map from a first dimension to a second dimension to obtain a dimension reduction processing result, wherein the first convolutional layer uses a first convolutional kernel and a first span; using a second convolutional layer to perform feature coding on the dimension reduction processing result to obtain the first coding result, wherein the second convolutional layer uses a second convolutional kernel and the first span.

3. The feature map processing method of claim 1, wherein, the feature space coding processing on the original feature map to obtain the second coding result comprises: using a third convolutional layer to map the feature resolution of the original feature map from a first resolution to a second resolution to obtain a mapping processing result, wherein the third convolutional layer uses a second convolutional kernel and a second span; using a fourth convolutional layer to perform feature coding on the mapping processing result to obtain a third coding result, wherein the fourth convolutional layer uses the second convolutional kernel and a first span; using an up-sampling layer to up-sample the third coding result to obtain the second coding result.

4. The feature map processing method of claim 1, wherein, the feature space coding processing on the original feature map to obtain the second coding result comprises: performing first-level feature space coding processing on the original feature map to obtain the first intermediate result; performing second-level feature space coding processing on the original feature map to obtain the second intermediate result; performing splicing processing on the first intermediate result and the second intermediate result to obtain a third intermediate result; Upsample the third intermediate result by using an upsampling layer to obtain the second encoding result.

5. The feature map processing method of claim 4, wherein, The first-level feature space encoding processing on the original feature map includes: The feature resolution of the original feature map is mapped from a first resolution to a second resolution by using a fifth convolutional layer to obtain a first mapping processing result, wherein the fifth convolutional layer uses a second convolutional kernel and a second stride; The mapping processing result is encoded by using a sixth convolutional layer to obtain the first intermediate result, wherein the sixth convolutional layer uses the second convolutional kernel and a first stride.

6. The feature map processing method of claim 4, wherein, The second-level feature space encoding processing on the original feature map includes: The feature resolution of the original feature map is mapped from a second resolution to a third resolution by using a seventh convolutional layer to obtain a second mapping processing result, wherein the seventh convolutional layer uses a second convolutional kernel and a second stride; The mapping processing result is encoded by using an eighth convolutional layer to obtain a fourth intermediate result, wherein the eighth convolutional layer uses the second convolutional kernel and a first stride; The fourth intermediate result is upsampled by using the upsampling layer to obtain the second intermediate result.

7. The feature map processing method of claim 1, wherein, The feature fusion processing on the first encoding result and the second encoding result includes: The first encoding result and the second encoding result are spliced in the feature dimension by using a splicing layer to obtain a splicing processing result; The splicing processing result is added in the feature dimension by using a feature addition layer to obtain the target feature map.

8. The feature map processing method of claim 1, wherein, The feature map processing method is applied to at least one of the following network models: Target detection model, target recognition model, image segmentation model.

9. A feature map processing method characterized by, It includes: Obtain an original feature map; The original feature map is encoded by using a first network model in a parallel encoding manner to obtain a first encoding result and by using a second network model to obtain a second encoding result, wherein the first network model uses a bottleneck network structure in the feature dimension, and the second network model uses a bottleneck network structure in the feature space; The feature fusion processing on the first encoding result and the second encoding result includes: The original feature map is encoded by using a first network model in a parallel encoding manner to obtain a first encoding result and by using a second network model to obtain a second encoding result, wherein the first network model uses a bottleneck network structure in the feature dimension, and the second network model uses a bottleneck network structure in the feature space; The feature fusion processing on the first encoding result and the second encoding result includes: The original feature map is encoded by using a first network model in a parallel encoding manner to obtain a first encoding result and by using a second network model to obtain a second encoding result, wherein the first network model uses a bottleneck network structure in the feature dimension, and the second network model uses a bottleneck network structure in the feature space; The original feature map is encoded by using a first network model in a parallel encoding manner to obtain a first encoding result and by using a second network model to obtain a second encoding result, wherein the first network model uses a bottleneck network structure in the feature dimension, and the second network model uses a bottleneck network structure in the feature space; The network structure of the first double link is used to reduce the feature dimension of the original feature map from a first dimension to a second dimension, to obtain a dimension reduction processing result; and the dimension reduction processing result is subjected to feature coding, to obtain the first coding result; The feature space coding processing of the original feature map to obtain the second coding result includes: The network structure of the second double link is used to perform coding processing on the feature spaces of the first level and the second level of the original feature map in parallel, to obtain the first intermediate result and the second intermediate result respectively; The second coding result is obtained based on the first intermediate result and the second intermediate result.

10. A feature map processing method, characterized by, It includes: receiving an original feature map of a current input; sending the original feature map to a server; receiving a target feature map from the server, wherein the target feature map is a feature map obtained by the server through feature fusion processing on a first coding result and a second coding result, the first coding result is a coding result obtained by the server through a network structure of a first double link to reduce the feature dimension of the original feature map from a first dimension to a second dimension in parallel, to obtain a dimension reduction processing result, and then to perform feature coding on the dimension reduction processing result, the second coding result is a coding result obtained by the server through a network structure of a second double link to perform coding processing on the feature spaces of the first level and the second level of the original feature map in parallel, to obtain the first intermediate result and the second intermediate result respectively, and then to obtain the coding result based on the first intermediate result and the second intermediate result, and the first coding result and the second coding result are obtained through parallel coding of the network structure of the double link.

11. A feature map processing method, characterized by, It includes: receiving an original feature map from a client; using a parallel coding mode to perform feature dimension coding processing on the original feature map to obtain a first coding result, and to perform feature space coding processing on the original feature map to obtain a second coding result; performing feature fusion processing on the first coding result and the second coding result to obtain a target feature map; returning the target feature map to the client; wherein the feature dimension coding processing on the original feature map to obtain the first coding result, and the feature space coding processing on the original feature map to obtain the second coding result, includes: using a network structure of a first double link to perform feature dimension coding processing on the original feature map in parallel to obtain the first coding result, and to perform feature space coding processing on the original feature map to obtain the second coding result; the using a network structure of a first double link to perform feature dimension coding processing on the original feature map in parallel to obtain the first coding result includes: using a network structure of the first double link to reduce the feature dimension of the original feature map from a first dimension to a second dimension, to obtain a dimension reduction processing result; and performing feature coding on the dimension reduction processing result, to obtain the first coding result; the feature space coding processing of the original feature map to obtain the second coding result includes: Adopting a second double-link network structure, a feature space of a first level and a second level of the original feature map is encoded in parallel to obtain a first intermediate result and a second intermediate result respectively; The second encoding result is obtained based on the first intermediate result and the second intermediate result.

12. A feature map processing method, characterized by, Comprise: An original feature map corresponding to an original input image is acquired; An original feature map is encoded in parallel in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result; A target feature map is obtained by fusing the first encoding result and the second encoding result; Class information and position information of a target object in the original input image are detected based on the target feature map; The original feature map is encoded in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result, comprising: The original feature map is encoded in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result by using a first double-link network structure in parallel; The original feature map is encoded in a feature dimension to obtain a first encoding result by using a first double-link network structure, comprising: The feature dimension of the original feature map is reduced from a first dimension to a second dimension by using a first double-link network structure to obtain a dimension reduction result; and the dimension reduction result is encoded to obtain the first encoding result; The original feature map is encoded in a feature space to obtain a second encoding result, comprising: Adopting a second double-link network structure, a feature space of a first level and a second level of the original feature map is encoded in parallel to obtain a first intermediate result and a second intermediate result respectively; The second encoding result is obtained based on the first intermediate result and the second intermediate result.

13. A feature map processing apparatus, characterized by comprising: Comprise: An original feature map corresponding to an original input image is acquired; An original feature map is encoded in parallel in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result; A target feature map is obtained by fusing the first encoding result and the second encoding result; Class information and position information of a target object in the original input image are detected based on the target feature map; The original feature map is encoded in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result, comprising: The original feature map is encoded in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result by using a first double-link network structure in parallel; The original feature map is encoded in a feature dimension to obtain a first encoding result by using a first double-link network structure, comprising: The feature dimension of the original feature map is reduced from a first dimension to a second dimension by using a first double-link network structure to obtain a dimension reduction result; and the dimension reduction result is encoded to obtain the first encoding result; The original feature map is encoded in a feature space to obtain a second encoding result, comprising: Adopting a second double-link network structure, a feature space of a first level and a second level of the original feature map is encoded in parallel to obtain a first intermediate result and a second intermediate result respectively; The second encoding result is obtained based on the first intermediate result and the second intermediate result. Comprise: An original feature map corresponding to an original input image is acquired; An original feature map is encoded in parallel in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result; A target feature map is obtained by fusing the first encoding result and the second encoding result; Class information and position information of a target object in the original input image are detected based on the target feature map; The original feature map is encoded in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result, comprising: The original feature map is encoded in a feature dimension to obtain a first encoding result and in a feature space to obtain a second encoding result by using a first double-link network structure in parallel; The original feature map is encoded in a feature dimension to obtain a first encoding result by using a first double-link network structure, comprising: The feature dimension of the original feature map is reduced from a first dimension to a second dimension by using a first double-link network structure to obtain a dimension reduction result; and the dimension reduction result is encoded to obtain the first encoding result; The original feature map is encoded in a feature space to obtain a second encoding result, comprising: Adopting a second double-link network structure, a feature space of a first level and a second level of the original feature map is encoded in parallel to obtain a first intermediate result and a second intermediate result respectively; The second encoding result is obtained based on the first intermediate result and the second intermediate result. The encoding processing module is further configured to: utilize a network structure of a first double link to reduce a feature dimension of the original feature map from a first dimension to a second dimension, to obtain a dimension reduction processing result; perform feature encoding on the dimension reduction processing result, to obtain the first encoding result; utilize a network structure of a second double link to perform encoding processing on feature spaces of a first level and a second level of the original feature map in parallel, to obtain a first intermediate result and a second intermediate result respectively; and obtain the second encoding result based on the first intermediate result and the second intermediate result.

14. A non-volatile storage medium, comprising: The non-volatile storage medium comprises a stored program, wherein the program, when executed, controls a device in which the non-volatile storage medium is located to perform the feature map processing method of any one of claims 1 to 12.

15. An electronic device, comprising: Comprise: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following processing steps: obtaining an original feature map; performing feature dimension encoding processing on the original feature map in a parallel encoding manner, to obtain a first encoding result, and performing feature space encoding processing on the original feature map, to obtain a second encoding result; performing feature fusion processing on the first encoding result and the second encoding result, to obtain a target feature map; wherein the feature dimension encoding processing on the original feature map to obtain the first encoding result, and the feature space encoding processing on the original feature map to obtain the second encoding result, comprise: utilizing a network structure of a first double link to perform feature dimension encoding processing on the original feature map in parallel, to obtain the first encoding result, and performing feature space encoding processing on the original feature map, to obtain the second encoding result; the utilizing a network structure of a first double link to perform feature dimension encoding processing on the original feature map in parallel to obtain the first encoding result, comprises: utilizing a network structure of a first double link to reduce a feature dimension of the original feature map from a first dimension to a second dimension, to obtain a dimension reduction processing result; and performing feature encoding on the dimension reduction processing result, to obtain the first encoding result; the performing feature space encoding processing on the original feature map to obtain the second encoding result, comprises: utilizing a network structure of a second double link to perform encoding processing on feature spaces of a first level and a second level of the original feature map in parallel, to obtain a first intermediate result and a second intermediate result respectively; obtaining the second encoding result based on the first intermediate result and the second intermediate result.

Images