A feature map upsampling method, terminal and storage medium

Abstract

The present invention discloses a feature map upsampling method, a terminal and a storage medium. The method divides a feature map to be processed in the channel dimension, performs feature map processing on each feature map having a smaller number of channels than the feature map to be processed, and then aggregates the feature maps to complete the upsampling. This achieves independent processing of features on each channel of the feature map to be processed, can extract features with richer semantics, and improves the performance of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a feature map upsampling method, a terminal and a storage medium. Background Art

[0002] Many deep learning-based image processing methods require feature map upsampling operations, such as image super-resolution. Most of the upsampling modules in existing neural networks perform further feature extraction on the input feature map and then use sub-pixel convolution to complete the final upsampling operation. However, this method of directly processing the initial feature map as a whole limits the semantic information of the features, thereby limiting the performance of image processing.

[0003] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the present invention provides a feature map upsampling method, a terminal and a storage medium, aiming to solve the problem of insufficient feature semantic information caused by overall processing of the initial feature map in the process of upsampling the feature map in the prior art.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A first aspect of the present invention provides a feature map upsampling method, the method comprising:

[0007] Obtain a feature map to be processed, input the feature map to be processed into a pre-trained first neural network, and obtain a plurality of intermediate feature maps through the first neural network, wherein the number of channels of each of the intermediate feature maps is less than the number of channels of the feature map to be processed;

[0008] Inputting the multiple intermediate feature maps into a pre-trained second neural network, and obtaining a target feature map through the second neural network, wherein the length and width of the target feature map are respectively N times the length and width of the feature map to be processed, where N is a positive integer;

[0009] Among them, the first neural network includes at least one segmentation module, and the first input feature map is segmented in each segmentation module. After obtaining multiple first feature maps, they are respectively input into the feature processing module to obtain the second feature map output by the feature processing module, and the number of channels of each of the first feature maps is 1 / N of the number of channels of the first input feature map.

[0010] The feature map upsampling method, wherein the first neural network includes M segmentation module groups, and the mth segmentation module group includes N 2(m-1)The input of the first segmentation module group is the feature map to be processed, and the input of each segmentation module in each of the remaining segmentation module groups is each feature map output by the previous segmentation module group.

[0011] The feature map upsampling method, wherein the first input feature map is segmented to obtain multiple first feature maps, includes:

[0012] Performing channel expansion on the first input feature map to obtain a fourth feature map, where the number of channels of the fourth feature map is N times the number of channels of the first input feature map;

[0013] Segment the fourth feature map to obtain N 2 The first feature map.

[0014] The feature map upsampling method, where N = X M , X is a positive integer.

[0015] The feature map upsampling method, wherein the second neural network includes M aggregation module groups, and the mth aggregation module group includes N 2(M-m) The input of the first aggregation module group is the output of the first neural network, and the input of each aggregation module in each of the remaining aggregation module groups is the output of the N aggregation modules in the previous aggregation module group. 2 The output of the aggregation module, the target feature map obtained by the second neural network, includes:

[0016] In the target aggregation module in the target aggregation module group:

[0017] Perform channel expansion on each of the second input feature maps to obtain each fifth feature map, where the number of channels of the fifth feature map is N times that of the second input feature map;

[0018] Aggregate the fifth feature maps to obtain a sixth feature map, where the number of channels of the sixth feature map is N times the number of channels of the fifth feature map. 2 times;

[0019] The pixel points on each channel of the sixth feature map are rearranged to obtain a third feature map, wherein the number of channels of the third feature map is equal to the number of channels of the fifth feature map, and the length and width of the third feature map are respectively X times that of the sixth feature map.

[0020] The feature map upsampling method, wherein the aggregating the fifth feature maps to obtain a sixth feature map, includes:

[0021] The fifth feature maps are rearranged and combined in the channel dimension to obtain the sixth feature map.

[0022] The feature map upsampling method, wherein the pixel points on each channel of the sixth feature map are rearranged to obtain a third feature map, comprises:

[0023] The sixth characteristic map X 2 The pixel points at the same spatial position on the channels are combined into an X*X pixel block to obtain a channel of the third feature map.

[0024] The feature map upsampling method, wherein the first neural network and the second neural network are trained based on multiple groups of training data, each group of training data includes a sample image and a sample target image corresponding to the sample image.

[0025] According to a second aspect of the present invention, a terminal is provided, comprising a processor and a storage medium communicatively connected to the processor, wherein the storage medium is suitable for storing a plurality of instructions, and the processor is suitable for calling the instructions in the storage medium to execute the steps of implementing any of the feature map upsampling methods described above.

[0026] According to a third aspect of the present invention, a storage medium is provided, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the feature map upsampling method described in any one of the above items.

[0027] Compared with the prior art, the present invention provides a feature map upsampling method, a terminal and a storage medium. The method provided by the present invention completes upsampling by dividing the feature map to be processed in the channel dimension, performing feature map processing on each feature map having a smaller number of channels than the feature map to be processed, and then aggregating the feature maps, thereby achieving independent processing of features on each channel of the feature map to be processed, extracting features with richer semantics, and improving image processing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of an embodiment of a feature map upsampling method provided by the present invention;

[0029] Figure 2 A schematic diagram of a segmentation module in an embodiment of a feature map upsampling method provided by the present invention;

[0030] Figure 3 A schematic diagram of an aggregation module in an embodiment of a feature map upsampling method provided by the present invention;

[0031] Figure 4 A schematic diagram of the principles of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0033] Embodiment 1

[0034] The feature map upsampling method provided by the present invention can be applied in a terminal, and the terminal can upsample the feature map by the feature map upsampling method provided by the present invention. The terminal can be, but is not limited to, various computers, mobile phones, tablet computers, vehicle-mounted computers, and portable wearable devices.

[0035] like Figure 1 As shown, in one embodiment of the feature map upsampling method, the steps include:

[0036] S100, obtaining a feature map to be processed, inputting the feature map to be processed into a pre-trained first neural network, and obtaining a plurality of intermediate feature maps through the first neural network.

[0037] Specifically, the feature map upsampling method provided in this embodiment can be used as a part of an image processing task and implemented by an upsampling module in a neural network that performs the image processing task. The image processing task includes but is not limited to various image processing tasks such as image super-resolution and image segmentation. When the image processing task includes a step that requires upsampling of the feature map, the feature map upsampling method provided in this embodiment can be used for upsampling. The feature map to be processed can be a feature map generated after initial feature extraction of the image, or it can be a feature map output in the preceding step of the image processing task.

[0038] In this embodiment, when it is necessary to upsample the feature map to be processed, the feature map to be processed is first segmented in the channel dimension by a first neural network, the first neural network includes multiple segmentation modules, the input of each segmentation module is the first input feature map, the first input feature map is segmented in each segmentation module, and multiple first feature maps are obtained and respectively input into the feature processing module to obtain a second feature map output by the feature processing module, the number of channels of each of the first feature maps is 1 / N of the number of channels of the first input feature map, N is a positive integer, N is the multiple of upsampling the feature map to be processed, the length and width of the feature map obtained after upsampling the feature map N times should be N times the length and width of the initial feature map, respectively, and the number of channels remains unchanged, for example, when N is equal to 2, if the size of the feature map to be processed is H*W*c, H, W, c are length, width, and number of channels, respectively, then, after upsampling the feature map to be processed N times, the size of the target feature map obtained should be 2H*2W*c.

[0039] Specifically, the feature processing module may be a feature processing module in the upsampling process of an existing image processing neural network, that is, the structure of the feature map processing module in the feature map upsampling method provided in this embodiment is the same as the feature map processing module in the existing upsampling process, for example, it may be a feature extraction module, specifically, it may be a plurality of convolution operations, etc. It is not difficult to see that the feature map upsampling method provided in this embodiment may be applied to the upsampling process of various image processing neural networks, and the existing feature processing for overall operation of the feature map is changed to feature processing for independent operation of each feature map after segmentation, so as to improve the performance of various image processing neural networks.

[0040] The parameters of each module in the first neural network are determined during the training process of the first neural network, that is, the parameters of each feature processing module are determined during the training process of the first neural network. The first neural network is a part of the image processing task neural network, and is trained as a whole with other neural network modules in the image processing task neural network (such as the second neural network described later). The image processing task neural network is trained by multiple sets of training data, each set of training data includes a sample image and a sample target image corresponding to the sample image. For example, when the image processing task is an image super-resolution task, the sample image is a low-resolution image, and the sample target image corresponding to the sample image is a high-resolution image corresponding to the low-resolution image. After the training is completed, the image processing task neural network can perform the corresponding image processing task to process the input image. The first neural network is synchronously trained as a part of the image processing task neural network, that is, the first neural network is trained according to multiple sets of training data, each set of training data includes a sample image and a sample target image corresponding to the sample image.

[0041] It is not difficult to see from the above description that since each of the feature processing modules processes the feature map independently, after the training of the first neural network is completed, the parameters of each of the feature processing modules may be different. That is to say, in this embodiment, the feature map to be processed is not feature processed as a whole, but the feature map to be processed is segmented into multiple intermediate feature maps in the channel dimension and then feature processed. In this way, compared with all channels using feature processing modules with the same parameters for feature extraction, different channels of the feature map to be processed can use feature processing modules with different parameters for feature extraction, so that richer semantic information can be obtained.

[0042] The number of channels of the intermediate feature map is smaller than the number of channels of the feature map to be processed. In a possible implementation, the number of segmentation modules in the first neural network can be a fixed value of 1. In this case, the first input feature map is the feature map to be processed, and the second feature map is the intermediate feature map. That is to say, the feature map to be processed is directly segmented on the channel dimension to obtain multiple first feature maps, which are then input into the feature processing module respectively, and multiple intermediate feature maps are output. That is to say, the feature map to be processed is segmented into multiple intermediate feature maps at one time.

[0043] In this embodiment, in order to extract features with richer semantic information, a segmentation module group is set to gradually segment the feature map to be processed to obtain more intermediate feature maps. Specifically, the first neural network includes M segmentation module groups, and the mth segmentation module group includes N 2(m-1)The first segmentation module group has an input of the feature map to be processed, and the inputs of the segmentation modules in each of the remaining segmentation module groups are the feature maps output by the previous segmentation module group. The first input feature map is segmented to obtain multiple first feature maps, including:

[0044] S110, performing channel expansion on the first input feature map to obtain a fourth feature map, wherein the number of channels of the fourth feature map is N times the number of channels of the first input feature map;

[0045] S120, segment the fourth feature map to obtain N 2 The first feature map.

[0046] like Figure 2 As shown, in each of the segmentation modules, the first input feature map input to the segmentation module is firstly channel-expanded. Channel expansion can be implemented by a convolution layer. Specifically, for the first input feature map F GF ∈R H×W×C , using the convolutional layer, F GF ∈R H×W×C The number of channels is mapped to 2C, generating the fourth feature map F of the extended channel EGF , the operation process can be expressed as:

[0047] F EGF =H EC (F GF ) (1)

[0048] Among them, H EC Represents a channel expansion operation.

[0049] After performing channel expansion on the first input feature map to obtain the fourth feature map, the fourth feature map is segmented to obtain N 2 The number of channels of each of the first feature maps is 1 / N of the number of channels of the first input feature map. Specifically, the segmentation is to divide (C4) / (N 2 ) channels are extracted as a new feature map, and C4 is the number of channels of the fourth feature map. Figure 2 As shown, taking the sampling multiple of 2 as an example, in each of the segmentation modules, for the first input feature map with an input size of H*W*c, the number of channels is first expanded to 2 times to obtain the fourth feature map with a size of H*W*2c, and then segmented to obtain the first feature map with a size of H*W*c / 2.

[0050] In each segmentation module, after obtaining the first feature map, the first feature map is input into the feature processing module to obtain a second feature map, and the size of the second feature map is the same as that of the first feature map. The specific operation process can be expressed by the following formula:

[0051]

[0052] in, represents the operation of the i-th feature processing module in the segmentation module, Represents the fourth characteristic graph F EGF the i-th first feature map obtained after segmentation; is the i-th second feature map.

[0053] It is not difficult to see that, in this embodiment, each of the segmentation modules can output N 2 The number of segmentation modules in the next segmentation module group is equal to the number of feature maps output in the previous segmentation module group. The first segmentation module group includes one segmentation module, outputting N 2 The second feature map, the second segmentation module group includes N 2 The segmentation modules, the output of the first segmentation module group is N 2 The second feature maps are respectively used as the input of each segmentation module in the second segmentation module group, and so on. The output of the last segmentation module group is the intermediate feature map. It is not difficult to see that in this embodiment, N can be obtained through the first neural network. 2M The number of channels of each intermediate feature map is 1 / (N 2M ).

[0054] The value of M can be any positive integer, such as 1, 2, 3, etc. In a possible implementation, M satisfies: N = X M , X is a positive integer, for example, X can be 2, 3, etc.

[0055] Please refer again Figure 1 After obtaining a plurality of intermediate feature maps through the first neural network, the feature map upsampling method provided in this embodiment further includes the steps of:

[0056] S200, inputting the multiple intermediate feature maps into a pre-trained second neural network, and obtaining a target feature map through the second neural network.

[0057] The length and width of the target feature map are respectively N times the length and width of the feature map to be processed. Specifically, the second neural network includes at least one aggregation module, and the second input feature map is aggregated in each aggregation module to obtain a third feature map, and the length and width of each of the third feature maps are respectively N times the length and width of the second input feature map.

[0058] The parameters of each aggregation module in the second neural network are determined during the training process of the second neural network. The second neural network is a part of the image processing task neural network, and is trained as a whole with other neural network modules in the image processing task neural network (such as the first neural network mentioned above). The image processing task neural network is trained by multiple sets of training data, each set of training data includes a sample image and a sample target image corresponding to the sample image. For example, when the image processing task is an image super-resolution task, the sample image is a low-resolution image, and the sample target image corresponding to the sample image is a high-resolution image corresponding to the low-resolution image. After the training is completed, the image processing task neural network can perform the corresponding image processing task to process the input image. The second neural network is synchronously trained as a part of the image processing task neural network, that is, the second neural network is trained based on multiple sets of training data, each set of training data includes a sample image and a sample target image corresponding to the sample image.

[0059] In a possible implementation, the number of aggregation modules in the second neural network is a fixed value of 1, that is, the multiple intermediate feature maps are aggregated at one time to obtain the target feature map. In this embodiment, the intermediate feature maps are gradually aggregated. Specifically, the second neural network includes M aggregation module groups, and the mth aggregation module group includes N 2(M-m) The input of the first aggregation module group is the output of the first neural network, and the input of each aggregation module in each of the remaining aggregation module groups is the output of the N aggregation modules in the previous aggregation module group. 2 The output of the aggregation module, the target feature map obtained by the second neural network, includes:

[0060] In the target aggregation module in each aggregation module group:

[0061] S210, performing channel expansion on each of the second input feature maps to obtain each fifth feature map, wherein the number of channels of the fifth feature map is N times that of the second input feature map;

[0062] S220, aggregate the fifth feature maps to obtain a sixth feature map, where the number of channels of the sixth feature map is N times the number of channels of the fifth feature map. 2 times;

[0063] S230. Rearrange the pixel points on each channel of the sixth feature map to obtain a third feature map, wherein the number of channels of the third feature map is equal to the number of channels of the fifth feature map, and the length and width of the third feature map are respectively X times of the sixth feature map.

[0064] The input of each aggregation module is N 2 feature maps, that is, for each aggregation module, the second input feature map has N 2 In each aggregation module, each of the second input feature maps is channel expanded to obtain each fifth feature map, such as Figure 3 As shown, taking the sampling multiple of 2 as an example, the size of the second input feature map is H*W*c in , the size of the fifth feature map obtained after channel expansion of the second input feature map is H*W*c out , c out =2c in Channel expansion can be achieved through convolutional layers. For details, please refer to the description of channel expansion of the first input feature map in the previous article.

[0065] In a single aggregation module, after obtaining the fifth feature map, each of the fifth feature maps is aggregated to obtain a sixth feature map, specifically:

[0066] The channels of the fifth feature maps are combined to obtain the sixth feature map.

[0067] In a single aggregation module, such as Figure 3 As shown, the number of the fifth feature maps is equal to the number of the second input feature maps, which is N 2 , combine each channel of each of the fifth feature maps together to obtain N channels whose number of channels is the number of channels of the fifth feature map 2 times the sixth feature map, specifically, the channels of the fifth feature maps are arranged and combined in sequence according to the order of the fifth feature maps or in other orders.

[0068] After obtaining the sixth feature map, rearranging the pixels on each channel of the sixth feature map to obtain the third feature map may be implemented by using a PixelShuffle operation, which specifically includes:

[0069] The sixth characteristic map X 2 The pixels at the same spatial position on the channels are combined into an X*X pixel block to obtain a channel of the third feature map.

[0070] The number of channels of the third feature map is equal to the number of channels of the fifth feature map. 2 The pixels at the same spatial position on each channel are extracted to obtain X 2 pixel points, these pixel points are combined to obtain a pixel block of size X*X, so that X in the sixth feature map 2 A new channel can be obtained by converting the channels into X channels, and the length and width of the new channel are X times that of the sixth feature map. It is not difficult to see that in all the previous operations, only the number of channels in the feature map size changes, and the length and width remain unchanged. Therefore, the length and width of the third feature map are X times the length and width of the fifth feature map, and the number of channels is N times that of the second input feature map. Figure 3 As shown, the size of the second input feature map is H*W*c in , then the size of the third feature map is (X*H)*(X*W)*c out , c out =N*c in

[0071] The second input feature map is the input of the target aggregation module, and the inputs of each aggregation module in each aggregation module group are respectively N in the previous aggregation module group. 2 The output of the aggregation modules, in the first aggregation module group, includes N 2(M-1) modules, the input of the first aggregation module group is the output of the first neural network, that is, N 2M The number of channels of each intermediate feature map is 1 / (N 2M ), each N 2 The intermediate feature maps are used as inputs of an aggregation module in the first aggregation module group, and the output of each aggregation module is one of the sixth feature maps. In this way, the output of the first aggregation module group is N 2(M-1) The length and width of the sixth feature map output by the first aggregation module group are respectively X times the length and width of the feature map to be processed, and the number of channels is (N times) the number of channels of the feature map to be processed. 2 ) / (N 2M ) times, the second aggregation module group includes N 2(M-2) The first aggregation module group outputs N 2(M-1) In the sixth characteristic graph, each N 2 The sixth feature map is the input of each aggregation module in the second aggregation module, and the length and width of the sixth feature map output by the second aggregation module group are respectively X times the length and width of the feature map to be processed 2 times, the number of channels is (N 4 ) / (N2M ) times, and so on, the last aggregation module group includes an aggregation module, and the length and width of the sixth feature map output are respectively X times the feature map to be processed M times, the channel is the (N 2M ) / (N 2M ) times, that is, the number of channels is equal to the feature map to be processed, and N = X M , that is to say, the sixth feature map output by the last aggregation module is the target feature map.

[0072] To summarize, this embodiment provides a feature map upsampling method, which completes the upsampling by dividing the feature map to be processed in the channel dimension, performing feature map processing on each feature map having a smaller number of channels than the feature map to be processed, and then aggregating them. This achieves independent processing of the features on each channel of the feature map to be processed, and is capable of extracting features with richer semantics, thereby improving the performance of image processing.

[0073] It should be understood that, although the various steps in the flowcharts given in the accompanying drawings of the present specification are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0074] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0075] Embodiment 2

[0076] Based on the above embodiments, the present invention also provides a terminal, such as Figure 4 As shown, the terminal includes a processor 10 and a memory 20. It can be understood that Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0077] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a feature map upsampling program 30 is stored on the memory 20, and the feature map upsampling program 30 can be executed by the processor 10, thereby realizing the feature map upsampling method in the present invention.

[0078] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other chip, used to run the program code or process data stored in the memory 20, such as executing the feature map upsampling method described in Example 1.

[0079] Embodiment 3

[0080] The present invention also provides a storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the feature map upsampling method as described above.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature map upsampling method, characterized in that: The method comprises: Obtain a feature map to be processed, input the feature map to be processed into a pre-trained first neural network, and obtain a plurality of intermediate feature maps through the first neural network, wherein the number of channels of each of the intermediate feature maps is less than the number of channels of the feature map to be processed; The first neural network includes M segmentation module groups, and the mth segmentation module group includes N 2(m-1) segmentation modules, the input of the first segmentation module group is the feature map to be processed, and the input of each segmentation module in each of the remaining segmentation module groups is each feature map output by the previous segmentation module group; N=X M , X is a positive integer; Inputting the multiple intermediate feature maps into a pre-trained second neural network, and obtaining a target feature map through the second neural network, wherein the length and width of the target feature map are respectively N times the length and width of the feature map to be processed, where N is a positive integer; The second neural network includes M aggregation module groups, and the mth aggregation module group includes N 2(M-m) The input of the first aggregation module group is the output of the first neural network, and the input of each aggregation module in each of the remaining aggregation module groups is the output of the N aggregation modules in the previous aggregation module group. 2 The output of the aggregation module, the target feature map obtained by the second neural network, includes: In the target aggregation module in the target aggregation module group: Perform channel expansion on each of the second input feature maps to obtain each fifth feature map, where the number of channels of the fifth feature map is N times that of the second input feature map; Aggregate the fifth feature maps to obtain a sixth feature map, where the number of channels of the sixth feature map is N times the number of channels of the fifth feature map. 2 times; Rearranging the pixel points on each channel of the sixth feature map to obtain a third feature map, wherein the number of channels of the third feature map is equal to the number of channels of the fifth feature map, and the length and width of the third feature map are respectively X times of the sixth feature map; Among them, the first neural network includes at least one segmentation module, and the first input feature map is segmented in each segmentation module. After obtaining multiple first feature maps, they are respectively input into the feature processing module to obtain the second feature map output by the feature processing module, and the number of channels of each of the first feature maps is 1 / N of the number of channels of the first input feature map.

2. The feature map upsampling method according to claim 1, characterized in that: The step of segmenting the first input feature map to obtain a plurality of first feature maps includes: Performing channel expansion on the first input feature map to obtain a fourth feature map, where the number of channels of the fourth feature map is N times the number of channels of the first input feature map; Segment the fourth feature map to obtain N 2 The first feature map.

3. The feature map upsampling method according to claim 1, characterized in that: The step of aggregating the fifth feature maps to obtain a sixth feature map includes: The fifth feature maps are rearranged and combined in the channel dimension to obtain the sixth feature map.

4. The feature map upsampling method according to claim 1, characterized in that: The step of rearranging the pixel points on each channel of the sixth feature map to obtain a third feature map includes: The sixth characteristic map X 2 The pixel points at the same spatial position on the channels are combined into an X*X pixel block to obtain a channel of the third feature map.

5. The feature map upsampling method according to any one of claims 1 to 4, characterized in that: The first neural network and the second neural network are trained based on multiple sets of training data, each set of training data includes a sample image and a sample target image corresponding to the sample image.

6. A terminal, characterized in that: The terminal includes: a processor, a storage medium communicatively connected to the processor, the storage medium being suitable for storing a plurality of instructions, and the processor being suitable for calling the instructions in the storage medium to execute the steps of the feature map upsampling method described in any one of claims 1 to 5 above.

7. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the feature map upsampling method as described in any one of claims 1-5.

Images