Image processing method, device, computer equipment and storage medium

Abstract

The present invention discloses an image processing method, device, computer equipment and storage medium, the method comprising: acquiring an original image, and obtaining a plurality of feature maps according to the original image; performing feature fusion processing according to the resolution level based on a first resolution sub-map, a second resolution sub-map, a third resolution sub-map and a fourth resolution sub-map to obtain a target feature map; performing feature fusion processing according to the target feature map and the original image to obtain a final target feature map; inputting the final target feature map into a first preset convolutional layer for convolution processing to obtain a refined target feature map. The present invention performs feature fusion processing according to the resolution level based on four feature sub-maps with different resolutions to obtain a target feature map, then performs feature fusion processing on the target feature map and the original image to obtain a final target feature map, then inputs the final target feature map into the first preset convolutional layer for convolution processing, and finally obtains a refined target feature map, thereby improving the effect of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method, device, computer equipment and storage medium. Background Art

[0002] In related technologies, with the rise of artificial intelligence, applications based on artificial intelligence algorithms, such as recognition, detection, matting, tracking, generation, and super-resolution image enhancement, have swept all aspects of life. As deep learning algorithms continue to be implemented in real user application scenarios, image processing based on multi-scale features of images is becoming more and more common, but the image processing effect of existing algorithms is poor. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides an image processing method, device, computer equipment and storage medium, which can improve the effect of image processing.

[0004] An image processing method according to a first aspect of the present invention includes:

[0005] Acquire an original image, and obtain a plurality of feature maps according to the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions;

[0006] Based on the first resolution sub-image, the second resolution sub-image, the third resolution sub-image, and the fourth resolution sub-image, feature fusion processing is performed according to the resolution to obtain a target feature image;

[0007] Performing feature fusion processing on the target feature map and the original image to obtain a final target feature map;

[0008] The final target feature map is input into the first preset convolutional layer for convolution processing to obtain a refined target feature map.

[0009] The image processing method according to the embodiment of the present invention has at least the following beneficial effects: based on four feature sub-graphs with different resolutions, feature fusion processing is performed according to high and low resolutions to obtain a target feature graph, and then the target feature graph and the original image are subjected to feature fusion processing to obtain a final target feature graph, and then the final target feature graph is input into the first preset convolution layer for convolution processing, and finally a refined target feature graph is obtained, thereby improving the image processing effect.

[0010] According to some embodiments of the present invention, obtaining a target feature map by fusing the first resolution sub-image, the second resolution sub-image, the third resolution sub-image, and the fourth resolution sub-image according to high and low resolutions includes:

[0011] Performing a first feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a first target feature image;

[0012] Feature fusion processing is performed based on the first target feature map, the third resolution sub-map and the fourth resolution sub-map to obtain a target feature map.

[0013] According to some embodiments of the present invention, the further comprising:

[0014] Performing a second feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a second target feature image;

[0015] Inputting the second target feature map and the original image into a second preset convolutional layer for convolution processing to obtain a first refined target feature map;

[0016] Inputting the first refined target feature map and the refined target feature map into a preset loss function to obtain a loss value;

[0017] The final refined target feature map is obtained by adjusting the loss value.

[0018] According to some embodiments of the present invention, obtaining the loss value further includes:

[0019] Performing a third feature fusion process based on the third resolution sub-image and the fourth resolution sub-image to obtain a third target feature image;

[0020] Inputting the third target feature map and the original image into a second preset convolutional layer for convolution processing to obtain a second refined target feature map;

[0021] The first refined target feature map, the second refined target feature map and the refined target feature map are input into a preset loss function to obtain a loss value.

[0022] According to some embodiments of the present invention, performing feature fusion processing on the target feature map and the original image to obtain a final target feature map includes:

[0023] Performing convolution processing on the target feature map and interpolating and amplifying it to obtain an amplified target feature map;

[0024] The enlarged target feature map and the original image are cascaded to obtain a final target feature map.

[0025] According to some embodiments of the present invention, performing a first feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a first target feature image includes:

[0026] Obtaining an attention weight according to the first resolution sub-image and the second resolution sub-image;

[0027] A first target feature map is obtained according to the attention weight, the first resolution sub-map and the second resolution sub-map.

[0028] According to some embodiments of the present invention, the further comprising:

[0029] The number of layers of the first preset convolutional layer is greater than the number of layers of the second preset convolutional layer.

[0030] An image processing device according to a second aspect of an embodiment of the present invention includes:

[0031] A first processing module: used to obtain an original image, and obtain a plurality of feature maps according to the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions;

[0032] A second processing module: configured to perform feature fusion processing based on the first resolution sub-image, the second resolution sub-image, the third resolution sub-image, and the fourth resolution sub-image according to the resolution to obtain a target feature image;

[0033] The third processing module is used to perform feature fusion processing according to the target feature map and the original image to obtain a final target feature map;

[0034] The fourth processing module is used to input the final target feature map into the first preset convolution layer for convolution processing to obtain a refined target feature map.

[0035] According to a third aspect of an embodiment of the present invention, a computer device comprises a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor is used to execute:

[0036] A method as described in any one of the embodiments of the first aspect of the present invention.

[0037] According to a storage medium of a fourth aspect of an embodiment of the present invention, the storage medium is a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the computer is used to execute:

[0038] A method as described in any one of the embodiments of the first aspect of the present invention.

[0039] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0041] Figure 1 An overall flow chart of an image processing method provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of another image processing method provided by an embodiment of the present invention;

[0043] Figure 3 A flowchart of another image processing method provided by an embodiment of the present invention;

[0044] Figure 4 This is a flowchart of the image processing method provided in Example 1 and Example 2 of the present invention. DETAILED DESCRIPTION

[0045] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0046] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0047] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0048] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0049] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0050] First, several nouns involved in the present invention are analyzed:

[0051] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science. AI attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0052] AttaNet (Attention-Augmented Network): It mainly consists of two modules: Strip Attention Module (SAM) and Attention Fusion Module (AFM).

[0053] Self-attention mechanism: The attention mechanism enables the neural network to focus on a subset of its input (or features), select specific inputs, and can be applied to any type of input regardless of its shape. In the case of limited computing power, the attention mechanism is a resource allocation solution that is the main means to solve the problem of information overload, allocating computing resources to more important tasks.

[0054] In related technologies, with the rise of AI, AI-based applications such as recognition, detection, matting, tracking, generation, and super-resolution image enhancement have swept all aspects of life. As deep learning algorithms continue to be implemented in real user application scenarios, it is found that most AI algorithms have high platform requirements. Therefore, how to make our AI algorithms run on low-cost terminal devices is the top problem we need to solve. This also brings about a decrease in the expected effect and a decrease in accuracy. How to balance performance and effect is the pain point and difficulty of the current AI algorithm implementation.

[0055] Based on this, an embodiment of the present invention provides an image processing method, apparatus, computer device and storage medium, and the image processing method includes but is not limited to the following steps:

[0056] An original image is acquired, and a plurality of feature maps are obtained based on the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions; based on the first resolution sub-map, the second resolution sub-map, the third resolution sub-map, and the fourth resolution sub-map, feature fusion processing is performed according to the resolution to obtain a target feature map; feature fusion processing is performed based on the target feature map and the original image to obtain a final target feature map; the final target feature map is input into a first preset convolution layer for convolution processing to obtain a refined target feature map.

[0057] It should be noted that the embodiment of the present invention mainly realizes image processing based on the EfficientNet-b3 backbone network + AttaNet basic network framework + Refine refinement layer.

[0058] According to the technical solution of the embodiment of the present invention, the embodiment of the present invention can perform feature fusion processing based on four feature sub-graphs with different resolutions to obtain a target feature graph, then perform feature fusion processing on the target feature graph and the original image to obtain a final target feature graph, then input the final target feature graph into the first preset convolution layer for convolution processing, and finally obtain a refined target feature graph, thereby improving the effect of image processing. In particular, in processing image matting, both the matting effect and the matting speed are effectively improved.

[0059] In summary, the embodiment of the present invention obtains four feature maps of different resolutions based on the backbone network EfficientNet-b3, performs feature fusion based on these feature maps, and then obtains three output feature maps of different resolutions after information fusion with the original image through the refinement layer. The final image processing effect is obtained by continuous fitting and training through loss functions such as F1 / MSE / SSIM / Laplacian pyramid.

[0060] The embodiments of the present invention provide an image processing method, an apparatus, a computer device, and a storage medium, which are specifically described through the following embodiments. First, the image processing method in the embodiments of the present invention is described.

[0061] Reference Figure 1 , the image processing method according to the embodiment of the present invention includes but is not limited to steps S110 to S140.

[0062] Step S110, acquiring an original image, and obtaining a plurality of feature maps according to the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions;

[0063] Step S120, performing feature fusion processing based on the first resolution sub-image, the second resolution sub-image, the third resolution sub-image, and the fourth resolution sub-image according to the resolution to obtain a target feature image;

[0064] Step S130, performing feature fusion processing according to the target feature map and the original image to obtain a final target feature map;

[0065] Step S140, inputting the final target feature map into the first preset convolutional layer for convolution processing to obtain a refined target feature map.

[0066] In step S110, the original image is input into the EfficientNet-b3 backbone network to obtain four output feature maps of different resolutions. For example, the RGB three-channel color image of the input original image (1x3x320x320) is output as follows: the fourth resolution sub-image (1x16x160x160), the third resolution sub-image (1x24x80x80), the second resolution sub-image (1x48x40x40) and the first resolution sub-image (1x120x20x20).

[0067] In step S120, feature fusion is performed based on four different resolutions according to the resolution level to obtain a target feature map. In practice, in order to increase the speed of image processing, the method of reducing the image resolution is generally used. However, this will affect the image processing effect. Therefore, in this step, high and low resolution features are fused based on four different resolutions, which ensures the image processing effect under high-resolution images, and improves the image processing speed by using low-resolution images to assist imaging.

[0068] In step S130, in order to further improve the effect of image processing, the original image and the target feature map are introduced to perform feature fusion to obtain the final target feature map.

[0069] In step S140, the final target feature map is input into the first preset convolution layer for convolution processing, aiming to refine the target feature map through convolution processing, so that the final target feature map has better effect.

[0070] In some embodiments, reference Figure 2 , step S120 includes but is not limited to steps S210 to S230.

[0071] Step S210, performing a first feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a first target feature image;

[0072] Step S220, performing feature fusion processing based on the first target feature map, the third resolution sub-map and the fourth resolution sub-map to obtain a target feature map.

[0073] Specifically, the first target feature map is obtained by first performing feature fusion on two low-resolution sub-images, that is, the first target feature map is obtained based on the first resolution sub-image and the second resolution sub-image, and then the target feature map is obtained based on the first target feature map and the two high-resolution sub-images, and the target feature map is obtained based on the first target feature map, the third resolution sub-image and the fourth resolution sub-image. Image fusion based on the resolution ensures the image processing effect under the high-resolution image, and improves the image processing speed by using the low-resolution image to assist imaging.

[0074] It is understandable that the attention mechanism is introduced for better image fusion effect. Specifically, the attention weight is obtained according to the first resolution sub-image and the second resolution sub-image; and the first target feature map is obtained according to the attention weight, the first resolution sub-image and the second resolution sub-image.

[0075] In some embodiments, reference Figure 3 The image processing method according to the embodiment of the present invention further includes but is not limited to steps S310 to S340.

[0076] Step S310, performing a second feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a second target feature image;

[0077] Step S320, inputting the second target feature map and the original image into a second preset convolutional layer for convolution processing to obtain a first refined target feature map;

[0078] Step S330, inputting the first refined target feature map and the refined target feature map into a preset loss function to obtain a loss value;

[0079] Step S340, obtaining the final refined target feature map according to the adjusted loss value.

[0080] Specifically, in order to further improve the image processing effect, the first refined target feature map is obtained by fusion of the features of the first resolution sub-image, the second resolution sub-image and the original image. The first refined target feature map is used as an auxiliary, and the loss value is adjusted to further refine the refined target feature map to obtain a final refined target feature map with better image effect.

[0081] It should be noted that the preset loss function can be a regression loss function such as L1 / MAE / MSE.

[0082] Specifically, in order to highlight the auxiliary role of the first refined target feature map, the number of first preset convolutional layers is limited to be greater than the number of second preset convolutional layers.

[0083] It is understandable that the loss value can also be obtained by the following steps, including:

[0084] Performing a third feature fusion process based on the third resolution sub-image and the fourth resolution sub-image to obtain a third target feature image;

[0085] Inputting the third target feature map and the original image into the second preset convolution layer for convolution processing to obtain a second refined target feature map;

[0086] The first refined target feature map, the second refined target feature map and the refined target feature map are input into a preset loss function to obtain a loss value.

[0087] It should be noted that the high-resolution feature map fusion is used for subsequent image edge feature extraction and fitting.

[0088] It can be understood that feature fusion processing is performed based on the target feature map and the original image. Specifically, the target feature map is convolved and interpolated to obtain an enlarged target feature map; the enlarged target feature map and the original image are cascaded to obtain the final target feature map.

[0089] The image processing method of the present invention is explained below through three practical examples.

[0090] Example 1, refer to Figure 4 , assuming that the original image is a (1x3x320x320) RGB three-channel color image, then in Example 1 the method includes:

[0091] The head network structure design uses EfficientNet-b3 as the backbone network, based on its output of four output feature maps with different resolutions, namely: the fourth resolution sub-map (1x16x160x160), the third resolution sub-map (1x24x80x80), the second resolution sub-map (1x48x40x40) and the first resolution sub-map (1x120x20x20);

[0092] A first target feature map (1x64x80x80) is obtained by performing a first feature fusion process on the second resolution sub-image (1x48x40x40) and the first resolution sub-image (1x120x20x20);

[0093] Based on the first target feature map (1x64x80x80), the third resolution sub-map (1x24x80x80) and the fourth resolution sub-map (1x16x160x160), feature fusion is performed to obtain the target feature map (1x64x320x320);

[0094] Based on the target feature map (1x64x320x320) and the original image (1x3x320x320), feature fusion is performed to obtain the final target feature map (1x67x320x320);

[0095] The final target feature map (1x67x320x320) is input into three convolutional layers for convolution processing to obtain a refined target feature map (1x1x320x320).

[0096] Example 2, refer to Figure 4 Based on Example 1, processing of the refined target feature map is added, and in Example 2, the method further includes:

[0097] A second target feature map (1x64x40x40) is obtained by performing a second feature fusion process on the second resolution sub-map (1x48x40x40) and the first resolution sub-map (1x120x20x20).

[0098] Based on the second target feature map (1x64x40x40) and the original image (1x3x320x320), two convolution layers are input for convolution processing to obtain the first refined target feature map (1x1x40x40);

[0099] Based on the third resolution sub-image (1x24x80x80) and the fourth resolution sub-image (1x16x160x160), feature fusion is performed to obtain the third target feature map (1x27x160x160);

[0100] Based on the third target feature map (1x27x160x160) and the original image (1x3x320x320), two convolution layers are input for convolution processing to obtain the second refined target feature map (1x1x160x160);

[0101] Input the first refined target feature map (1x1x40x40), the second refined target feature map (1x1x160x160), and the refined target feature map (1x1x320x320) into the preset loss function to obtain a loss value;

[0102] The final refined target feature map (1x1x320x320) is obtained according to the adjusted loss value.

[0103] Example 3: Assume that the input is a (1x3x320x320) RGB three-channel color image.

[0104] 1) The head network structure design uses EfficientNet-b3 as the backbone network, based on its output feature maps of four different resolutions, namely: high-resolution feature maps (1x16x160x160 and 1x24x80x80), low-resolution feature maps (1x48x40x40 and 1x120x20x20).

[0105] 2) Obtain the above four feature maps with different resolutions and perform feature map fusion;

[0106] (1) First, the low-resolution feature map (1x48x40x40) is passed through a convolutional layer (input is 48, output is 64, kernel size is 3, and step size is 1) to obtain the first feature map (1x64x40x40);

[0107] (2) The first feature map (1x64x40x40) and the low-resolution feature map (1x120x20x20) are used for attention calculation to obtain an attention weight (1x64x1x1). The specific implementation details are: first, the low-resolution feature map (1x120x20x20) is enlarged by 2 times to (1x120x40x40), and then cascaded with the first feature map (1x64x40x40) to obtain (1x184x40x40), which enters a convolutional layer (input is 184, output is 64, kern el size is 1, step size is 1) to get the second feature map (1x64x40x40), then enter an average pooling layer (pooling width is 40, pooling height is 40) to get the output (1x64x1x1) weight information, then pass through a convolution layer (input is 64, output is 64, kernel size is 1, step size is 1) to get (1x64x1x1) weight information, finally pass through the batch normalization layer and the HardSigmoid layer to finally get the calculated attention weight information value (1x64x1x1);

[0108] (3) Another low-resolution feature map (1x120x20x20) is passed through a convolutional layer (input is 120, output is 64, kernel size is 3, stride is 1) to obtain the third feature map (1x64x20x20);

[0109] (4) Perform weighted summation on the third feature map and the first feature map. Specifically, directly multiply the obtained attention weight of each layer of (1x64x1x1) with the third feature map (1x64x20x20) to obtain a feature map (1x64x20x20) with attention weight information, and then enlarge the feature map (1x64x20x20) by 2 times to obtain a feature map (1x64x40x40); in addition, 1 minus the weight of each layer of (1x64x1x1) is required to obtain the attention weight of each layer of the first feature map (1x64x40x40), and multiply the two to obtain a feature map (1x64x40x40) with attention weight; finally, add the two feature maps of the same resolution (1x64x40x40) to obtain the fourth feature map (1x64x40x40);

[0110] (5) The fourth feature map passes through a convolution layer (input is 64, output is 64, kernel size is 1, step size is 1) to obtain a smoothed 1x64x40x40 feature map, which is the fifth feature map (1x64x40x40) output by the head network;

[0111] (6) The high-resolution feature map (1x24x80x80) output by the backbone network is enlarged by 2 times to obtain a feature map (1x24x160x160) and cascaded with another high-resolution feature map (1x16x160x160) to obtain a feature map (1x40x160x160), and then passed through a convolutional layer (input is 40, output is 24, kernel size is 3, step size is 1) to obtain the sixth feature map (1x24x160x160);

[0112] (7) Output network structure design: The fifth feature map (1x64x40x40) is enlarged by 2 times to obtain a feature map (1x64x80x80). In addition, the high-resolution feature map (1x16x160x160) output by the backbone network is reduced by 2 times to obtain (1x16x80x80). Then, the above-obtained (1x64x80x80) feature map, (1x16x80x80) feature map and the high-resolution feature map (1x24x80x80) output by the backbone network are cascaded to obtain a feature map (1x104x80x80). Then, the convolutional layer (input is 104, output is 64, kernel size is 3, step size is 1) is smoothed to finally obtain a feature map (1x64x80x80), which is the seventh feature map (1x64x80x80) output by the head network.

[0113] (8) Output network structure design The feature maps obtained by the fifth feature map (1x64x40x40), the sixth feature map (1x24x160x160) and the seventh feature map (1x64x80x80) are respectively fused with the original image information; the specific implementation is: the seventh feature map (1x64x80x80) is enlarged four times to obtain a feature map (1x64x320x320), and then combined with the original image information (1x3x320x 320) to obtain a (1x67x320x320) feature map; the sixth feature map (1x24x160x160) is cascaded with the original image information (1x3x160x160) that is reduced by two times to obtain a (1x27x160x160) feature map; the fifth feature map (1x64x40x40) is cascaded with the original image information (1x3x40x40) that is reduced by 8 times to obtain a (1x67x40x40) feature map;

[0114] (9) Output network structure design: The (1x67x320x320) feature map obtained in (8) is refined through three convolutional layers; enters the first convolution layer (input 67, output 24, kernel size 3, step size 1) to obtain a 1x24x320x320 feature map; enters the second convolution layer (input 24, output 24, kernel size 3, step size 1) to obtain a 1x24x320x320 feature map; enters the third convolution layer (input 24, output 1, kernel size 1, step size 1) to obtain a 1x1x320x320 feature map; finally enters the HardSigmoid layer to obtain the refined target feature map (1x1x320x320);

[0115] (10) The 1x27x160x160 feature map obtained in (8) passes through two convolutional layers; enters the first convolution layer (input is 27, output is 24, kernel size is 3, step size is 1) to obtain a 1x24x160x160 feature map, enters the second convolution layer (input is 24, output is 1, kernel size is 1, step size is 1) to obtain a 1x1x160x160 feature map, and finally enters the Hard Sigmoid layer to obtain the second refined target feature map (1x1x160x160);

[0116] (11) The 1x67x40x40 feature map obtained in (8) is passed through two convolutional layers; it enters the first convolution layer (input is 67, output is 24, kernel size is 3, step size is 1) to obtain a (1x24x40x40) feature map; it enters the second convolution layer (input is 24, output is 1, kernel size is 1, step size is 1) to obtain a (1x1x40x40) feature map; finally, it enters the Hard Sigmoid layer to obtain the first refined target feature map (1x1x40x40).

[0117] An embodiment of the present invention provides an image processing device, which includes but is not limited to:

[0118] A first processing module: used to obtain an original image, and obtain a plurality of feature maps according to the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions;

[0119] The second processing module is used to perform feature fusion processing based on the first resolution sub-image, the second resolution sub-image, the third resolution sub-image and the fourth resolution sub-image according to the resolution to obtain a target feature image;

[0120] The third processing module is used to perform feature fusion processing according to the target feature map and the original image to obtain the final target feature map;

[0121] The fourth processing module is used to input the final target feature map into the first preset convolution layer for convolution processing to obtain a refined target feature map.

[0122] The device of the embodiment of the present invention performs feature fusion processing based on four feature sub-graphs with different resolutions to obtain a target feature map, then performs feature fusion processing on the target feature map and the original image to obtain a final target feature map, and then inputs the final target feature map into the first preset convolution layer for convolution processing, and finally obtains a refined target feature map, thereby improving the image processing effect.

[0123] Among them, the specific execution steps of an image processing device refer to the above-mentioned image processing method, which will not be repeated here.

[0124] An embodiment of the present invention also provides a computer device, comprising: at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method in any of the above method embodiments.

[0125] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, which are executed by one or more control processors, and one or more control processors execute the method in the above method embodiment, for example, executing the above described Figure 1 Steps S110 to S140 of the method, Figure 2 Steps S210 to S220 of the method, Figure 3 Method steps S310 to S340 in.

[0126] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network nodes. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer-readable storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0128] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the purpose of the present invention. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. An image processing method, characterized in that: include: Acquire an original image, and obtain a plurality of feature maps according to the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions, the resolution of the first resolution sub-map being smaller than the resolution of the second resolution sub-map, the resolution of the second resolution sub-map being smaller than the resolution of the third resolution sub-map, and the resolution of the third resolution sub-map being smaller than the resolution of the fourth resolution sub-map; Performing a first feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a first target feature image; Performing feature fusion processing based on the first target feature map, the third resolution sub-map, and the fourth resolution sub-map to obtain a target feature map; Performing feature fusion processing on the target feature map and the original image to obtain a final target feature map; The final target feature map is input into the first preset convolutional layer for convolution processing to obtain a refined target feature map.

2. The image processing method according to claim 1, characterized in that: Also includes: Performing a second feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a second target feature image; Inputting the second target feature map and the original image into a second preset convolutional layer for convolution processing to obtain a first refined target feature map; Inputting the first refined target feature map and the refined target feature map into a preset loss function to obtain a loss value; The final refined target feature map is obtained by adjusting the loss value.

3. The image processing method according to claim 2, characterized in that: The obtaining of the loss value also includes: Performing a third feature fusion process based on the third resolution sub-image and the fourth resolution sub-image to obtain a third target feature image; Inputting the third target feature map and the original image into a second preset convolutional layer for convolution processing to obtain a second refined target feature map; The first refined target feature map, the second refined target feature map and the refined target feature map are input into a preset loss function to obtain a loss value.

4. The image processing method according to claim 1, characterized in that: The performing feature fusion processing according to the target feature map and the original image to obtain a final target feature map includes: Performing convolution processing on the target feature map and interpolating and amplifying it to obtain an amplified target feature map; The enlarged target feature map and the original image are cascaded to obtain a final target feature map.

5. The image processing method according to claim 1, characterized in that: The performing a first feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a first target feature image includes: Obtaining an attention weight according to the first resolution sub-image and the second resolution sub-image; A first target feature map is obtained according to the attention weight, the first resolution sub-map and the second resolution sub-map.

6. The image processing method according to claim 3, characterized in that: Also includes: The number of layers of the first preset convolutional layer is greater than the number of layers of the second preset convolutional layer.

7. An image processing device, characterized in that: include: A first processing module: used to obtain an original image, and obtain a plurality of feature maps according to the original image; wherein the plurality of feature maps include a first resolution sub-map, a second resolution sub-map, a third resolution sub-map, and a fourth resolution sub-map having different resolutions, the resolution of the first resolution sub-map being smaller than the resolution of the second resolution sub-map, the resolution of the second resolution sub-map being smaller than the resolution of the third resolution sub-map, and the resolution of the third resolution sub-map being smaller than the resolution of the fourth resolution sub-map; A second processing module: configured to perform a first feature fusion process based on the first resolution sub-image and the second resolution sub-image to obtain a first target feature map; and perform a feature fusion process based on the first target feature map, the third resolution sub-image and the fourth resolution sub-image to obtain a target feature map; The third processing module is used to perform feature fusion processing according to the target feature map and the original image to obtain a final target feature map; The fourth processing module is used to input the final target feature map into the first preset convolution layer for convolution processing to obtain a refined target feature map.

8. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a program, and when the program is executed by the processor, the processor is used to execute: The method according to any one of claims 1 to 6.

9. A storage medium, the storage medium being a computer-readable storage medium, characterized in that: The computer readable storage has a computer program, and when the computer program is executed by the computer, the computer is used to perform: The method according to any one of claims 1 to 6.

Images