Multi-resolution image processing method and device

Abstract

The invention discloses a multi-resolution image processing method, and aims to effectively utilize images with different resolutions to carry out image analysis through a neural network model. A pre-trained neural network model is used in the two implementation modes of the method, the model comprises a plurality of input branches, and each branch corresponds to image data of one resolution. In the equivalent down-sampling method, a feature map of a high-resolution branch is spliced with an image of a low-resolution branch after passing through an equivalent down-sampling layer, then the spliced image is sent to a next feature extraction layer, and finally, a feature map of a lowest-resolution branch is sent to a backbone network to execute an image analysis task. In the equivalent up-sampling method, the feature map of the low-resolution branch is spliced with the image of the high-resolution branch after passing through the equivalent up-sampling layer, and then the spliced image is sent to the next equivalent up-sampling layer. And finally, the feature map of the highest resolution branch is sent to the backbone network to execute an image analysis task, so that the information of an initial image is effectively reserved, and the processing precision of a subsequent model is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a method and device for processing multi-resolution images. Background Art

[0002] With the development of sensor technology, the resolution of modern images is getting higher and higher, but it also brings problems such as large data volume and high processing difficulty. Many satellite sensors provide image data with multiple spatial resolutions. For example, the Sentinel-2 satellite provides images with 10-meter, 20-meter and 60-meter resolutions at the same time. In order to use these image data with different resolutions in the model, it is usually necessary to preprocess the image to unify the resolution before inputting the model. There are two main methods for preprocessing images:

[0003] First, the selection method: select data of a specific resolution from images of different resolutions for processing, for example, only use data with a resolution of 10 meters. Second, the sampling method: unify images of different resolutions to the same resolution by upsampling or downsampling. For example, upsample images with a resolution of 20 meters and 60 meters to a resolution of 10 meters.

[0004] Both of the above existing technologies divide the process of using images with different resolutions into two stages: first, preprocess the images with different resolutions to unify the resolution, and then input the processed single-resolution images into the model. This mode of preprocessing images with different resolutions before inputting them into the model has obvious defects, whether it is selecting a specific resolution or performing up-sampling or down-sampling: selecting resolution will lead to information loss, while up-sampling or down-sampling will introduce noise and increase the computational burden, which ultimately limits the model's effective use of multi-resolution image information. Summary of the invention

[0005] In view of the above problems, this application is proposed to provide a multi-resolution image processing method, apparatus, related equipment and computer program product to achieve effective use of images with different resolutions. The specific solution is as follows:

[0006] In a first aspect, a multi-resolution image processing method is provided, comprising:

[0007] Obtain N types of image data with different resolutions, where N is greater than or equal to 2;

[0008] Obtain a pre-trained neural network model, the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches includes a feature extraction layer, and each of the N input branches except the input branch with the lowest resolution is connected in series with an equivalent downsampling layer after the feature extraction layer, the feature extraction layer of the input branch with the lowest resolution is connected in series with a backbone network, and in descending order of resolution, the equivalent downsampling layer of an input branch with a higher resolution among two adjacent input branches is connected to the feature extraction layer of an input branch with a lower resolution, and the equivalent downsampling layer of the input branch with a higher resolution can downsample the input data to the same resolution as the image data input by the input branch with a lower resolution, and then splice it with the image data input by the input branch with a lower resolution and send it to the feature extraction layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data;

[0009] According to the corresponding relationship of resolutions, N types of image data are input into N input branches of corresponding resolutions in the neural network model, and image analysis results output by the backbone network of the neural network model are obtained.

[0010] Preferably, the feature extraction layer comprises: a convolution layer, a batch normalization layer and an activation function layer connected in series in sequence, and the convolution layer, the batch normalization layer and the activation function layer are used to realize feature extraction.

[0011] Preferably, the equivalent downsampling layer includes: a pooling layer or a convolutional layer with a step size greater than 1.

[0012] Preferably, the backbone network of the neural network model is used to perform image segmentation tasks. During training, the neural network model is trained using image data of different resolutions as training samples and image segmentation annotation results of image data of different resolutions as training labels.

[0013] In a second aspect, another multi-resolution image processing method is provided, including:

[0014] Obtain N types of image data with different resolutions, where N is greater than or equal to 2;

[0015] A pre-trained neural network model is obtained, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches except the input branch with the highest resolution includes an equivalent upsampling layer, and the input branch with the highest resolution includes a backbone network, and in the order of resolution from low to high, the equivalent upsampling layer of an input branch with a low resolution of two adjacent input branches is connected to the equivalent upsampling layer of an input branch with a high resolution, and the equivalent upsampling layer of the input branch with a low resolution can upsample the input data to the same resolution as the image data input by the input branch with a high resolution, and then splice it with the image data input by the input branch with a high resolution and send it to the equivalent upsampling layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data;

[0016] According to the corresponding relationship of resolutions, N types of image data are input into N input branches of corresponding resolutions in the neural network model, and image analysis results output by the backbone network of the neural network model are obtained.

[0017] Preferably, the equivalent upsampling layer comprises:

[0018] The first convolutional layer, reshape layer, first normalization layer and first activation function layer are connected in series.

[0019] The first convolutional layer is used to extract features from input data through multiple different channels to obtain multi-channel feature information;

[0020] The reshaping layer is used to rearrange the multi-channel feature information output by the first convolutional layer to obtain high-resolution feature information to achieve equivalent upsampling.

[0021] Preferably, the equivalent upsampling layer further includes:

[0022] A second convolutional layer, a second batch normalization layer, and a second activation function layer are sequentially connected in series above the first convolutional layer, and the second convolutional layer, the second batch normalization layer, and the second activation function layer are used to realize feature extraction.

[0023] Preferably, the backbone network of the neural network model is used to perform image classification tasks. During training, the neural network model is trained using image data of different resolutions as training samples and image classification annotation results of image data of different resolutions as training labels.

[0024] In a third aspect, a multi-resolution image processing device is provided, comprising:

[0025] An image data acquisition unit, used to acquire image data of N different resolutions, where N is greater than or equal to 2;

[0026] A neural network model acquisition unit is used to acquire a pre-trained neural network model, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches includes a feature extraction layer, and each of the N input branches except the input branch with the lowest resolution is connected in series with an equivalent downsampling layer after the feature extraction layer, and the feature extraction layer of the input branch with the lowest resolution is connected in series with a backbone network, and in the order of resolution from high to low, the equivalent downsampling layer of an input branch with a higher resolution among two adjacent input branches is connected to the feature extraction layer of an input branch with a lower resolution, and the equivalent downsampling layer of the input branch with a higher resolution can downsample the input data to the same resolution as the image data input by the input branch with a lower resolution, and then splice it with the image data input by the input branch with a lower resolution and send it to the feature extraction layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data;

[0027] The image analysis unit is used to input N types of image data into N input branches of corresponding resolutions in the neural network model according to the corresponding relationship of resolutions, and obtain the image analysis results output by the backbone network of the neural network model.

[0028] In a fourth aspect, another multi-resolution image processing device is provided, comprising:

[0029] An image data acquisition unit, used to acquire image data of N different resolutions, where N is greater than or equal to 2;

[0030] A neural network model acquisition unit, used to acquire a pre-trained neural network model, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches except the input branch with the highest resolution includes an equivalent upsampling layer, and the input branch with the highest resolution includes a backbone network, and in the order of resolution from low to high, the equivalent upsampling layer of an input branch with a low resolution of two adjacent input branches is connected to the equivalent upsampling layer of an input branch with a high resolution, and the equivalent upsampling layer of the input branch with a low resolution can upsample the input data to the same resolution as the image data input by the input branch with a high resolution, and then splice it with the image data input by the input branch with a high resolution and send it to the equivalent upsampling layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data;

[0031] The image analysis unit is used to input N types of image data into N input branches of corresponding resolutions in the neural network model according to the corresponding relationship of resolutions, and obtain the image analysis results output by the backbone network of the neural network model.

[0032] By means of the above technical scheme, the present application provides two multi-resolution image processing methods. Through the design of the model structure, the step of pre-processing images of different resolutions to unify the resolution before inputting the images into the model in the traditional method is removed, thereby avoiding the problems of information loss and noise introduction caused by it. By inputting images of different resolutions into different layers inside the neural network model for equivalent downsampling / upsampling operations, the images of different resolutions are fused in the feature space, so that the features of images of different resolutions are comparable in the same space, laying the foundation for the subsequent model processing. Through the steps of feature space conversion, equivalent downsampling / upsampling and iterative processing, image data of different resolutions are effectively utilized. The images input into the model of the neural network are original images of different resolutions, which not only avoids information loss and noise introduction, but also more effectively retains the information of the initial image, significantly improving the processing accuracy of the subsequent model.

[0033] This application introduces two solutions for fusing image data of different resolutions and performing image analysis through the internal structural design of a neural network model for the two scenarios of upsampling and downsampling of image data. The two solutions are respectively applicable to upsampling scenarios and downsampling scenarios. The two solutions have a common inventive concept: both use a neural network model to fuse images of different resolutions and perform image analysis and processing. This is their common innovation point. They are related to each other and have their own emphasis, reflecting different implementation methods of the same inventive concept. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0035] Figure 1 A schematic diagram of an implementation system architecture of a multi-resolution image processing method provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of a terminal structure provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of a server structure provided in an embodiment of the present application;

[0038] Figure 4A schematic diagram of a process flow of a multi-resolution image processing method provided in an embodiment of the present application;

[0039] Figure 5 A schematic diagram of the structure of a neural network model for processing multi-resolution images is shown;

[0040] Figure 6 An example of a structural diagram of a feature extraction layer is shown;

[0041] Figure 7 A schematic flow chart of another multi-resolution image processing method provided in an embodiment of the present application;

[0042] Figure 8 The schematic diagram of the structure of another neural network model for processing multi-resolution images is illustrated;

[0043] Fig. 9 An example of a structural diagram of an equivalent downsampling layer is shown;

[0044] Fig.10 A schematic diagram of the structure of a multi-resolution image processing device provided in an embodiment of the present application;

[0045] Fig.11 A schematic diagram of the structure of another multi-resolution image processing device provided in an embodiment of the present application;

[0046] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] This application provides a multi-resolution image processing method that can be applied to Figure 1 The system architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0049] The terminal 100 or the server 200 can be used alone to execute the multi-resolution image processing method provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used in collaboration to execute the multi-resolution image processing method provided in the embodiment of the present application.

[0050] Next describe Figure 1 The product form of the mid-terminal 100;

[0051] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a teaching screen, a wearable device, a conference terminal, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a drone, a satellite, a remote sensing ground station, an edge computing device, a vehicle-mounted computing platform, etc., and the embodiment of the present application does not impose any restrictions on this.

[0052] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0053] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160, a speaker 161, a microphone 162, an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0054] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0055] Among them, other input devices 132 can receive input data and so on.

[0056] The display unit 140 may be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 may be used to display various interactive interfaces, processing results, etc. in the multi-resolution image processing method.

[0057] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0058] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0059] Among them, the memory 120 can be used to store software codes related to the multi-resolution image processing method, the processor 170 can execute the steps of the multi-resolution image processing method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve corresponding functions.

[0060] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0061] In the embodiment of the present application, the RF unit 110 can send data to the server 200 and receive the processing results sent by the server 200. Exemplarily, the RF unit 110 sends the received image data of different resolutions to the server 200, and the server 200 obtains the pre-trained neural network model to analyze the image data of different resolutions and returns the analysis results to the terminal 100.

[0062] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0063] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0064] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0065] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0066] Next describe Figure 1 The product form of the server 200;

[0067] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0068] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0069] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0070] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0071] The memory 204 may be used to store software codes related to the multi-resolution image processing method, and the processor 202 may execute the steps of the multi-resolution image processing method and may also schedule other units to implement corresponding functions.

[0072] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0073] The present application provides two multi-resolution image processing methods, which are applied to Figure 1 Taking the computer device in the example as an example, the computer device can be Figure 1 The terminal 100 or the system consisting of the terminal 100 and the server 200. Figure 4 , a multi-resolution image processing method specifically comprises the following steps:

[0074] Step S100: Obtain N types of image data with different resolutions, where N is greater than or equal to 2.

[0075] Specifically, N types of image data with different resolutions are obtained, where N is at least 2. These image data can come from different sensors, such as satellites, drones, or ground cameras; or they can be images obtained by the same sensor under different parameter settings, such as different flight altitudes, focal lengths, or exposure times. Image data with different resolutions provide information of different scales. Low-resolution images contain broader scene information, while high-resolution images retain more details.

[0076] Step S110, obtaining a pre-trained neural network model, the model includes N input branches corresponding to the N image data with different resolutions, each of the N input branches includes a feature extraction layer, and each of the N input branches except the input branch with the lowest resolution is connected in series with an equivalent downsampling layer after the feature extraction layer, the feature extraction layer of the input branch with the lowest resolution is connected in series with a backbone network, and in order of resolution from high to low, the equivalent downsampling layer of an input branch with a higher resolution among two adjacent input branches is connected to the feature extraction layer of an input branch with a lower resolution, and the equivalent downsampling layer of the input branch with a higher resolution can downsample the input data to the same resolution as the image data input by the input branch with a lower resolution, and then splice it with the image data input by the input branch with a lower resolution and send it to the feature extraction layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data.

[0077] In a possible implementation, the backbone network of the neural network model in this embodiment is used to perform image segmentation tasks. The neural network model is trained with image data of different resolutions as training samples, and the image segmentation annotation results of image data of different resolutions are used as training labels. The pre-trained neural network model has a structure designed to process image data inputs of different resolutions. This model has N input branches, each branch corresponding to a resolution. Pre-training means that this model has been trained on a large dataset of images of different resolutions, and the model has automatically learned the parameters of each module, so that the up and down sampling of the model does not directly introduce noise, and has a certain feature extraction capability. The input of the model is the original image of different resolutions. The design of the model structure enables the model to process images of different resolutions, solving the problem that the traditional network structure cannot process images of different resolutions.

[0078] Step S120: According to the corresponding relationship between the resolutions, N types of image data are input into N input branches of corresponding resolutions in the neural network model, and the image analysis results output by the backbone network of the neural network model are obtained.

[0079] Specifically, N types of image data with different resolutions are input into the corresponding branches. Due to the design of the model structure, images with different resolutions will be gradually fused and finally processed by the backbone network. The output of the backbone network is the final image analysis result.

[0080] The backbone network receives the image from the lowest resolution branch and performs a predetermined image analysis task. This task can be image classification, object detection, semantic segmentation, etc. The backbone network can be any network structure suitable for the task.

[0081] Figure 5 The schematic diagram of the structure of a neural network model for processing multi-resolution images is illustrated.

[0082] like Figure 5 As shown in the figure, the structure of a neural network model for processing multi-resolution images is shown. Taking N=3 as an example, the image data with input resolutions of 60m, 20m and 10m are gradually downsampled to fuse the high-resolution images into the low-resolution branches.

[0083] The model receives three different resolutions of image data (60m, 20m, 10m) as input, and each resolution corresponds to an independent input branch. Each input branch contains a feature extraction layer Block1, which is used to extract the features of the image and keep the resolution of the input image consistent with the output image. Except for the branch with the lowest resolution (60m), the branch with a resolution of 10m is connected to an equivalent downsampling layer pooling or conv after the feature extraction layer for equivalent downsampling operation, and the branch with a resolution of 20m is connected to an equivalent downsampling layer pooling or conv after the feature extraction layer for equivalent downsampling operation. The equivalent downsampling layer pooling or conv of the 10m resolution branch reduces the resolution of its output image to the same as the input resolution of the 20m branch, and the equivalent downsampling layer pooling or conv of the 20m resolution branch reduces the resolution of its output image to the same as the input resolution of the 60m branch.

[0084] The image output by the equivalent downsampling layer of the high-resolution branch is spliced ​​with the input image of the adjacent low-resolution branch (in the figure For example, the output image of the equivalent downsampling layer pooling or conv of the 10m resolution branch is concatenated with the input image of the 20m resolution branch, and then sent to the feature extraction layer Block1 of the 20m resolution branch for further feature extraction. Similarly, the output image of the equivalent downsampling layer pooling or conv of the 20m resolution branch is concatenated with the input image of the 60m resolution branch, and then sent to the feature extraction layer Block1 of the 60m resolution branch for further feature extraction.

[0085] The feature extraction layer Block1 of the lowest resolution (60m) branch is connected to the backbone network. The network receives images fused with different resolutions and performs predetermined image analysis tasks (such as image classification, target detection, etc.).

[0086] The following examples refer to Figure 6 , the structure of the feature extraction layer Block1 is introduced. Figure 6The feature extraction layer Block1 in contains two identical substructures, each of which consists of the following three layers in series:

[0087] Convolution layer (conv): The convolution layer is the core operation of feature extraction. It uses a set of learnable convolution kernels to perform convolution operations on the input image data to extract different local features. Each convolution kernel corresponds to a specific feature pattern, such as edges, corners, textures, etc. The convolution operation can effectively capture the spatial structure information of the image and has translation invariance, which means that no matter where the target appears in the image, the convolution layer can extract the same features. The parameters of the convolution layer include the size of the convolution kernel, stride, and padding.

[0088] Batch Normalization (BN): The role of the Batch Normalization layer is to normalize the distribution of feature maps, accelerate model training and improve generalization. During the training process, due to the continuous update of network parameters, the input distribution of each layer will change, which is called internal covariate shift. The Batch Normalization layer normalizes the feature maps of each batch (mini-batch) and converts the mean and variance of the feature maps to 0 and 1, thereby reducing the impact of internal covariate shift. The Batch Normalization layer can also introduce learnable scale and shift parameters to further improve the expressiveness of the model.

[0089] Activation function layer (relu): The activation function layer introduces nonlinearity and enhances the expressiveness of the model. Without the activation function, the neural network can only perform linear transformations and cannot learn complex nonlinear relationships. The activation function introduces nonlinearity into the network, allowing the network to approximate arbitrarily complex functions.

[0090] These two substructures are connected in sequence to form the feature extraction layer Block1. Through this combination, Block1 can effectively extract image features while keeping the resolution of the input image and the output image consistent. In addition, the substructures including the convolution layer, the batch normalization layer, and the activation function layer can also be multiple to jointly constitute the feature extraction layer in this application, and a multi-level structure can be used to extract deeper features.

[0091] The design of the feature extraction layer Block1 effectively combines the convolution layer, batch normalization layer and activation function layer to achieve efficient feature extraction. The feature extraction layer Block1 can automatically learn parameters so that the extracted feature map has spatially approximate linear properties in the feature space, so that the subsequent equivalent downsampling operation will not have the consequence of introducing noise by direct interpolation. Its modular design also facilitates the construction of more complex network structures, thereby better adapting to different application scenarios and needs.

[0092] The following embodiment introduces the equivalent downsampling layer in the foregoing embodiment, where the equivalent downsampling layer includes a pooling layer or a convolutional layer with a step size greater than 1.

[0093] Specifically, the equivalent downsampling layer is a key component for achieving the fusion of features of different resolutions. It downsamples the high-resolution feature map to the same dimension as the low-resolution feature map for feature splicing. The pooling operation of the pooling layer reduces the size of the feature map by aggregating the pixel values ​​in the local area of ​​the feature map, where the pooling operation can be performed once or multiple times. In addition, the size of the feature map can also be reduced through a convolutional layer with a step size greater than 1. This method can perform feature extraction while downsampling.

[0094] The backbone network in the aforementioned embodiment plays a vital role in the image analysis task. It is responsible for using the aforementioned fused images of uniform resolution to complete specific image analysis tasks. Depending on different application scenarios, the backbone network can implement a variety of image analysis tasks, such as: Image classification: The backbone network can convert the feature map of the input image into a category probability for identifying the category to which the image belongs. Target detection: The backbone network can be used to extract the features of the target image and combine it with the target detection algorithm. Semantic segmentation: The backbone network can assign a semantic label to each pixel, thereby segmenting the image into different regions.

[0095] The present application provides an optional scenario example, taking the classification of land parcels using satellite remote sensing images as an example: in this scenario, the backbone network can be used to input the fused satellite remote sensing images of uniform resolution into the classifier to achieve the classification of land parcels, such as identifying different land use types, crop types, etc.

[0096] The design of the backbone network needs to be adjusted according to the specific application scenario and task requirements. The framework based on multi-scale feature fusion provided in this application can flexibly adapt to different backbone networks to achieve various image analysis tasks.

[0097] Reference Figure 7 Another multi-resolution image processing method of the present application specifically includes the following steps:

[0098] Step S200: Obtain N types of image data with different resolutions, where N is greater than or equal to 2.

[0099] Step S210, obtaining a pre-trained neural network model, the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches except the input branch with the highest resolution includes an equivalent upsampling layer, the input branch with the highest resolution includes a backbone network, in order of resolution from low to high, the equivalent upsampling layer of an input branch with a low resolution of two adjacent input branches is connected to the equivalent upsampling layer of an input branch with a high resolution, and the equivalent upsampling layer of the input branch with a low resolution can upsample the input data to the same resolution as the image data input by the input branch with a high resolution, and then splice it with the image data input by the input branch with a high resolution and send it to the equivalent upsampling layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data.

[0100] In a possible implementation, the backbone network of the neural network model in this embodiment is used to perform image classification tasks. The neural network model is trained using image data of different resolutions as training samples and image classification annotation results of image data of different resolutions as training labels. The output of the backbone network is the final image classification result.

[0101] Step S220: According to the corresponding relationship between the resolutions, N types of image data are input into N input branches of corresponding resolutions in the neural network model, and the image analysis results output by the backbone network of the neural network model are obtained.

[0102] Specifically, the backbone network can be any common convolutional neural network architecture or a custom network designed for a specific task. The model can be initialized with pre-trained weights, which can speed up model training and improve performance, especially when training data is limited. The pre-trained weights can come from a general model trained on a large dataset or a specific model trained on a similar task.

[0103] Figure 8 The schematic diagram illustrates the structure of another neural network model for processing multi-resolution images.

[0104] like Figure 8 As shown in the figure, another neural network model structure for processing multi-resolution images is shown. In contrast to the previously provided model, this model uses a progressive up-sampling method to fuse images of different resolutions. Taking N=3 as an example, the input image data have resolutions of 10m, 20m, and 60m respectively.

[0105] The model receives image data of three different resolutions (10m, 20m, 60m) as input, and each resolution of the image corresponds to an independent input branch.

[0106] In addition to the highest resolution (10m) branch, the 60m resolution branch contains an equivalent upsampling layer Block3, and the 20m resolution branch contains an equivalent upsampling layer Block2. The equivalent upsampling layer Block3 of the 60m resolution branch increases the resolution of its output image to the same resolution as the input image of the 20m resolution branch, and the equivalent upsampling layer Block2 of the 20m resolution branch increases the resolution of its output image to the same resolution as the input image of the 10m resolution branch.

[0107] The output image of the equivalent upsampling layer of the low-resolution branch is spliced ​​with the input image of the adjacent high-resolution branch (in the figure For example, the output image of the 60m resolution branch equivalent upsampling layer Block3 is spliced ​​with the input image of the 20m resolution branch, and then sent to the equivalent upsampling layer Block2 of the 20m resolution branch. Similarly, the output image of the 20m resolution branch equivalent upsampling layer Block2 is spliced ​​with the input image of the 10m resolution branch, and sent to the backbone network of the 10m branch.

[0108] The highest resolution (10m) branch is directly connected to the backbone network after stitching the images to perform the predetermined image analysis tasks.

[0109] The following embodiment introduces the equivalent upsampling layer in the above embodiment:

[0110] The equivalent upsampling layer may include a first convolution layer, a reshaping layer, a first batch of normalization layers, and a first activation function layer connected in series in sequence, wherein the first convolution layer is used to extract features of the input data through a plurality of different channels to obtain feature information of multiple channels;

[0111] The reshaping layer is used to rearrange the multi-channel feature information output by the first convolutional layer to obtain high-resolution feature information to achieve equivalent upsampling.

[0112] Specifically, the first convolutional layer increases the number of channels. By increasing the number of channels, more feature maps can be provided, and the convolutional layer can extract richer feature information, allowing the reshaping operation to effectively improve the resolution. This can be understood as the reorganization and expansion of features, creating favorable conditions for upsampling. The increased number of channels allows the network to capture finer details and represent more complex information, and its size and number need to be adjusted according to the specific application scenario.

[0113] The output of the first convolutional layer is fed into the reshape layer. The reshape layer is the core operation of equivalent upsampling. It achieves upsampling by adjusting the dimension of feature information, that is, converting information in the channel dimension to information in the spatial dimension. For example, suppose the size of the input feature map is C×H×W, where C is the number of channels, H and W are the height and width respectively. The reshape layer can convert the size of the feature map to C'×H'×W', where C'<C,H'> H, W'>W, and C×H×W=C'×H'×W'. This achieves the effect of increasing the spatial resolution of the feature map without changing the total number of pixels, which is equivalent to upsampling.

[0114] The reshaped feature maps are fed into the first batch of normalization layers, which normalize the distribution of feature maps, accelerate the training speed of the model, and improve the generalization ability of the model. It normalizes the feature maps of each batch to make their mean 0 and variance 1.

[0115] Finally, the feature map is sent to the first activation function layer. The activation function layer introduces nonlinearity to enhance the model's expressiveness. After being processed by the activation function layer, a feature map with the same resolution as the high-resolution branch input is obtained.

[0116] The equivalent upsampling layer in this embodiment implements an efficient, information-preserving upsampling method within the neural network model through a clever combination of operations such as convolution, reshaping, batch normalization, and activation functions, and lays a solid foundation for subsequent image analysis tasks.

[0117] The following embodiment further expands the structure of the equivalent upsampling layer, and the equivalent upsampling layer further includes:

[0118] A second convolutional layer, a second batch normalization layer, and a second activation function layer are sequentially connected in series above the first convolutional layer, and the second convolutional layer, the second batch normalization layer, and the second activation function layer are used to realize feature extraction.

[0119] Specifically, these three layers constitute a feature extraction module that aims to extract meaningful features from low-resolution inputs before upsampling. The second convolutional layer learns the spatial hierarchy of features. Different convolution kernels capture different aspects of the input data, such as edges, textures, and patterns. The output of this layer is a richer feature representation of the input. The second batch normalization layer helps alleviate the problem of internal covariate shift, allowing the use of higher learning rates and reducing sensitivity to initialization. The second activation function layer introduces nonlinearity into the network, enabling it to learn complex nonlinear relationships between input and output, which enhances the network's expressive power.

[0120] Fig. 9 The following is a schematic diagram of the structure of an equivalent downsampling layer.

[0121] The equivalent upsampling layer is not just a simple upsampling, but combines feature extraction and dimensionality transformation to achieve more effective upsampling. This embodiment takes the above-mentioned equivalent upsampling layer Block3 as an example to introduce its structure. Fig. 9 As shown, Fig. 9 The right side of the figure shows the data dimensions corresponding to different levels in the model structure. The structure is described as follows:

[0122] Feature extraction part: The second convolutional layer (conv) is used to extract local features of the input feature map. The dimension of the input image corresponding to the second convolutional layer is , where c1 is the number of channels, x is the width and height of the feature map, and the input impact of the second convolutional layer is consistent with the dimension of the output image. The second batch normalization layer (bn) normalizes the feature map output by the second convolutional layer to improve training stability. The second activation function layer (relu) introduces nonlinearity.

[0123] Equivalent upsampling part: The first convolution layer (conv) is used to increase the number of channels of the feature map. The dimension of the input image corresponding to the first convolution layer is , the dimension of the output image feature map is The reshape layer adjusts the dimension of the feature map and reshapes the feature map of 9c1 channels into c1 channels. The size of each channel becomes , achieving equivalent upsampling, thereby improving the spatial resolution. The dimension of the output feature map corresponding to the reshaping layer is The first normalization layers and the first activation function layer (bn+relu) process the reshaped feature maps.

[0124] In summary, Block3 first performs feature extraction, then implements equivalent upsampling by increasing the number of channels through convolution operations and reshaping operations, and finally performs batch normalization and activation function processing. This design enables the equivalent upsampling layer to not only improve the resolution, but also effectively extract and retain feature information. The backbone network in the aforementioned embodiment is responsible for using the aforementioned fused images of uniform resolution to complete specific image analysis tasks. Depending on different application scenarios, the backbone network can implement a variety of image analysis tasks, such as image classification, target detection, semantic segmentation, etc.

[0125] This application provides an optional scenario example, taking crop prediction using satellite remote sensing images as an example: in this scenario, the backbone network can be used to input the fused satellite remote sensing images of uniform resolution into the backbone network, such as a regression model or a classification model, to predict crop yield, type, growth stage, etc. The backbone network can output the probabilities or predicted values ​​of different crop types corresponding to the images, thereby generating a crop prediction map that intuitively displays the distribution and prediction results of different crops in the region.

[0126] The design of the backbone network needs to be adjusted according to the specific application scenarios and task requirements. The framework based on multi-scale feature fusion provided in this application can flexibly adapt to different backbone networks to achieve various image analysis tasks, including but not limited to image classification, target detection, semantic segmentation, and crop prediction.

[0127] The following is a description of a multi-resolution image processing device provided in an embodiment of the present application. The multi-resolution image processing device described below and the multi-resolution image processing method described above can refer to each other.

[0128] See also Fig.10 , Fig.10 A schematic diagram of the structure of a multi-resolution image processing device disclosed in an embodiment of the present application.

[0129] like Fig.10 As shown, the device may include:

[0130] An image data acquisition unit 11, used to acquire image data of N different resolutions, where N is greater than or equal to 2;

[0131] The model acquisition unit 12 is used to acquire a pre-trained neural network model, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches includes a feature extraction layer, and each of the N input branches except the input branch with the lowest resolution is connected in series with an equivalent downsampling layer after the feature extraction layer, and the feature extraction layer of the input branch with the lowest resolution is connected in series with a backbone network, and in the order of resolution from high to low, the equivalent downsampling layer of an input branch with a higher resolution among two adjacent input branches is connected to the feature extraction layer of an input branch with a lower resolution, and the equivalent downsampling layer of the input branch with a higher resolution can downsample the input data to the same resolution as the image data input by the input branch with a lower resolution, and then splice it with the image data input by the input branch with a lower resolution and send it to the feature extraction layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data;

[0132] The image analysis unit 13 is used to input N types of image data into N input branches of corresponding resolutions in the neural network model according to the corresponding relationship of resolutions, and obtain the image analysis results output by the backbone network of the neural network model.

[0133] Optionally, the feature extraction layer includes:

[0134] The convolutional layer, batch normalization layer and activation function layer are connected in series in sequence, and the convolutional layer, batch normalization layer and activation function layer are used to realize feature extraction.

[0135] Optionally, the equivalent downsampling includes a pooling layer or a convolution layer with a step size greater than 1.

[0136] Optionally, the device may also include a neural network model training unit 14, wherein the backbone network of the neural network model is used to perform image segmentation tasks, and the neural network model is trained using image data of different resolutions as training samples and image segmentation annotation results of image data of different resolutions as training labels.

[0137] See also Fig.11 , Fig.11 It is a schematic diagram of the structure of another multi-resolution image processing device disclosed in an embodiment of the present application.

[0138] An image data acquisition unit 21, used to acquire image data of N different resolutions, where N is greater than or equal to 2;

[0139] The model acquisition unit 22 is used to acquire a pre-trained neural network model, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches except the input branch with the highest resolution includes an equivalent upsampling layer, and the input branch with the highest resolution includes a backbone network, and in the order of resolution from low to high, the equivalent upsampling layer of an input branch with a low resolution of two adjacent input branches is connected to the equivalent upsampling layer of an input branch with a high resolution, and the equivalent upsampling layer of the input branch with a low resolution can upsample the input data to the same resolution as the image data input by the input branch with a high resolution, and then splice it with the image data input by the input branch with a high resolution and send it to the equivalent upsampling layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data;

[0140] The image analysis unit 23 is used to input N types of image data into N input branches of corresponding resolutions in the neural network model according to the corresponding relationship between resolutions, and obtain the image analysis results output by the backbone network of the neural network model.

[0141] Optionally, the equivalent upsampling layer includes: a first convolutional layer, a reshaping layer, a first batch of normalization layers and a first activation function layer connected in series in sequence,

[0142] The first convolutional layer is used to extract features from input data through multiple different channels to obtain multi-channel feature information;

[0143] The reshaping layer is used to rearrange the multi-channel feature information output by the first convolutional layer to obtain high-resolution feature information to achieve equivalent upsampling.

[0144] Optionally, the equivalent upsampling layer also includes: a second convolutional layer, a second batch normalization layer and a second activation function layer connected in series on top of the first convolutional layer, and the second convolutional layer, the second batch normalization layer and the second activation function layer are used to realize feature extraction.

[0145] Optionally, the device may also include a neural network model training unit 24, wherein the backbone network of the neural network model is used to perform image classification tasks, and the neural network model is trained using image data of different resolutions as training samples and image classification annotation results of image data of different resolutions as training labels.

[0146] The present application also provides an electronic device in an embodiment. Fig.12 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, tablet computers, teaching large screens, wearable devices, etc. Fig.12 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0147] like Fig.12 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603, so as to implement the multi-resolution image processing method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0148] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.12 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0149] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any multi-resolution image processing method provided in the embodiment of the present application.

[0150] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any multi-resolution image processing method provided in the embodiment of the present application.

[0151] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0152] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0153] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0154] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A method for processing multi-resolution images, characterized in that: include: Obtain N types of image data with different resolutions, where N is greater than or equal to 2; Obtain a pre-trained neural network model, the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches includes a feature extraction layer, and each of the N input branches except the input branch with the lowest resolution is connected in series with an equivalent downsampling layer after the feature extraction layer, the feature extraction layer of the input branch with the lowest resolution is connected in series with a backbone network, and in descending order of resolution, the equivalent downsampling layer of an input branch with a higher resolution among two adjacent input branches is connected to the feature extraction layer of an input branch with a lower resolution, and the equivalent downsampling layer of the input branch with a higher resolution can downsample the input data to the same resolution as the image data input by the input branch with a lower resolution, and then splice it with the image data input by the input branch with a lower resolution and send it to the feature extraction layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data; According to the corresponding relationship of resolutions, N types of image data are input into N input branches of corresponding resolutions in the neural network model, and image analysis results output by the backbone network of the neural network model are obtained.

2. The method according to claim 1, characterized in that The feature extraction layer comprises: The convolutional layer, batch normalization layer and activation function layer are connected in series in sequence, and the convolutional layer, batch normalization layer and activation function layer are used to realize feature extraction.

3. The method according to claim 1, characterized in that The equivalent downsampling layer comprises: Pooling layers or convolutional layers with stride greater than 1.

4. The method according to claim 1, characterized in that: The backbone network of the neural network model is used to perform image segmentation tasks. During training, the neural network model is trained using image data of different resolutions as training samples and image segmentation annotation results of image data of different resolutions as training labels.

5. A method for processing multi-resolution images, characterized in that: include: Obtain N types of image data with different resolutions, where N is greater than or equal to 2; A pre-trained neural network model is obtained, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches except the input branch with the highest resolution includes an equivalent upsampling layer, and the input branch with the highest resolution includes a backbone network, and in the order of resolution from low to high, the equivalent upsampling layer of an input branch with a low resolution of two adjacent input branches is connected to the equivalent upsampling layer of an input branch with a high resolution, and the equivalent upsampling layer of the input branch with a low resolution can upsample the input data to the same resolution as the image data input by the input branch with a high resolution, and then splice it with the image data input by the input branch with a high resolution and send it to the equivalent upsampling layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data; According to the corresponding relationship of resolutions, N types of image data are input into N input branches of corresponding resolutions in the neural network model, and image analysis results output by the backbone network of the neural network model are obtained.

6. The method according to claim 5, characterized in that The equivalent upsampling layer comprises: The first convolutional layer, reshape layer, first normalization layer and first activation function layer are connected in series. The first convolutional layer is used to extract features from input data through multiple different channels to obtain multi-channel feature information; The reshaping layer is used to rearrange the multi-channel feature information output by the first convolutional layer to obtain high-resolution feature information to achieve equivalent upsampling.

7. The method according to claim 6, characterized in that The equivalent upsampling layer also includes: A second convolution layer, a second batch normalization layer, and a second activation function layer are sequentially connected in series before the first convolution layer, and the second convolution layer, the second batch normalization layer, and the second activation function layer are used to realize feature extraction.

8. The method according to claim 5, characterized in that The backbone network of the neural network model is used to perform image classification tasks. During training, the neural network model is trained using image data of different resolutions as training samples and image classification annotation results of image data of different resolutions as training labels.

9. A multi-resolution image processing device, characterized in that: include: An image data acquisition unit, used to acquire image data of N different resolutions, where N is greater than or equal to 2; A neural network model acquisition unit is used to acquire a pre-trained neural network model, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches includes a feature extraction layer, and each of the N input branches except the input branch with the lowest resolution is connected in series with an equivalent downsampling layer after the feature extraction layer, and the feature extraction layer of the input branch with the lowest resolution is connected in series with a backbone network, and in the order of resolution from high to low, the equivalent downsampling layer of an input branch with a higher resolution among two adjacent input branches is connected to the feature extraction layer of an input branch with a lower resolution, and the equivalent downsampling layer of the input branch with a higher resolution can downsample the input data to the same resolution as the image data input by the input branch with a lower resolution, and then splice it with the image data input by the input branch with a lower resolution and send it to the feature extraction layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data; The image analysis unit is used to input N types of image data into N input branches of corresponding resolutions in the neural network model according to the corresponding relationship of resolutions, and obtain the image analysis results output by the backbone network of the neural network model.

10. A multi-resolution image processing device, characterized in that: include: An image data acquisition unit, used to acquire image data of N different resolutions, where N is greater than or equal to 2; A neural network model acquisition unit, used to acquire a pre-trained neural network model, wherein the model includes N input branches corresponding to the N image data of different resolutions, each of the N input branches except the input branch with the highest resolution includes an equivalent upsampling layer, and the input branch with the highest resolution includes a backbone network, and in the order of resolution from low to high, the equivalent upsampling layer of an input branch with a low resolution of two adjacent input branches is connected to the equivalent upsampling layer of an input branch with a high resolution, and the equivalent upsampling layer of the input branch with a low resolution can upsample the input data to the same resolution as the image data input by the input branch with a high resolution, and then splice it with the image data input by the input branch with a high resolution and send it to the equivalent upsampling layer; the backbone network is used to perform a predetermined image analysis task based on the input feature data; The image analysis unit is used to input N types of image data into N input branches of corresponding resolutions in the neural network model according to the corresponding relationship of resolutions, and obtain the image analysis results output by the backbone network of the neural network model.

Images