Infrared image processing method and device, equipment and storage medium

Abstract

The invention provides an infrared image processing method and device, equipment and a storage medium, the method is applied to computer equipment deployed with an image processing system, and the method comprises the steps: obtaining a low-resolution infrared image, and dividing the scale type of the low-resolution infrared image through a recognition module to obtain the infrared image of each scale; performing feature extraction on the infrared image of each scale to obtain feature information of each scale; performing graphic transformation on the feature information through a fusion module to obtain high-frequency feature information and low-frequency feature information; performing feature fusion on the high-frequency feature information and the low-frequency feature information to obtain image features; the image features are restored through the sampling module to obtain the high-resolution infrared image, and the image quality of the redrawn infrared image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to an infrared image processing method, apparatus, device, and storage medium. Background Art

[0002] The super-resolution reconstruction technology is to perform image processing on a group of low-resolution image groups with relevant information or complementary information to obtain an image with a higher resolution than the original image. Using the super-resolution reconstruction technology can achieve the restoration of low-resolution images.

[0003] In the prior art, the objects of the super-resolution reconstruction technology are mostly color images. For low-resolution infrared images, through the method of transfer learning, the redrawing of low-resolution infrared images is realized to obtain high-resolution infrared images.

[0004] However, in the prior art, there are differences in the features between infrared images and color images. Transfer learning cannot flexibly change the reconstruction strategy, which reduces the image quality of the redrawn infrared images. Summary of the Invention

[0005] The present application provides an infrared image processing method, apparatus, device, and storage medium to solve the problem of reduced image quality of redrawn infrared images existing in the prior art.

[0006] In a first aspect, the present application provides an infrared image processing method, including:

[0007] Obtain a low-resolution infrared image, and divide the scale types of the low-resolution infrared image through the recognition module to obtain infrared images of each scale;

[0008] Extract features from the infrared images of each scale to obtain feature information of each scale;

[0009] Perform a graphical transformation on the feature information through the fusion module to obtain high-frequency feature information and low-frequency feature information;

[0010] Perform feature fusion on the high-frequency feature information and the low-frequency feature information to obtain image features;

[0011] Restore the image features through the sampling module to obtain a high-resolution infrared image.

[0012] In a possible design, the step of performing feature fusion on the high-frequency feature information and the low-frequency feature information to obtain image features includes: generating refined high-frequency feature information by the restoration module according to the high-frequency feature information; performing graphic transformation on the refined high-frequency feature information and the low-frequency feature information by the fusion module to obtain refined feature information; and generating image features by fusing the refined feature information through the fusion module.

[0013] In a possible design, after generating the image features by fusing the refined feature information through the fusion module, the method further includes: obtaining initial weights of the high-frequency feature information and the low-frequency feature information, where the initial weights are generated by a weight model; performing secondary mapping on the initial weights by the restoration module to obtain fusion weights; and calculating denoised image features according to the fusion weights and the image features.

[0014] In a possible design, the step of performing feature extraction on the infrared images of each scale to obtain feature information of each scale includes: reducing the resolution of the infrared images of each scale by the sampling module; and performing feature extraction on the infrared images of each scale with reduced resolution by the restoration module to obtain feature information of each scale.

[0015] In a possible design, the step of restoring the image features by the sampling module to obtain a high-resolution infrared image includes: enlarging the scale of the low-resolution infrared image by the sampling module; restoring the image features by the restoration module to obtain enlarged feature information; and adding the enlarged feature information to the infrared image with the enlarged scale by the recognition module to obtain a high-resolution infrared image.

[0016] In a second aspect, the present application provides an infrared image processing apparatus, which is applied to a computer device equipped with an image processing system, where the image processing system has multiple recognition modules, sampling modules, restoration modules, and fusion modules; the apparatus includes:

[0017] An acquisition module, configured to acquire a low-resolution infrared image, and divide the low-resolution infrared image into infrared images of each scale by the recognition module;

[0018] An extraction module, configured to perform feature extraction on the infrared images of each scale to obtain feature information of each scale;

[0019] A graphic transformation module, configured to perform graphic transformation on the feature information through the fusion module to obtain high-frequency feature information and low-frequency feature information;

[0020] A fusion module, configured to perform feature fusion on the high-frequency feature information and the low-frequency feature information to obtain image features;

[0021] A restoration module, configured to restore the image features through the sampling module to obtain a high-resolution infrared image.

[0022] In a possible design, the fusion module includes: a first generation unit, configured to generate refined high-frequency feature information according to the high-frequency feature information through the restoration module; a graphic transformation unit, configured to perform graphic transformation on the refined high-frequency feature information and the low-frequency feature information through the fusion module to obtain refined feature information; a second generation unit, configured to fuse the refined feature information through the fusion module to generate image features.

[0023] In a possible design, the fusion module further includes: an acquisition unit, configured to acquire initial weights of the high-frequency feature information and the low-frequency feature information, where the initial weights are generated through a weight model; a mapping unit, configured to perform secondary mapping on the initial weights through the restoration module to obtain fusion weights; a calculation unit, configured to calculate denoised image features according to the fusion weights and the image features.

[0024] In a third aspect, the present application provides a computer device, including:

[0025] At least one processor and a memory;

[0026] The memory stores computer-executable instructions;

[0027] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the infrared image processing method as described in the first aspect and various possible designs of the first aspect above.

[0028] In a fourth aspect, the present application provides a computer storage medium, where computer-executable instructions are stored in the computer storage medium, and when a processor executes the computer-executable instructions, the infrared image processing method as described in the first aspect and various possible designs of the first aspect above is implemented.

[0029] The infrared image processing method, device, equipment and storage medium provided by this application divide the scale of the infrared image through an identification module, extract the feature information of each scale, divide the feature information into high-frequency feature information and low-frequency feature information through a fusion module, fuse the high-frequency feature information and the low-frequency feature information, and use a sampling module to restore the image features to obtain a high-resolution infrared image. Compared with the prior art, the feature information is divided into high frequency and low frequency, and the high-frequency information and low-frequency information of each scale are fused respectively, which is more in line with the image features of the infrared image, makes up for the shortcomings of the transfer learning reconstruction strategy, and improves the image quality of the redrawn infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0031] Figure 1 It is a schematic diagram of the system structure of the computer equipment provided by the embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of the flow of the infrared image processing method provided by an embodiment of the present application;

[0033] Figure 3 It is a schematic diagram of the structure of the infrared image processing device provided by the embodiment of the present application;

[0034] Figure 4 It is a schematic diagram of the hardware structure of the computer equipment provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0036] Super-resolution reconstruction technology is to process a group of low-resolution image groups with relevant or complementary information to obtain an image with a higher resolution than the original image. Using super-resolution reconstruction technology can achieve the restoration of low-resolution images. In the prior art, the objects of super-resolution reconstruction technology are mostly color images. For low-resolution infrared images, through the way of transfer learning, the redrawing of low-resolution infrared images is realized to obtain high-resolution infrared images. However, in the prior art, there are differences in the features between infrared images and color images, and transfer learning cannot flexibly change the reconstruction strategy, which reduces the image quality of redrawn infrared images.

[0037] To solve the above technical problems, the embodiments of the present application propose the following technical concept: The inventor thought of dividing the infrared image into scales. By dividing the feature information into high-frequency and low-frequency parts, fusing the high-frequency feature information of the same scale, fusing the low-frequency information of the same scale, and then using the sampling module to restore the image features to obtain a high-resolution infrared image. Compared with the prior art, by fusing the features separately, the problem that transfer learning cannot flexibly reconstruct the strategy is solved, and the image quality of redrawn infrared images is improved. The following will be described in detail with specific embodiments.

[0038] Figure 1 It is a schematic diagram of the system structure of the computer device provided by the embodiments of the present application. As Figure 1 shown, the computer device includes: a receiving device 101, a processor 102, and a display device 103.

[0039] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the article recognition method. In other feasible embodiments of the present application, the above architecture may include more or fewer components than shown in the figure, or combine some components, or split some components, or different component arrangements, which can be specifically determined according to the actual application scenario and will not be limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0040] In the specific implementation process, the receiving device 101 can be an input / output interface or a communication interface, and can obtain low-resolution infrared images.

[0041] The processor 102 can extract the feature information of the infrared image and restore the image features to obtain a high-resolution infrared image.

[0042] The display device 103 can be used to display the above high-resolution infrared image, etc.

[0043] The display device can also be a touch display screen, which is used to receive user instructions while displaying the above content to realize the operation interaction with the user.

[0044] It should be understood that the above-mentioned processor can be implemented by a processor reading instructions in a memory and executing the instructions, or can be implemented by chip circuits.

[0045] In addition, the network architecture and service scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art know that with the evolution of the network architecture and the emergence of new service scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0046] Embodiment 1

[0047] Figure 2 It is a schematic flowchart of an infrared image processing method provided by an embodiment of the present application. The execution subject of this embodiment can be a computer device deployed with an image processing system, and no special limitation is made here in this embodiment. As Figure 2 shown, the method includes:

[0048] S201: Obtain a low-resolution infrared image, and divide the scale types of the low-resolution infrared image through an identification module to obtain infrared images of each scale.

[0049] In this embodiment, an image processing system is deployed on a computer device, and the architecture of the image processing system is a deep neural network architecture.

[0050] In this embodiment, the identification module can be a convolutional layer.

[0051] Among them, the convolutional layer includes a 3×3 convolutional block, a ReLu (Rectified Linear Unit) activation function, and a batch normalization processing module.

[0052] Among them, the ReLu activation function is used to alleviate the problem of gradient disappearance as the number of network layers of the neural network increases.

[0053] Among them, the batch normalization processing module is used to perform normalization processing on each batch of data.

[0054] Specifically, the computer device deployed with the image processing system obtains a low-resolution infrared image input by the user, transmits the image to the identification module, the identification module initially extracts image features, and divides the scale types of the infrared image according to the image features to obtain infrared images of each scale.

[0055] In this embodiment, the scale type of the infrared image represents the resolution of the infrared image.

[0056] S202: Extract features from the infrared images of each scale to obtain feature information of each scale.

[0057] Specifically, the resolution of infrared images at each scale is reduced by the sampling module, and the infrared images with reduced resolution are transmitted to the restoration module. The restoration module extracts image features from the infrared images with reduced resolution to obtain the feature information of infrared images at each scale.

[0058] Among them, the sampling module is a downsampling module, and the downsampling module reduces the resolution of infrared images to obtain more comprehensive image features of infrared images.

[0059] Among them, the sampling module uses two two-fold downsampling modules.

[0060] Among them, the restoration module is a residual dense module.

[0061] In this embodiment, the medium-scale infrared image goes through one downsampling module.

[0062] In this embodiment, the low-scale infrared image goes through two downsampling modules.

[0063] S203: The feature information is subjected to graphic transformation through the fusion module to obtain high-frequency feature information and low-frequency feature information.

[0064] In this embodiment, the graphic transformation can be wavelet transformation.

[0065] In this embodiment, the fusion module is an adaptive wavelet weighted fusion module. The adaptive wavelet weighted fusion module divides the input features into high-frequency features and low-frequency features through wavelet transformation.

[0066] Specifically, the adaptive wavelet weighted fusion module divides the feature information into high-frequency feature information and low-frequency feature information through wavelet transformation.

[0067] S204: The high-frequency feature information and the low-frequency feature information are subjected to feature fusion to obtain image features.

[0068] Specifically, the residual dense module learns the high-frequency feature information to generate refined high-frequency feature information. The fusion module learns the refined high-frequency feature information through inverse wavelet transformation for the low-frequency feature information, and fuses the two kinds of feature information to obtain image features.

[0069] In this embodiment, the image features include but are not limited to color features, texture features, shape features, and spatial relationship features.

[0070] S205: The image features are restored through the sampling module to obtain high-resolution infrared images.

[0071] Specifically, the upsampling module enlarges the resolution of the low-resolution infrared image, and the residual dense module enlarges the image features and adds the image features to the infrared image after the enlarged resolution, so as to achieve the feature superposition of the infrared image to obtain a high-resolution infrared image.

[0072] As can be seen from the above embodiments, the recognition module divides the scale of the infrared image and extracts the feature information of each scale. The fusion module divides the feature information into high-frequency feature information and low-frequency feature information, fuses the high-frequency feature information and the low-frequency feature information, and uses the sampling module to restore the image features to obtain a high-resolution infrared image. Compared with the prior art, the feature information is divided into high frequency and low frequency, and the high-frequency information and low-frequency information of each scale are fused respectively, which is more in line with the image characteristics of the infrared image, makes up for the shortcomings of the transfer learning reconstruction strategy, and improves the image quality of the redrawn infrared image.

[0073] Embodiment 2

[0074] In an embodiment of the present application, step S204 describes the process of fusing feature information, which is described in detail as follows:

[0075] S2041: The refinement module generates refined high-frequency feature information according to the high-frequency feature information.

[0076] In this embodiment, the refinement module is a residual dense module.

[0077] Among them, the residual dense module is a tool for image restoration, mainly including two parts: dense connection and residual learning. The combination of dense connection and residual learning enables the residual dense block to obtain more and more robust features with fewer parameters.

[0078] Among them, dense connection can realize the reuse of underlying features, enrich the features of the deep network, and prevent the features from being difficult to decode after being highly abstracted.

[0079] Among them, residual learning can efficiently extract and refine features, and effectively prevent feature degradation.

[0080] Exemplarily, the feature information includes but is not limited to color features, texture features, shape features, and spatial relationship features.

[0081] Specifically, the residual dense block learns the high-frequency feature information of each scale through residual learning, and prevents the high-frequency feature information from being difficult to decode through dense connection, and generates refined high-frequency feature information of each scale.

[0082] S2042: The fusion module performs graphic transformation on the refined high-frequency feature information and low-frequency feature information to obtain refined feature information.

[0083] In this embodiment, the graphic transformation is an inverse wavelet transform.

[0084] Specifically, the high-frequency feature information of each scale and the low-frequency feature information of each scale are subjected to inverse wavelet transform through the adaptive wavelet weighted fusion module to obtain the high-frequency features after the refinement of low-frequency feature learning.

[0085] S2043: Generate image features by fusing the refined feature information through the fusion module.

[0086] Specifically, the refined feature information of each scale is multiplied by the fusion weight and then added through the adaptive wavelet weighted fusion module to generate the refined image features.

[0087] As can be seen from the above embodiments, by refining the high-frequency feature information through the restoration module and using the fusion module to fuse the high-frequency feature information and the low-frequency feature information, the forced learning of the high-frequency information by the low-frequency information is realized, making up for the shortcoming that the existing convolutional neural network prefers to learn low-frequency feature information.

[0088] Embodiment Three

[0089] In an embodiment of the present application, after step S2043, it further includes:

[0090] S2044: Obtain the initial weights of the high-frequency feature information and the low-frequency feature information, where the initial weights are generated by the weight model.

[0091] In this embodiment, the weight model is a multi-layer perceptron.

[0092] Specifically, the multi-layer perceptron obtains the high-frequency features and the low-frequency features, and generates an initial weight allocation result according to the image features of the infrared image.

[0093] Among them, the high-frequency feature information of the infrared image shows the features of the infrared image, and the weight assigned by the multi-layer perceptron to the high-frequency feature information is greater than the weight assigned to the low-frequency feature information.

[0094] In this embodiment, the high-frequency feature information and the low-frequency feature information are first subjected to addition of feature elements, the added feature elements are subjected to average pooling, and after being linearly mapped through a 1×1 convolutional layer, they are transmitted to the multi-layer perceptron.

[0095] Among them, average pooling is used to obtain more feature information.

[0096] Among them, using a 1×1 convolutional layer for linear mapping is used to obtain linear feature data and prevent data fitting.

[0097] S2045: Perform a secondary mapping on the initial weights through the restoration module to obtain the fusion weights.

[0098] Specifically, the residual dense module is used to perform quadratic non-linear mapping learning on two groups of different weights, namely high-frequency and low-frequency weights, to obtain high-frequency fusion weights and low-frequency fusion weights.

[0099] Among them, the non-linear mapping is used to fuse the high-frequency weights and low-frequency weights of the image.

[0100] Specifically, the fusion weights are passed through the S function for scale adjustment.

[0101] Among them, the S function is the system function of the visualization simulation tool.

[0102] S2046: Calculate the denoised image features based on the fusion weights and the image features.

[0103] Specifically, multiply the fusion weights with the image features and then sum them to obtain the denoised image features.

[0104] As can be seen from the above embodiments, the initial weights of the high-frequency feature information and the low-frequency feature information are generated through the weight model, and the fusion weights are obtained through the restoration module. The denoising of the image features is achieved by using the fusion weights, reducing the influence of image noise on the image features and improving the clarity of the image features.

[0105] Embodiment 4

[0106] In an embodiment of the present application, step S202 describes the process of extracting feature information, which is described in detail as follows:

[0107] S2021: Reduce the resolution of the infrared images at each scale through the sampling module.

[0108] In this embodiment, the sampling module is a downsampling module.

[0109] Among them, the sampling module uses two downsampling modules with a factor of two.

[0110] In this embodiment, the infrared image at the medium scale undergoes one downsampling module.

[0111] In this embodiment, the infrared image at the low scale undergoes two downsampling modules.

[0112] Specifically, the downsampling module reduces the resolution of the image by inserting zero values between each pixel point of the infrared image.

[0113] S2022: Extract the feature information of each scale from the infrared images at each scale with reduced resolution through the restoration module.

[0114] In this embodiment, the restoration module is a residual dense module.

[0115] Specifically, the residual dense module learns the infrared image features at each scale through residuals, and prevents the feature information from being difficult to decode through dense connections, generating the feature information at each scale.

[0116] As can be seen from the above embodiments, by using the sampling module to reduce the resolution of the infrared image, and using the restoration module to extract the feature information of the infrared image with reduced resolution, the difficulty of feature extraction of the infrared image is reduced by reducing the resolution of the infrared image.

[0117] Embodiment Five

[0118] In an embodiment of the present application, step S205 describes the process of obtaining a high-resolution infrared image, which is described in detail as follows:

[0119] S2051: Expand the scale of the low-resolution infrared image through the sampling module.

[0120] In this embodiment, the sampling module is an upsampling module.

[0121] Specifically, the upsampling module uses an interpolation algorithm to insert new image elements to enhance the resolution of the infrared image.

[0122] Among them, the interpolation algorithm includes but is not limited to nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation, etc.

[0123] S2052: Restore the image features through the restoration module to obtain the expanded feature information.

[0124] In this embodiment, the restoration module is a residual dense module.

[0125] Specifically, the residual dense module learns the feature information through residuals, and prevents the feature information from being difficult to decode through dense connections, generating the expanded feature information.

[0126] S2053: Add the expanded feature information to the infrared image after expanding the scale through the recognition module to obtain a high-resolution infrared image.

[0127] In this embodiment, the recognition module is a convolutional layer.

[0128] Specifically, the convolutional layer performs a convolution operation on the feature information and each pixel region of the infrared image, and superimposes the feature information on the original feature information to realize adding the feature information to the infrared image.

[0129] As can be seen from the above embodiments, by using the sampling module to expand the scale of the original infrared image, using the restoration module to expand the feature information, and using the recognition module to add the feature information to the infrared image after expansion, the feature information is superimposed to obtain a high-resolution infrared image, improving the image quality of the redrawn infrared image.

[0130] Embodiment Six

[0131] Figure 3 It is a schematic structural diagram of the infrared image processing device provided by the embodiment of the present application. As Figure 3 shown, the infrared image processing device 30 includes: an acquisition module 301, an extraction module 302, a graphic transformation module 303, a fusion module 304, and a restoration module 305.

[0132] The acquisition module 301 is used to acquire a low-resolution infrared image, and divide the scale type of the low-resolution infrared image through the recognition module to obtain infrared images of each scale.

[0133] The extraction module 302 is used to extract features from the infrared images of each scale to obtain feature information of each scale.

[0134] The graphic transformation module 303 is used to perform graphic transformation on the feature information through the fusion module to obtain high-frequency feature information and low-frequency feature information.

[0135] The fusion module 304 is used to perform feature fusion on the high-frequency feature information and the low-frequency feature information to obtain image features.

[0136] The restoration module 305 is used to restore the image features through the sampling module to obtain a high-resolution infrared image.

[0137] In an embodiment of the present application, the fusion module 304 includes:

[0138] The first generation unit 3041 is used to generate refined high-frequency feature information through the restoration module according to the high-frequency feature information.

[0139] The graphic transformation unit 3042 is used to perform graphic transformation on the refined high-frequency feature information and the low-frequency feature information through the fusion module to obtain refined feature information.

[0140] The second generation unit 3043 is used to generate image features by fusing the refined feature information through the fusion module.

[0141] In an embodiment of the present application, the fusion module 304 further includes:

[0142] The acquisition unit 3044 is used to acquire the initial weights of the high-frequency feature information and the low-frequency feature information, where the initial weights are generated through a weight model.

[0143] The mapping unit 3045 is used to perform a secondary mapping on the initial weights through the restoration module to obtain fusion weights.

[0144] The calculation unit 3046 is used to calculate the denoised image features according to the fusion weights and the image features.

[0145] In one embodiment of the present application, the extraction module 302 includes:

[0146] A downsampling unit 3021, configured to reduce the resolution of infrared images at each scale through a sampling module.

[0147] An extraction unit 3022, configured to extract features from the infrared images at each scale with reduced resolution through a restoration module to obtain feature information at each scale.

[0148] In one embodiment of the present application, the restoration module 305 includes:

[0149] An upsampling unit 3051, configured to expand the scale of a low-resolution infrared image through a sampling module.

[0150] A reduction unit 3052, configured to restore image features through a restoration module to obtain expanded feature information.

[0151] An addition unit 3053, configured to add the expanded feature information to the infrared image with the expanded scale through an identification module to obtain a high-resolution infrared image.

[0152] The device provided in this embodiment can be used to execute the technical solutions of the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0153] Embodiment Seven

[0154] Figure 4 It is a schematic diagram of the hardware structure of a computer device provided in an embodiment of the present application. As Figure 4 shown, the computer device includes: at least one processor 401 and a memory 402; the memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that at least one processor executes the above infrared image processing method.

[0155] Optionally, the memory 402 can be either independent or integrated with the processor 401.

[0156] When the memory 402 is independently provided, the computer device further includes a bus 403, configured to connect the memory 402 and the processor 401.

[0157] Embodiment Eight

[0158] An embodiment of the present application further provides a computer storage medium, in which computer-executable instructions are stored, and when a processor executes the computer-executable instructions, the above infrared image processing method is implemented.

[0159] Embodiment Nine

[0160] The embodiment of the present application also provides a computer program product, including a computer program, which when executed by a processor, implements the infrared image processing method as described above.

[0161] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned module division is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be in electrical, mechanical or other forms.

[0162] The modules described above as separate components may or may not be physically separated. The components displayed as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to implement the solution of this embodiment.

[0163] In addition, each functional module in various embodiments of the present application can be integrated in a processing unit, or each module can exist physically alone, or two or more modules can be integrated in a unit. The unit formed by the above modules can be implemented in the form of hardware, or in the form of a hardware plus software functional unit.

[0164] The integrated module implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above software functional module is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods in various embodiments of the present application.

[0165] It should be understood that the above-mentioned processor can be a central processing unit (Central Processing Unit, abbreviated as CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, abbreviated as DSP), application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0166] The memory may include high-speed RAM memory and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a portable hard drive, a read-only memory, a magnetic disk, or an optical disc, etc.

[0167] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the accompanying drawings of this application are not limited to only one bus or one type of bus.

[0168] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0169] An exemplary storage medium is coupled to the processor, enabling the processor to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a master control device.

[0170] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: ROM, RAM, magnetic disks, or optical discs and other media that can store program codes.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An infrared image processing method, characterized in that, Applied to a computer device deployed with an image processing system, where the image processing system has multiple recognition modules, sampling modules, restoration modules, and fusion modules; the method includes: Obtain a low-resolution infrared image, and divide the scale types of the low-resolution infrared image through the recognition module to obtain infrared images of each scale; Extract features from the infrared images of each scale to obtain feature information of each scale; Perform graphic transformation on the feature information through the fusion module to obtain high-frequency feature information and low-frequency feature information; Fuse the high-frequency feature information and the low-frequency feature information to obtain image features; Restore the image features through the sampling module to obtain a high-resolution infrared image.

2. The method according to claim 1, characterized in that, The fusing the high-frequency feature information and the low-frequency feature information to obtain image features includes: Generate refined high-frequency feature information according to the high-frequency feature information through the restoration module; Perform graphic transformation on the refined high-frequency feature information and the low-frequency feature information through the fusion module to obtain refined feature information; Fuse the refined feature information through the fusion module to generate image features.

3. The method according to claim 2, characterized in that, After generating the image features by fusing the refined feature information through the fusion module, it further includes: Obtain the initial weights of the high-frequency feature information and the low-frequency feature information, where the initial weights are generated by a weight model; Perform a secondary mapping on the initial weights through the restoration module to obtain fusion weights; Calculate the denoised image features according to the fusion weights and the image features.

4. The method according to claim 1, characterized in that, The extracting features from the infrared images of each scale to obtain feature information of each scale includes: Reduce the resolution of the infrared images of each scale through the sampling module; Extract features from the infrared images of each scale with reduced resolution through the restoration module to obtain feature information of each scale.

5. The method according to any one of claims 1 to 4, characterized in that, The restoring the image features through the sampling module to obtain a high-resolution infrared image includes: Enlarge the scale of the low-resolution infrared image through the sampling module; Restore the image features through the restoration module to obtain enlarged feature information; Add the enlarged feature information to the infrared image with the enlarged scale through the recognition module to obtain a high-resolution infrared image.

6. An infrared image processing device, characterized in that, Applied to a computer device deployed with an image processing system, where the image processing system has multiple recognition modules, sampling modules, restoration modules, and fusion modules; the device includes: An acquisition module, configured to acquire a low-resolution infrared image, and divide the scale types of the low-resolution infrared image through the recognition module to obtain infrared images of each scale; An extraction module, configured to extract features from the infrared images of each scale to obtain feature information of each scale; A graphic transformation module, configured to perform graphic transformation on the feature information through the fusion module to obtain high-frequency feature information and low-frequency feature information; A fusion module, configured to fuse the high-frequency feature information and the low-frequency feature information to obtain image features; A restoration module for restoring the image features through the sampling module to obtain a high-resolution infrared image.

7. The device according to claim 6, characterized in that, The fusion module includes: A first generation unit for generating refined high-frequency feature information through the restoration module according to the high-frequency feature information. A graphic transformation unit for performing graphic transformation on the refined high-frequency feature information and the low-frequency feature information through the fusion module to obtain refined feature information. A second generation unit for generating image features by fusing the refined feature information through the fusion module.

8. The device according to claim 7, characterized in that, The fusion module further includes: An acquisition unit for acquiring the initial weights of the high-frequency feature information and the low-frequency feature information, where the initial weights are generated by a weight model. A mapping unit for performing a secondary mapping on the initial weights through the restoration module to obtain fusion weights. A calculation unit for calculating the denoised image features according to the fusion weights and the image features.

9. A computer device, characterized in that, It includes: At least one processor and a memory; The memory stores computer execution instructions; The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the infrared image processing method according to any one of claims 1 to 5.

10. A computer storage medium, characterized in that, Computer execution instructions are stored in the computer storage medium, and when the processor executes the computer execution instructions, the infrared image processing method according to any one of claims 1 to 5 is implemented.

Images