Multifold image super-division method and system, storage medium and electronic equipment

Abstract

The invention provides a multi-time image super-division method and system, a storage medium and electronic equipment, and the method comprises the following steps: splitting a low-resolution image based on pixel points, and obtaining each pixel point; splitting the low-resolution image based on pixel blocks to obtain each pixel block; extracting a pixel feature map of the pixel points; extracting a memory matrix of a specified pixel block where the pixel point is located; splicing the pixel feature map and the memory matrix to obtain a spliced matrix; acquiring an image block corresponding to the pixel point; and splicing the image blocks corresponding to the pixel points to obtain a high-resolution image corresponding to the low-resolution image. According to the multi-multiple image super-resolution reconstruction method and system, the storage medium and the electronic equipment, image super-resolution reconstruction of different multiples can be realized based on the pixel blocks and the pixel points, and the application requirements of different scenes are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and particularly relates to a multi-fold image super-resolution method, system, storage medium and electronic device. Background Art

[0002] The super-resolution reconstruction technology of images refers to restoring a given low-resolution image into a corresponding high-resolution image through a specific algorithm. Specifically, the image super-resolution reconstruction technology refers to the process of reconstructing a high-resolution image from a given low-resolution image by using relevant knowledge in the fields of digital image processing, computer vision, etc., through a specific algorithm and processing flow, so as to overcome or compensate for problems such as blurred imaging images, low quality, and insignificant regions of interest caused by the limitations of the image acquisition system or the acquisition environment itself.

[0003] Simply understood, super-resolution reconstruction is to change a small-sized image into a large-sized image, enrich the detailed texture of the image, and make the image more "clear".

[0004] However, the existing image super-resolution reconstruction in the prior art can only achieve super-resolution reconstruction of a specific multiple for a single model, and cannot flexibly achieve super-resolution reconstruction of different multiples. Therefore, for each existing super-resolution reconstruction of a specific multiple, a corresponding model needs to be trained, resulting in low efficiency and high cost. Summary of the Invention

[0005] In view of the above-mentioned disadvantages of the prior art, the purpose of the present invention is to provide a multi-fold image super-resolution method, system, storage medium and electronic device, which can achieve image super-resolution reconstruction of different multiples based on pixel blocks and pixel points, and meet the application requirements of different scenarios.

[0006] First aspect, the present invention provides a multi-fold image super-resolution method, and the method includes the following steps: splitting a low-resolution image based on pixel points to obtain each pixel point; splitting the low-resolution image based on pixel blocks to obtain each pixel block; extracting a pixel feature map of the pixel points; extracting a memory matrix of a specified pixel block where the pixel points are located; splicing the pixel feature map and the memory matrix to obtain a splicing matrix; obtaining an image block corresponding to the pixel point, where the splicing matrix and a super-resolution character are input into a transformer decoder to obtain a first pixel point; inputting the first pixel point and the super-resolution character into the transformer decoder to obtain a second pixel point; until the previously obtained (N*N - 1) pixel points and the super-resolution character are input into the transformer decoder to obtain the N*Nth pixel point; constructing an image block of size N*N based on the obtained N*N pixel points; N represents the super-resolution multiple; splicing the image blocks corresponding to each pixel point to obtain a high-resolution image corresponding to the low-resolution image.

[0007] In an implementation manner of the first aspect, splitting the low-resolution image based on pixel blocks to obtain each pixel block includes the following steps:

[0008] Setting the size of the low-resolution image as (M, L), and the size of the pixel block as (a, b);

[0009] Moving the pixel block from the upper left corner of the low-resolution image, one pixel at a time from left to right and from top to bottom, to obtain (M–a + 1)*(L–b + 1) pixel blocks.

[0010] In an implementation manner of the first aspect, extracting a pixel feature map of the pixel points based on a transformer encoder.

[0011] In an implementation manner of the first aspect, obtaining a memory matrix of the pixel block where the pixel points are located based on a transformer encoder.

[0012] Second aspect, the present invention provides a multi-fold image super-resolution system, and the system includes a first acquisition module, a second acquisition module, a first extraction module, a second extraction module, a first splicing module, a super-resolution module, and a second splicing module;

[0013] The first acquisition module is used to split a low-resolution image based on pixel points to obtain each pixel point;

[0014] The second acquisition module is used to split the low-resolution image based on pixel blocks to obtain each pixel block;

[0015] The first extraction module is used to extract the pixel feature map of the pixel point;

[0016] The second extraction module is used to extract the memory matrix of the specified pixel block where the pixel point is located;

[0017] The first splicing module is used to splice the pixel feature map and the memory matrix to obtain a splicing matrix;

[0018] The super-resolution module is used to obtain the image block corresponding to the pixel point. Among them, the splicing matrix and the super-resolution character are input into the transformer decoder to obtain the first pixel point; the first pixel point and the super-resolution character are input into the transformer decoder to obtain the second pixel point; until the previously obtained (N*N - 1) pixel points and the super-resolution character are input into the transformer decoder to obtain the N*Nth pixel point; an image block of size N*N is constructed based on the obtained N*N pixel points; N represents the super-resolution multiple;

[0019] The second splicing module is used to splice the image blocks corresponding to the respective pixel points to obtain the high-resolution image corresponding to the low-resolution image.

[0020] In one implementation manner of the second aspect, splitting the low-resolution image based on pixel blocks to obtain each pixel block includes the following steps:

[0021] Set the size of the low-resolution image to (M, L), and the size of the pixel block to (a, b);

[0022] The pixel block is moved from the upper left corner of the low-resolution image one pixel at a time from left to right and top to bottom to obtain (M - a + 1)*(L - b + 1) pixel blocks.

[0023] In one implementation manner of the second aspect, the pixel feature map of the pixel point is extracted based on the transformer encoder.

[0024] In one implementation manner of the second aspect, the memory matrix of the pixel block where the pixel point is located is obtained based on the transformer encoder.

[0025] In a third aspect, the present invention provides an electronic device, and the electronic device includes: a processor and a memory;

[0026] The memory is used to store a computer program;

[0027] The processor is used to execute the computer program stored in the memory so that the electronic device executes the above multi-fold image super-resolution method.

[0028] Fourthly, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by an electronic device, the above-mentioned multi-fold image super-resolution method is implemented.

[0029] As described above, the multi-fold image super-resolution method, system, storage medium and electronic device of the present invention have the following beneficial effects:

[0030] (1) It can realize the super-resolution reconstruction of images with different multiples based on pixel blocks and pixel points, meeting the application requirements of different scenarios;

[0031] (2) It solves the problem that a single model cannot handle super-resolution of arbitrary multiples, and has strong practicability;

[0032] (3) It has a high degree of intelligence and greatly improves the user experience. Description of the Drawings

[0033] Figure 1 It shows a schematic diagram of the scenario of the electronic device of the present invention in an embodiment;

[0034] Figure 2 It shows a flowchart of the multi-fold image super-resolution method of the present invention in an embodiment;

[0035] Figure 3 It shows a schematic diagram of the architecture of the multi-fold image super-resolution method of the present invention in an embodiment;

[0036] Figure 4 It shows a schematic diagram of the structure of the multi-fold image super-resolution system of the present invention in an embodiment;

[0037] Figure 5 It shows a schematic diagram of the structure of the electronic device of the present invention in an embodiment. Detailed Embodiments

[0038] The following uses specific specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand the other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0039] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, the diagrams only show the components related to the present invention, rather than being drawn according to the number, shape and size of the components in actual implementation. The types, numbers and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0040] The following embodiments of the present invention provide a multi-fold image super-resolution method, which can be applied to an electronic device as shown in Figure 1 the figure. The electronic device described in the present invention may include a mobile phone 11 with a wireless charging function, a tablet computer 12, a notebook computer 13, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The specific type of the electronic device is not limited in the embodiments of the present invention.

[0041] For example, the electronic device may be a station (STAION, ST) in a WLAN with a wireless charging function, a cellular phone with a wireless charging function, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA) device, a handheld device with a wireless charging function, a computing device or other processing device, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating on a wireless system and a next-generation communication system, for example, a mobile terminal in a 5G network, a mobile terminal in a future evolved public land mobile network (PLMN), or a mobile terminal in a future evolved non-terrestrial network (NTN), etc.

[0042] For example, the electronic device can communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0043] Next, the technical solutions in the embodiments of the present invention will be described in detail with reference to the accompanying drawings in the embodiments of the present invention.

[0044] As Figure 2 and Figure 3 shown, in one embodiment, the multi-fold image super-resolution method of the present invention includes steps S1 to S7.

[0045] Step S1: Split the low-resolution image based on pixel points to obtain each pixel point.

[0046] Specifically, when the low-resolution image is split into pixel points, the corresponding pixel points can be obtained. For example, assume that the size of the low-resolution image is (M, M). After splitting the image into pixel points and arranging them in order (from left to right, from top to bottom), M * M pixel points will be obtained. As Figure 3 shown, for a (4, 4) low-resolution image, it can be split into 16 pixel points, numbered 0, 1, 2... 15 in order from left to right and from top to bottom.

[0047] Step S2: Split the low-resolution image based on pixel blocks to obtain each pixel block.

[0048] Specifically, set the size of the low-resolution image as (M, L), and the size of the pixel block as (a, b). Then, starting from the upper left corner of the low-resolution image, move the pixel block one pixel at a time from left to right and from top to bottom to obtain (M–a + 1)*(L–b + 1) pixel blocks.

[0049] For example, if the size of the low-resolution image is (M, M) and the size of the pixel block is (2, 2), then (M - 1)*(M - 1) pixel blocks can be obtained.

[0050] Step S3: Extract the pixel feature map of the pixel points.

[0051] Specifically, based on the transformer encoder, extract the pixel feature map of the pixel points. Among them, the transformer is a deep learning model architecture used for natural language processing (NLP) and other sequence-to-sequence tasks. The transformer architecture introduces the self-attention mechanism, which is a key innovation and makes it perform excellently in processing sequence data. The transformer usually includes an encoder for processing the input sequence and a decoder for generating the output sequence, which makes it suitable for sequence-to-sequence tasks such as machine translation.

[0052] In the present invention, input the pixel points into the transformer encoder to obtain the corresponding pixel feature map.

[0053] Step S4: Extract the memory matrix of the specified pixel block where the pixel points are located.

[0054] Specifically, each pixel point corresponds to one or more pixel blocks. For example, when the number of the pixel point is included within a certain pixel block, it is considered that the pixel point is within the pixel block.

[0055] In the present invention, when the pixel point corresponds to one pixel block, the pixel block is used as the specified pixel block; when the pixel point corresponds to multiple pixel blocks, select the pixel block with the largest sum of pixel values within the pixel blocks as the specified pixel block. Based on the transformer encoder, obtain the memory matrix of the specified pixel block where the pixel points are located. That is, input the specified pixel block into the transformer encoder to obtain the corresponding memory matrix.

[0056] Step S5: Concatenate the pixel feature map and the memory matrix to obtain a concatenated matrix.

[0057] Specifically, set the shape of the pixel point feature map to (1*1, D), and the shape of the memory matrix to (2*2, D), where D represents the embedding dimension and 2*2 represents the size of the pixel block. Then, the shape of the concatenated matrix is (1*1 + 2*2, D).

[0058] Step S6: Obtain the image block corresponding to the pixel point. Specifically, input the concatenated matrix and the super-resolution character into the transformer decoder to obtain the first pixel point; input the first pixel point and the super-resolution character into the transformer decoder to obtain the second pixel point; until input the previously obtained (N*N - 1) pixel points and the super-resolution character into the transformer decoder to obtain the N*Nth pixel point; construct the image block of size N*N based on the obtained N*N pixel points; N represents the super-resolution multiple defined by the user.

[0059] Specifically, set the super-resolution character to <super resolution>, which is used to indicate the completion of super-resolution reconstruction. For each pixel point, input the concatenated matrix and the super-resolution character into the transformer decoder to obtain the first pixel point; input the first pixel point and the super-resolution character into the transformer decoder to obtain the second pixel point; input the first pixel point, the second pixel point and the super-resolution character into the transformer decoder to obtain the third pixel point. Repeat this process iteratively until input the previously obtained (N*N - 1) pixel points and the super-resolution character into the transformer decoder to obtain the N*Nth pixel point. Reshape the obtained N*N pixel points into a shape of N*N, then the image block of size N*N can be constructed.

[0060] Step S7: Concatenate the image blocks corresponding to each pixel point to obtain the high-resolution image corresponding to the low-resolution image.

[0061] Specifically, after obtaining the image block of each pixel point, concatenate the corresponding image blocks in the arrangement order of the pixel points, then the high-resolution image corresponding to the low-resolution image can be obtained.

[0062] The protection scope of the multi-fold image super-resolution method described in the embodiments of the present invention is not limited to the execution order of the steps listed in this embodiment. Any solution achieved by adding or subtracting steps of the prior art and replacing steps according to the principles of the present invention is included in the protection scope of the present invention.

[0063] An embodiment of the present invention also provides a multi-fold image super-resolution system. The multi-fold image super-resolution system can implement the multi-fold image super-resolution method described in the present invention. However, the implementation device of the multi-fold image super-resolution system described in the present invention includes, but is not limited to, the structure of the multi-fold image super-resolution system listed in this embodiment. Any structural deformation and replacement of the prior art made according to the principle of the present invention are included in the protection scope of the present invention.

[0064] As Figure 4 shown, in one embodiment, the multi-fold image super-resolution system of the present invention includes a first acquisition module 41, a second acquisition module 42, a first extraction module 43, a second extraction module 44, a first splicing module 45, a super-resolution module 46, and a second splicing module 47.

[0065] The first acquisition module 41 is used to split the low-resolution image based on pixel points to obtain each pixel point.

[0066] The second acquisition module 42 is used to split the low-resolution image based on pixel blocks to obtain each pixel block.

[0067] The first extraction module 43 is connected to the first acquisition module 41 and is used to extract the pixel feature map of the pixel points.

[0068] The second extraction module 44 is connected to the second acquisition module 42 and is used to extract the memory matrix of the specified pixel block where the pixel points are located.

[0069] The first splicing module 45 is connected to the first extraction module 43 and the second extraction module 44 and is used to splice the pixel feature map and the memory matrix to obtain a splicing matrix.

[0070] The super-resolution module 46 is connected to the first splicing module 45 and is used to obtain the image block corresponding to the pixel point. Among them, the splicing matrix and the super-resolution character are input into the transformer decoder to obtain the first pixel point; the first pixel point and the super-resolution character are input into the transformer decoder to obtain the second pixel point; until the previously obtained (N*N - 1) pixel points and the super-resolution character are input into the transformer decoder to obtain the N*Nth pixel point; an N*N-sized image block is constructed based on the obtained N*N pixel points; N represents the super-resolution multiple.

[0071] The second splicing module 47 is connected to the super-resolution module 46 and is used to splice the image blocks corresponding to each pixel point to obtain the high-resolution image corresponding to the low-resolution image.

[0072] Among them, the structures and principles of the first acquisition module 41, the second acquisition module 42, the first extraction module 43, the second extraction module 44, the first splicing module 45, the super-resolution module 46, and the second splicing module 47 correspond one by one to the steps in the above multi-fold image super-resolution method, so they will not be elaborated here.

[0073] In several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules / units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of devices or modules or units can be in an electrical, mechanical or other form.

[0074] The modules / units described as separate components may or may not be physically separated. The components displayed as modules / units may or may not be physical modules, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the modules / units can be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, in each embodiment of the present invention, the various functional modules / units can be integrated in a processing module, or each module / unit can exist physically alone, or two or more modules / units can be integrated in one module / unit.

[0075] Those of ordinary skill in the art should also further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but this implementation should not be considered to exceed the scope of the present invention.

[0076] Embodiments of the present invention also provide a computer-readable storage medium. Those of ordinary skill in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing a processor through a program. The program can be stored in a computer-readable storage medium. The storage medium is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrating one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)), etc.

[0077] Embodiments of the present invention also provide an electronic device. The electronic device includes a processor and a memory.

[0078] The memory is used to store a computer program.

[0079] The memory includes various media that can store program codes, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disc.

[0080] The processor is connected to the memory and is used to execute the computer program stored in the memory, so that the electronic device executes the above multi-fold image super-resolution method.

[0081] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it can also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0082] Such as Figure 5As shown, the electronic device of the present invention is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors or processing units 51, a memory 52, and a bus 53 that connects different system components (including the memory 52 and the processing unit 51).

[0083] The bus 53 represents one or more of several types of bus architectures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0084] The electronic device typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0085] The memory 52 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 521 and / or cache memory 522. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 523 may be used for reading and writing on non-removable, non-volatile magnetic media ( Figure 5 not shown, commonly referred to as a "hard disk drive"). Although Figure 5 not shown in the figure, a disk drive for reading and writing on a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing on a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 53 through one or more data media interfaces. The memory 52 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0086] A program / utility 524 having a set (at least one) of program modules 5241 may be stored, for example, in the memory 52. Such program modules 5241 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 5241 generally perform the functions and / or methods described in the embodiments of the present invention.

[0087] The electronic device can also communicate with one or more external devices (such as a keyboard, a pointing device, a display, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device that enables the electronic device to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 54. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 55. As Figure 5 shown, the network adapter 55 communicates with other modules of the electronic device through the bus 53. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0088] The above embodiments are only illustrative of the principles and effects of the present invention, and are not used to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A multi-fold image super-resolution method, characterized in that, The method includes the following steps: Split the low-resolution image based on pixel points to obtain each pixel point; Split the low-resolution image based on pixel blocks to obtain each pixel block; Extract the pixel feature map of the pixel points; Extract the memory matrix of the specified pixel block where the pixel points are located; Concatenate the pixel feature map and the memory matrix to obtain a concatenated matrix; Obtain the image block corresponding to the pixel point, where the concatenated matrix and the super-resolution character are input into the transformer decoder to obtain the first pixel point; the first pixel point and the super-resolution character are input into the transformer decoder to obtain the second pixel point; until the first (N*N - 1) obtained pixel points and the super-resolution character are input into the transformer decoder to obtain the N*Nth pixel point; construct the image block of size N*N based on the obtained N*N pixel points; N represents the super-resolution multiple; Concatenate the image blocks corresponding to each pixel point to obtain the high-resolution image corresponding to the low-resolution image.

2. The multi-fold image super-resolution method according to claim 1, wherein: Splitting the low-resolution image based on pixel blocks to obtain each pixel block includes the following steps: Set the size of the low-resolution image to (M, L), and the size of the pixel block to (a, b); Move the pixel block from the upper left corner of the low-resolution image one pixel at a time from left to right and top to bottom to obtain (M – a + 1)*(L – b + 1) pixel blocks.

3. The multi-fold image super-resolution method according to claim 1, characterized in that: Extract the pixel feature map of the pixel points based on the transformer encoder.

4. The multi-fold image super-resolution method according to claim 1, wherein: Obtain the memory matrix of the pixel block where the pixel points are located based on the transformer encoder.

5. A multi-fold image super-resolution system, characterized in that, The system includes a first acquisition module, a second acquisition module, a first extraction module, a second extraction module, a first concatenation module, a super-resolution module, and a second concatenation module; The first acquisition module is used to split the low-resolution image based on pixel points to obtain each pixel point; The second acquisition module is used to split the low-resolution image based on pixel blocks to obtain each pixel block; The first extraction module is used to extract the pixel feature map of the pixel points; The second extraction module is used to extract the memory matrix of the specified pixel block where the pixel points are located; The first concatenation module is used to concatenate the pixel feature map and the memory matrix to obtain a concatenated matrix; The super-resolution module is used to obtain the image block corresponding to the pixel point, where the concatenated matrix and the super-resolution character are input into the transformer decoder to obtain the first pixel point; the first pixel point and the super-resolution character are input into the transformer decoder to obtain the second pixel point; until the first (N*N - 1) obtained pixel points and the super-resolution character are input into the transformer decoder to obtain the N*Nth pixel point; construct the image block of size N*N based on the obtained N*N pixel points; N represents the super-resolution multiple; The second concatenation module is used to concatenate the image blocks corresponding to each pixel point to obtain the high-resolution image corresponding to the low-resolution image.

6. The multi-fold image super-resolution system according to claim 5, wherein: The steps of splitting the low-resolution image based on pixel blocks and obtaining each pixel block are as follows: Set the size of the low-resolution image to (M, L), and the size of the pixel block to (a, b); Starting from the upper left corner of the low-resolution image, move the pixel block one pixel at a time from left to right and top to bottom to obtain (M–a+1)*(L–b+1) pixel blocks.

7. The multi-fold image super-resolution system according to claim 5, characterized in that: Extract the pixel feature map of the pixel points based on the transformer encoder.

8. The multi-fold image super-resolution system according to claim 5, wherein: Obtain the memory matrix of the pixel block where the pixel points are located based on the transformer encoder.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory is used to store a computer program; The processor is used to execute the computer program stored in the memory, so that the electronic device executes the multi-fold image super-resolution method described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the electronic device, it implements the multi-fold image super-resolution method described in any one of claims 1 to 4.

Images