Image processing method and related device

Abstract

The embodiment of the invention provides an image processing method and a related device, and the method comprises the steps: dividing a to-be-processed image into a plurality of sub-images, carrying out the up-sampling of each sub-image based on the image information of an adjacent sub-image of the sub-image and the image information of the sub-image, and carrying out the up-sampling of the sub-image, generating an ultrahigh resolution image corresponding to each sub-image; and based on the relative position information among the plurality of sub-images, splicing the ultra-high resolution images corresponding to the plurality of sub-images to generate an ultra-high resolution image corresponding to the to-be-processed image. According to the embodiment of the invention, in the process of generating the ultrahigh-resolution image corresponding to each sub-image based on the image information of the adjacent sub-images of the sub-images and the image information of the sub-images, the video memory overhead can be adaptively adjusted, and meanwhile, the global dependency relationship is processed; the situation that the resolution of the generated ultra-high-resolution image is limited due to overlarge video memory overhead is avoided, and the resolution of the ultra-high-resolution image is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and related devices. Background Art

[0002] Ultra-high-resolution images refer to images with a number of pixels far exceeding the resolution of ordinary images. They have the characteristics of rich details, accurate colors, and realistic visual effects, and play an important role in the field of target detection.

[0003] Currently, ultra-high-resolution images can be obtained by improving the resolution of low-pixel images, but the resolution of the generated ultra-high-resolution images is usually limited to 1024×1024 pixels or less, which is difficult to meet the application requirements that require higher-resolution images.

[0004] Therefore, a solution is urgently needed to solve the above technical problems. Summary of the invention

[0005] Based on the above problems, the present application provides an image processing method and related devices.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In the first aspect, the present application provides an image processing method, including:

[0008] Dividing the image to be processed into a plurality of sub-images, wherein the plurality of sub-images do not overlap each other;

[0009] For each of the sub-images, up-sampling the sub-image based on image information of neighboring sub-images of the sub-image and image information of the sub-image, to generate an ultra-high resolution image corresponding to each sub-image;

[0010] Based on the relative position information between the multiple sub-images, the ultra-high resolution images corresponding to the multiple sub-images are spliced ​​to generate an ultra-high resolution image corresponding to the image to be processed.

[0011] In a possible implementation manner, for each of the sub-images, upsampling the sub-image based on image information of adjacent sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image includes:

[0012] For each of the sub-images, performing information interaction between image information of a sub-image adjacent to the sub-image and image information of the sub-image to obtain image information after interaction;

[0013] Based on the interacted image information and the image information of the sub-images, the sub-images are up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

[0014] In a possible implementation manner, for each of the sub-images, upsampling the sub-image based on image information of adjacent sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image includes:

[0015] For each of the sub-images, based on the relative position encodings and hidden state information of neighboring sub-images of the sub-image, and the relative position encodings and hidden state information of the sub-image, determine a query vector, a key vector, and a value vector of the sub-image;

[0016] Based on the query vector, the key vector, and the value vector of each sub-image, each sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

[0017] In a possible implementation manner, for each of the sub-images, upsampling the sub-image based on image information of adjacent sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image includes:

[0018] According to the generation order of sub-images, for each sub-image, based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, the sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image, and the generation order of the sub-images includes generating them from left to right and from top to bottom in sequence according to the position of each sub-image in the image to be processed.

[0019] In a possible implementation manner, dividing the image to be processed into a plurality of sub-images includes:

[0020] Performing contrast enhancement processing on the image to be processed to obtain a contrast-enhanced image to be processed;

[0021] The contrast-enhanced image to be processed is divided into a plurality of sub-images.

[0022] In one possible implementation, the adjacent sub-images of the sub-image include at least one of a sub-image to the upper left of the sub-image, a sub-image above the sub-image, a sub-image to the upper right of the sub-image, a sub-image to the left of the sub-image, a sub-image to the right of the sub-image, a sub-image to the lower left of the sub-image, a sub-image below the sub-image, and a sub-image to the lower right of the sub-image.

[0023] A second aspect: The embodiment of the present application provides an image processing device, including:

[0024] A division unit, an upsampling unit, and a splicing unit;

[0025] The division unit is used to divide the image to be processed into a plurality of sub-images, wherein the plurality of sub-images do not overlap with each other;

[0026] The up-sampling unit is configured to up-sample each of the sub-images based on image information of neighboring sub-images of the sub-image and image information of the sub-image, so as to generate an ultra-high resolution image corresponding to each sub-image;

[0027] The stitching unit is used to stitch the ultra-high resolution images corresponding to the multiple sub-images based on the relative position information between the multiple sub-images to generate an ultra-high resolution image corresponding to the image to be processed.

[0028] In a possible implementation manner, the up-sampling unit is specifically configured to:

[0029] For each of the sub-images, performing information interaction between image information of a sub-image adjacent to the sub-image and image information of the sub-image to obtain image information after interaction;

[0030] Based on the interacted image information and the image information of the sub-images, the sub-images are up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

[0031] In a possible implementation manner, the up-sampling unit is specifically configured to:

[0032] For each of the sub-images, based on the relative position encodings and hidden state information of neighboring sub-images of the sub-image, and the relative position encodings and hidden state information of the sub-image, determine a query vector, a key vector, and a value vector of the sub-image;

[0033] Based on the query vector, the key vector, and the value vector of each sub-image, each sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

[0034] In a possible implementation manner, the up-sampling unit is specifically configured to:

[0035] According to the generation order of sub-images, for each sub-image, based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, the sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image, and the generation order of the sub-images includes generating them from left to right and from top to bottom in sequence according to the position of each sub-image in the image to be processed.

[0036] In a possible implementation, the division unit is specifically configured to:

[0037] Performing contrast enhancement processing on the image to be processed to obtain a contrast-enhanced image to be processed;

[0038] The contrast-enhanced image to be processed is divided into a plurality of sub-images.

[0039] A third aspect: An embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory;

[0040] The memory is used to store program code and transmit the program code to the processor;

[0041] The processor is used to execute the steps of an image processing method as described in the first aspect according to the instructions in the program code.

[0042] Fourth aspect: An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of an image processing method as described in the first aspect above are implemented.

[0043] Fifth aspect: An embodiment of the present application provides a computer program product, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps of an image processing method as described in the first aspect above are implemented.

[0044] Compared with the prior art, this application has the following beneficial effects:

[0045] The embodiment of the present application provides an image processing method and related device, which divides an image to be processed into multiple sub-images, and for each of the sub-images, based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, upsamples the sub-image to generate an ultra-high resolution image corresponding to each sub-image; based on the relative position information between the multiple sub-images, the ultra-high resolution images corresponding to the multiple sub-images are spliced ​​to generate an ultra-high resolution image corresponding to the image to be processed. In the embodiment of the present application, in the process of generating an ultra-high resolution image corresponding to each sub-image based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, the video memory overhead can be adaptively adjusted, and the global dependency relationship is processed at the same time, so as to avoid limiting the resolution of the generated ultra-high resolution image due to excessive video memory overhead, and effectively improve the resolution of the ultra-high resolution image. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of dividing an image to be processed provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of a sub-image and adjacent sub-images provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of a sub-image and adjacent sub-images provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a network structure for image processing provided in an embodiment of the present application;

[0052] Figure 6 A structural schematic diagram of an image processing device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] Currently, ultra-high-resolution images can be obtained by increasing the resolution of low-pixel images. However, since the video memory requirement increases dramatically with the increase in resolution, the resolution of the generated ultra-high-resolution images is usually limited to 1024×1024 pixels or less, which is difficult to meet the application requirements that require higher-resolution images.

[0055] For example, in application scenarios such as industrial panel defect detection, ultra-high resolution images with a resolution of up to 5120×5120 pixels are required, but the current method that can generate ultra-high resolution images of 1024×1024 pixels is difficult to meet its application requirements.

[0056] Based on this, an embodiment of the present application provides an image processing method, which divides an image to be processed into multiple sub-images, and for each of the sub-images, based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, upsamples the sub-image to generate an ultra-high resolution image corresponding to each sub-image; based on the relative position information between the multiple sub-images, the ultra-high resolution images corresponding to the multiple sub-images are spliced ​​to generate an ultra-high resolution image corresponding to the image to be processed. In the embodiment of the present application, in the process of generating an ultra-high resolution image corresponding to each sub-image based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, the video memory overhead can be adaptively adjusted, and the global dependency relationship is processed at the same time, so as to avoid limiting the resolution of the generated ultra-high resolution image due to excessive video memory overhead, and effectively improve the resolution of the ultra-high resolution image.

[0057] Combine the following Figure 1 An image processing method provided in an embodiment of the present application is introduced.

[0058] like Figure 1 As shown, the figure is a flowchart of an image processing method provided in an embodiment of the present application, including S101-S103.

[0059] S101: Divide an image to be processed into multiple sub-images.

[0060] The image to be processed may be a common image with a relatively low resolution.

[0061] like Figure 2 As shown in FIG. 1 , the figure is a schematic diagram of dividing an image to be processed provided by an embodiment of the present application. In the embodiment of the present application, the image to be processed 1001 can be divided into a plurality of non-overlapping sub-images 1002, each of which can be referred to as a "block". Figure 2 The image 1002 includes 5×5 sub-images 1002.

[0062] After the image to be processed is divided into a plurality of non-overlapping blocks, each block may be further divided into a plurality of sub-blocks, so that each block or sub-block may be processed independently without considering the entire image to be processed, thereby simplifying the image processing process.

[0063] In a possible implementation, in order to obtain an ultra-high resolution image with a higher resolution, the image to be processed may be subjected to contrast enhancement processing to obtain a contrast-enhanced image to be processed. The contrast-enhanced image to be processed may be divided into a plurality of sub-images.

[0064] Exemplarily, in the embodiments of the present application, the image to be processed can be subjected to histogram-based contrast enhancement processing, such as histogram equalization (HE) processing, adaptive histogram equalization (AHE) processing, and contrast-limited adaptive histogram equalization (CLAHE) processing, to improve the contrast of the image to be processed and obtain a contrast-enhanced image to be processed.

[0065] Among them, the histogram-based contrast enhancement processing is a method of improving the contrast of the image by adjusting the grayscale histogram of the image to make the image clearer. It can be understood that in the embodiments of the present application, the contrast enhancement processing method is not specifically limited, and the above only introduces the histogram-based contrast enhancement processing method as an example.

[0066] In the embodiment of the present application, the contrast of the multiple sub-images obtained by dividing the contrast-enhanced image to be processed is enhanced, so that a higher-resolution ultra-high-resolution image can be subsequently generated based on the contrast-enhanced sub-images.

[0067] S102 . For each of the sub-images, up-sample the sub-image based on image information of neighboring sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image.

[0068] Among them, Figure 3 As shown, the figure is a schematic diagram of a sub-image and adjacent sub-images provided in an embodiment of the present application, and the adjacent sub-images of the sub-image 1300 may include a sub-image 1301 at the upper left of the sub-image, a sub-image 1302 above the sub-image, a sub-image 1303 at the upper right of the sub-image, a sub-image 1304 to the left of the sub-image, a sub-image 1305 to the right of the sub-image, a sub-image 1306 at the lower left of the sub-image, a sub-image 1307 below the sub-image, and at least one of a sub-image 1308 at the lower right of the sub-image.

[0069] In a possible implementation, for each of the sub-images, image information of the sub-image's neighboring sub-images and the image information of the sub-image may be interacted with to obtain interacted image information; based on the interacted image information and the image information of the sub-image, the sub-images may be upsampled to generate an ultra-high-resolution image corresponding to each sub-image.

[0070] In the embodiment of the present application, by dividing the low-resolution image to be processed into multiple sub-images, during the first-layer upsampling process, each sub-image can not only process its own image information, but also interact with the surrounding adjacent sub-images through the cross-attention mechanism. In this way, each sub-image can better consider and integrate the visual information of the surrounding area in the process of generating the corresponding high-resolution image, thereby improving the coherence and accuracy of local details.

[0071] In one possible implementation, in an embodiment of the present application, the sub-images can be upsampled in the order in which the sub-images are generated, based on the image information of the sub-image's neighboring sub-images and the image information of the sub-image, to generate an ultra-high-resolution image corresponding to each sub-image.

[0072] The order of generating the sub-images includes generating them from left to right and from top to bottom in sequence according to the position of each sub-image in the image to be processed.

[0073] Exemplarily, the image to be processed can be represented as Among them, B 2 represents the number of blocks, that is, the number of sub-images; H represents the height of the image to be processed; W represents the width of the image to be processed; and C represents the dimension of the channel. Based on the height and width of the image to be processed and the number of sub-images, the size of each sub-image can be determined. That is, the height of the sub-image h = H / B, and the width of the sub-image w = W / B.

[0074] In the embodiment of the present application, multiple sub-images can be sequentially generated from left to right and from top to bottom according to the position of each sub-image in the image to be processed. In the process of generating an ultra-high resolution image corresponding to a sub-image, only the image information of the sub-image and the adjacent sub-images corresponding to the sub-image can be retained in the memory without retaining the image information of all sub-images, which can effectively reduce the video memory overhead and improve the resolution of the generated ultra-high resolution image.

[0075] In a possible implementation, for each of the sub-images, a query vector, a key vector, and a value vector of the sub-image may be determined based on the relative position coding and hidden state information of the adjacent sub-images of the sub-image, and the relative position coding and hidden state information of the sub-image. Based on the query vector, key vector, and value vector of each sub-image, each sub-image may be upsampled to generate an ultra-high resolution image corresponding to each sub-image.

[0076] like Figure 4As shown, this figure is a schematic diagram of a sub-image and adjacent sub-images provided in an embodiment of the present application. In the embodiment of the present application, based on the adjacent block attention mechanism in the Transformer model, based on the relative position encoding, hidden state information of the adjacent sub-images of the sub-image, and the relative position encoding and hidden state information of the sub-image, an ultra-high resolution image corresponding to each sub-image can be generated.

[0077] Among them, the Transformer model is a deep learning model based on the self-attention mechanism, which can process sequence data through the self-attention mechanism. For example, in the self-attention mechanism, the input data can be converted into three vectors: query vector (Query, Q), key vector (Key, K) and value vector (Value, V). By calculating the dot product of the query vector and the key vector, the attention weight of each query vector and other elements can be obtained. Multiplying the attention weight with the corresponding value vector and weighted summing them can get the final output.

[0078] like Figure 5 As shown in FIG. 1 , the figure is a schematic diagram of a network structure of image processing provided in an embodiment of the present application. In the embodiment of the present application, the adjacent block attention mechanism in the Transformer model can be expressed by formulas (1) to (4) as follows:

[0079]

[0080] Where Q represents the query vector; K represents the key vector; V represents the value vector; W Q represents the attention weight corresponding to the query vector; W K represents the attention weight corresponding to the key vector; W V represents the attention weight corresponding to the value vector; P i Relative position encoding at the block level; Represents the hidden state information (hidden state) of the nth layer, the ith row, and the jth column.

[0081] In an embodiment of the present application, a corresponding number of sub-images can be generated each time according to the memory size and stored in a key-value cache (KV-cache), so that in the process of generating an ultra-high-resolution image corresponding to each sub-image, the video memory overhead can be adaptively adjusted, and global dependencies can be processed at the same time to avoid limiting the resolution of the generated ultra-high-resolution image due to excessive video memory overhead, thereby effectively improving the resolution of the ultra-high-resolution image.

[0082] S103 . Based on the relative position information between the multiple sub-images, stitch the ultra-high resolution images corresponding to the multiple sub-images to generate an ultra-high resolution image corresponding to the image to be processed.

[0083] In the embodiment of the present application, during the process of processing the image to be processed, the size and resolution of the sub-images may change, but the layout and relative position relationship between the sub-images remain unchanged. Therefore, after generating ultra-high resolution images corresponding to multiple sub-images, the ultra-high resolution images corresponding to the multiple sub-images can be spliced ​​based on the relative position information between the multiple sub-images to generate an ultra-high resolution image corresponding to the image to be processed.

[0084] For ease of understanding, an image processing method provided in an embodiment of the present application is generally introduced below.

[0085] The embodiment of the present application can generate ultra-high resolution images suitable for panel inspection applications based on the DiT model. The DiT model is a diffusion model combined with the Transformer architecture, which combines the denoising diffusion probability model (DDPM) and the Transformer architecture, and can efficiently capture dependencies in the data and generate high-quality results.

[0086] The embodiment of the present application can improve the contrast of the image to be processed by performing a histogram-based contrast enhancement process on the image to be processed, thereby obtaining a contrast-enhanced image to be processed. The contrast-enhanced image to be processed is divided into a plurality of non-overlapping blocks, and on this basis, each block can be divided into a plurality of sub-blocks.

[0087] After dividing the contrast-enhanced image to be processed, an optimized neighboring block attention mechanism, namely the Unidirectional Nearby Block (NU Block), can be introduced based on the DiT model to adaptively adjust the video memory overhead in the process of generating ultra-high resolution images.

[0088] In the embodiment of the present application, in the process of performing gradual upsampling and generating the corresponding ultra-high resolution image, each block can not only process its own image information, but also interact with the surrounding neighboring blocks through the cross-attention mechanism. In this way, each block can better consider and integrate the visual information of the surrounding area in the process of generating the corresponding high-resolution image, thereby improving the coherence and accuracy of local details.

[0089] In summary, the embodiments of the present application provide an image processing method, which can adaptively adjust the video memory overhead in the process of generating an ultra-high resolution image corresponding to each sub-image based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, and process global dependencies at the same time, thereby avoiding limiting the resolution of the generated ultra-high resolution image due to excessive video memory overhead, and effectively improving the resolution of the ultra-high resolution image.

[0090] At the same time, the method provided in the embodiment of the present application is suitable for upsampling images of various shapes and resolutions, and has broad application prospects in the field of panel detection.

[0091] The present application provides an image processing device, see Figure 6 , this figure is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Its specific implementation method is consistent with the implementation method and the technical effect achieved in the embodiment of the above method, and some contents will not be repeated here.

[0092] An image processing device 1100, comprising:

[0093] A dividing unit 1101, an up-sampling unit 1102 and a splicing unit 1103;

[0094] The division unit 1101 is used to divide the image to be processed into a plurality of sub-images, wherein the plurality of sub-images do not overlap with each other;

[0095] The up-sampling unit 1102 is configured to up-sample each of the sub-images based on image information of neighboring sub-images of the sub-image and image information of the sub-image, so as to generate an ultra-high resolution image corresponding to each sub-image;

[0096] The stitching unit 1103 is used to stitch the ultra-high resolution images corresponding to the multiple sub-images based on the relative position information between the multiple sub-images to generate an ultra-high resolution image corresponding to the image to be processed.

[0097] In a possible implementation manner, the up-sampling unit is specifically configured to:

[0098] For each of the sub-images, performing information interaction between image information of a sub-image adjacent to the sub-image and image information of the sub-image to obtain image information after interaction;

[0099] Based on the interacted image information and the image information of the sub-images, the sub-images are up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

[0100] In a possible implementation manner, the up-sampling unit is specifically configured to:

[0101] For each of the sub-images, based on the relative position encodings and hidden state information of neighboring sub-images of the sub-image, and the relative position encodings and hidden state information of the sub-image, determine a query vector, a key vector, and a value vector of the sub-image;

[0102] Based on the query vector, the key vector, and the value vector of each sub-image, each sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

[0103] In a possible implementation manner, the up-sampling unit is specifically configured to:

[0104] According to the generation order of sub-images, for each sub-image, based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, the sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image, and the generation order of the sub-images includes generating them from left to right and from top to bottom in sequence according to the position of each sub-image in the image to be processed.

[0105] In a possible implementation, the division unit is specifically configured to:

[0106] Performing contrast enhancement processing on the image to be processed to obtain a contrast-enhanced image to be processed;

[0107] The contrast-enhanced image to be processed is divided into a plurality of sub-images.

[0108] In one possible implementation, the adjacent sub-images of the sub-image include at least one of a sub-image to the upper left of the sub-image, a sub-image above the sub-image, a sub-image to the upper right of the sub-image, a sub-image to the left of the sub-image, a sub-image to the right of the sub-image, a sub-image to the lower left of the sub-image, a sub-image below the sub-image, and a sub-image to the lower right of the sub-image.

[0109] In summary, an embodiment of the present application provides an image processing device, which can adaptively adjust the video memory overhead and process global dependencies while generating an ultra-high resolution image corresponding to each sub-image based on image information of adjacent sub-images of the sub-image and image information of the sub-image, thereby avoiding limiting the resolution of the generated ultra-high resolution image due to excessive video memory overhead, and effectively improving the resolution of the ultra-high resolution image.

[0110] At the same time, the device provided in the embodiment of the present application is suitable for upsampling images of various shapes and resolutions, and has broad application prospects in the field of panel detection.

[0111] An embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory;

[0112] The memory is used to store program code and transmit the program code to the processor;

[0113] The processor is used to execute the steps of the image processing method as described above according to the instructions in the program code.

[0114] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of an image processing method as described above are implemented.

[0115] An embodiment of the present application provides a computer program product, wherein a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps of an image processing method as described above are implemented.

[0116] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Dividing the image to be processed into a plurality of sub-images, wherein the plurality of sub-images do not overlap each other; For each of the sub-images, up-sampling the sub-image based on image information of neighboring sub-images of the sub-image and image information of the sub-image, to generate an ultra-high resolution image corresponding to each sub-image; Based on the relative position information between the multiple sub-images, the ultra-high resolution images corresponding to the multiple sub-images are spliced ​​to generate an ultra-high resolution image corresponding to the image to be processed.

2. The method according to claim 1, characterized in that The step of upsampling each of the sub-images based on image information of adjacent sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image includes: For each of the sub-images, performing information interaction between image information of a sub-image adjacent to the sub-image and image information of the sub-image to obtain image information after interaction; Based on the interacted image information and the image information of the sub-images, the sub-images are up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

3. The method according to claim 1, characterized in that The step of upsampling each of the sub-images based on image information of adjacent sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image includes: For each of the sub-images, based on the relative position encodings and hidden state information of neighboring sub-images of the sub-image, and the relative position encodings and hidden state information of the sub-image, determine a query vector, a key vector, and a value vector of the sub-image; Based on the query vector, the key vector, and the value vector of each sub-image, each sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image.

4. The method according to claim 1, characterized in that: The step of upsampling each of the sub-images based on image information of adjacent sub-images of the sub-image and image information of the sub-image to generate an ultra-high resolution image corresponding to each sub-image includes: According to the generation order of sub-images, for each sub-image, based on the image information of the adjacent sub-images of the sub-image and the image information of the sub-image, the sub-image is up-sampled to generate an ultra-high resolution image corresponding to each sub-image, and the generation order of the sub-images includes generating them from left to right and from top to bottom in sequence according to the position of each sub-image in the image to be processed.

5. The method according to any one of claims 1 to 4, characterized in that: The step of dividing the image to be processed into a plurality of sub-images comprises: Performing contrast enhancement processing on the image to be processed to obtain a contrast-enhanced image to be processed; The contrast-enhanced image to be processed is divided into a plurality of sub-images.

6. The method according to any one of claims 1 to 4, characterized in that: The adjacent sub-images of the sub-image include at least one of a sub-image to the upper left of the sub-image, a sub-image above the sub-image, a sub-image to the upper right of the sub-image, a sub-image to the left of the sub-image, a sub-image to the right of the sub-image, a sub-image to the lower left of the sub-image, a sub-image below the sub-image, and a sub-image to the lower right of the sub-image.

7. An image processing device, characterized in that: include: A division unit, an upsampling unit, and a splicing unit; The division unit is used to divide the image to be processed into a plurality of sub-images, wherein the plurality of sub-images do not overlap with each other; The up-sampling unit is configured to up-sample each of the sub-images based on image information of neighboring sub-images of the sub-image and image information of the sub-image, so as to generate an ultra-high resolution image corresponding to each sub-image; The stitching unit is used to stitch the ultra-high resolution images corresponding to the multiple sub-images based on the relative position information between the multiple sub-images to generate an ultra-high resolution image corresponding to the image to be processed.

8. A computer device, characterized in that: The computer device comprises: a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of an image processing method as described in any one of claims 1 to 6 according to the instructions in the program code.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of an image processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of an image processing method according to any one of claims 1 to 6 are implemented.

Images