Image processing method, system, device, storage medium, and program product

Abstract

The present disclosure provides an image processing method, a system, a device, a storage medium, and a program product. The method comprises: determining initial super-resolution image data of an initial image, the initial super-resolution image data being image data after super-resolution processing is performed on the initial image in a channel direction; on the basis of the size of a target image, performing transformation processing on pixel coordinates of each pixel in the initial super-resolution image data to obtain target pixel coordinates of each pixel in the initial super-resolution image data; and generating target image data on the basis of target pixel coordinates of each pixel in the initial super-resolution image data, the target image data being image data of a super-resolution image corresponding to the initial image, and the image size of the super-resolution image being the size of the target image. The image processing efficiency is improved.

Description

[0001] Image Processing Method, System, Device, Storage Medium, and Program Product This disclosure claims priority to Chinese patent application number 202410381709.4, filed with the China Patent Office on March 29, 2024, entitled "Image Processing Method, System, Device, Storage Medium, and Program Product," the entire contents of which are incorporated herein by reference. Technical Field This disclosure relates to the field of image processing technology, and more particularly to an image processing method, system, device, storage medium, and program product. Background: A processor can perform super-resolution processing on an image to improve image quality and clarity. For example, the processor can be a graphics processing unit (GPU), and the super-resolution processing can be artificial intelligence (AI) super-resolution processing. Currently, the process of super-resolution processing on an image by a processor involves a large number of neural network operations. After each neural network operation, the result must be stored in the processor's memory chip. Before the next neural network operation, the result of the previous neural network operation must be read from the processor's memory chip. For example, the processor's memory chip can be high-bandwidth memory (HBM). The large number of read and write operations in this process results in high read and write overhead, which in turn leads to low image processing efficiency. SUMMARY Various aspects of the present disclosure provide an image processing method, system, device, storage medium, and program product to improve image processing efficiency. In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising: determining initial super-resolved image data of an initial image, the initial super-resolved image data being image data after super-resolving the initial image in a channel direction; transforming pixel coordinates of each pixel in the initial super-resolved image data according to a target image size to obtain target pixel coordinates of each pixel in the initial super-resolved image data; and generating target image data based on the target pixel coordinates of each pixel in the initial super-resolved image data, the target image data being image data of a super-resolved image corresponding to the initial image, and the image size of the super-resolved image being the target image size.In a second aspect, an embodiment of the present disclosure provides an image processing method, which is applied to an image processor, and includes: determining initial super-resolution image data of an initial image, where the initial super-resolution image data is image data after super-resolution processing is performed on the initial image in a channel direction, and the initial image is an image to be super-resolved using artificial intelligence; transforming pixel coordinates of each pixel in the initial super-resolution image data according to a target image size to obtain target pixel coordinates of each pixel in the initial super-resolved image data; and generating target image data based on the target pixel coordinates of each pixel in the initial super-resolved image data, where the target image data is image data of a super-resolved image corresponding to the initial image, and the image size of the super-resolved image is the target image size. In a third aspect, an embodiment of the present disclosure provides an image processing device, comprising a determination module, a transformation module, and a generation module, wherein the determination module is configured to determine initial super-resolution image data of an initial image, where the initial super-resolution image data is image data after super-resolution processing of the initial image in a channel direction; the transformation module is configured to transform the pixel coordinates of each pixel in the initial super-resolution image data according to a target image size to obtain target pixel coordinates of each pixel in the initial super-resolution image data; and the generation module is configured to generate target image data according to the target pixel coordinates of each pixel in the initial super-resolution image data, where the target image data is image data of a super-resolution image corresponding to the initial image, and the image size of the super-resolution image is the target image size. In a fourth aspect, an embodiment of the present disclosure provides an image processing device, which is applied to an image processor, and the device includes a determination module, a transformation module, and a generation module, wherein the determination module is used to determine initial super-resolution image data of an initial image, and the intermediate image data is image data after super-resolution processing of the initial image in the channel direction, and the initial image is the image to be super-resolutioned by artificial intelligence; the transformation module is used to transform the pixel coordinates of each pixel in the initial super-resolution image data according to the target image size to obtain the target pixel coordinates of each pixel in the initial super-resolution image data; the generation module is used to generate target image data according to the target pixel coordinates of each pixel in the initial super-resolution image data, and the target image data is image data of the super-resolution image corresponding to the initial image, and the image size of the super-resolution image is the target image size.In a fifth aspect, embodiments of the present disclosure provide an image processing system, comprising an image processor and a memory chip, wherein the image processor is configured to perform the method according to any one of the first aspects to determine image data of a super-resolved image corresponding to an initial image, and store the image data of the super-resolved image in the memory chip. In a sixth aspect, embodiments of the present disclosure provide an image processing system, comprising an image processor and a memory chip, wherein the image processor is configured to perform the method according to any one of the second aspects to determine image data of a super-resolved image corresponding to an initial image, and store the image data of the super-resolved image in the memory chip. In a seventh aspect, embodiments of the present disclosure provide an image processing device, comprising: a memory and a processor; the memory storing computer-executable instructions; the processor executing the computer-executable instructions stored in the memory, causing the image processing device to perform the method according to any one of the first aspects. In an eighth aspect, embodiments of the present disclosure provide an image processing device, comprising: a memory and a processor; the memory storing computer-executable instructions; the processor executing the computer-executable instructions stored in the memory, causing the image processing device to perform the method according to any one of the second aspects. In a ninth aspect, embodiments of the present disclosure provide a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the computer-readable storage medium is used to implement the method according to any one of the first aspects. In a tenth aspect, embodiments of the present disclosure provide a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the computer-executable instructions are used to implement the method according to any one of the second aspects. In an eleventh aspect, embodiments of the present disclosure provide a computer program product, including a computer program. When the computer program is executed by a processor, the computer program is used to implement the method according to any one of the first aspects. In a twelfth aspect, embodiments of the present disclosure provide a computer program product, including a computer program. When the computer program is executed by a processor, the computer program is used to implement the method according to any one of the second aspects. Embodiments of the present disclosure provide an image processing method, system, device, storage medium, and program product. By using the above-mentioned method, initial super-resolution image data of an initial image can be determined; pixel coordinates of each pixel in the initial super-resolution image data can be transformed according to a target image size to obtain target pixel coordinates of each pixel in the initial super-resolution image data; and target image data can be generated based on the target pixel coordinates of each pixel in the initial super-resolution image data. Through the above method, in the process of super-resolution processing of the initial image, there is no need to perform pixel reorganization, which avoids a large number of storage and reading operations, thereby reducing read and write overhead and improving image processing efficiency.BRIEF DESCRIPTION OF THE DRAWINGS The drawings described herein are intended to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are intended to explain the present disclosure and are not intended to unduly limit the present disclosure. In the drawings: Figure 1 is a schematic diagram of a tensor provided in an exemplary embodiment of the present disclosure; Figure 2 is a flowchart of a convolution operation and pixel reassembly operation provided in an exemplary embodiment of the present disclosure; Figure 3 is a schematic diagram of a pixel reassembly provided in an exemplary embodiment of the present disclosure; Figure 4 is a flowchart of an image processing method provided in an exemplary embodiment of the present disclosure; Figure 5 is a flowchart of another image processing method provided in an exemplary embodiment of the present disclosure; Figure 6 is a schematic diagram of an image processing method provided in an exemplary embodiment of the present disclosure; Figure 7 is a schematic diagram of the structure of an image processing apparatus provided in an exemplary embodiment of the present disclosure; Figure 8 is a schematic diagram of the structure of another image processing apparatus provided in an exemplary embodiment of the present disclosure; Figure 9 is a schematic diagram of the structure of an image processing device provided in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS: It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display) referred to in this disclosure are all authorized by the user or fully authorized by all parties. The collection, use, and processing of the relevant data must comply with relevant laws, regulations, and standards, and corresponding operation portals are provided for users to choose to authorize or reject. To further clarify the objectives, technical solutions, and advantages of this disclosure, the technical solutions of this disclosure will be described clearly and completely below in conjunction with specific embodiments of this disclosure and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of this disclosure, and are not exhaustive. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. To facilitate understanding, the technical terms used in the embodiments of this disclosure are first explained.

[0002] 1) Image data storage format There are generally two types of image data storage formats, one is the channel-first storage format (NCHW), and the other is the channel-last storage format (NHWC).

[0003] N can indicate the number of batch images;

[0004] H can indicate the number of pixels in the vertical direction of the image;

[0005] W can indicate the number of pixels in the horizontal direction of the image;

[0006] C indicates the number of channels of an image. For an image, the image data can be stored as a three-dimensional tensor using the two storage formats mentioned above.

[0007] 2) Spatial Coordinates and Linear Offsets of Each Element in a Tensor: The spatial coordinates of an element can indicate the coordinate position of the element in the tensor space. The linear offset of an element can indicate the order of the element in the tensor. In the storage space of a tensor, the elements of the tensor can be stored linearly in ascending order of linear offset. The linear offset of an element can be determined based on the spatial coordinates of the element and the stride array of the tensor to which the element belongs. The stride array can include the stride of the tensor in each dimension. For the stride of any dimension, the stride can indicate the number of elements that need to be skipped to obtain the next element along that dimension in the storage space of the tensor. The stride array can be determined based on the shape of the tensor. For example, assuming the shape of the tensor is [H, W, C], the spatial coordinates of the elements in the tensor in the H dimension are X and C in the W dimension are C. As shown in FIG1 , in order of linear offset from small to large, elements a0, a1, a2, d1, and d2 can be stored in the storage space of tensor A in sequence.

[0008] 3) Al super resolution

[0009] AI super-resolution technology can increase the number of pixels in the vertical and horizontal directions of an image to improve image clarity. Currently, AI super-resolution can be performed on images using a processor. When performing AI super-resolution on an image, the processor generally uses a channel-first storage format. The shape of the three-dimensional tensor corresponding to the image data can be expressed as [H, W, C]. oDuring the AI ​​super-resolution process, a processor performs a large number of convolution operations on an image, and pixel shuffling operations are performed after the convolution operations. The following describes the process of the processor performing convolution operations and pixel shuffling operations on a three-dimensional tensor, taking a three-dimensional tensor of shape [H, W, C] as an example, with reference to FIG2 . FIG2 is a schematic flow diagram of convolution operations and pixel shuffling operations provided by an exemplary embodiment of the present disclosure. The rectangular boxes in FIG2 represent neural network operators, the oval boxes represent storage space, and the dashed boxes represent the pixel shuffling process. Referring to FIG2 , the processor may first perform a convolution operation on the three-dimensional tensor of shape [H, W, C] using a convolution operator to obtain a three-dimensional tensor of shape [H, W, 4C], and then store the three-dimensional tensor of shape [H, W, 4C] in the storage space. Secondly, the processor can read the three-dimensional tensor stored by the convolution operator from the storage space through the deformation operator 1, perform deformation processing on the read three-dimensional tensor, obtain a five-dimensional tensor with a shape of [H, W, 2, 2, C], and store the five-dimensional tensor with a shape of [H, W, 2, 2, C] into the storage space; secondly, the processor can read the five-dimensional tensor stored by the shape deformation operator 1 from the storage space through the transposition operator, perform transposition processing on the read five-dimensional tensor, obtain a five-dimensional tensor with a shape of [2, H, 2, W, C], and store the five-dimensional tensor with a shape of [2, H, 2, C] into the storage space. Next, the processor uses deformation operator 2 to read the five-dimensional tensor stored by the transpose operator from the storage space, deforms the read five-dimensional tensor to obtain a three-dimensional tensor with a shape of [2H, 2W, C], and stores the three-dimensional tensor with a shape of [2H, 2W, C] in the storage space. As shown in Figure 2, the processor first performs a convolution operation to quadruple the number of channels of the initial three-dimensional tensor (a three-dimensional tensor with a shape of [H, W, C]), obtaining an intermediate three-dimensional tensor (a three-dimensional tensor with a shape of [H, W, 4C]). Then, through a pixel reshaping operation, the intermediate three-dimensional tensor is converted into a three-dimensional tensor with a shape of [2H, 2W, C], thereby completing the vertical and horizontal super-resolution of the initial three-dimensional tensor. However, the convolution and pixel reshaping operations described above involve a large number of storage and read operations. In the specific implementation, the processor needs to write data to and read data from the processor's memory chip multiple times. Data reading and writing operations take a long time during the image processing process, resulting in low image processing efficiency.The inventors discovered that the process of pixel reorganization by a processor on a three-dimensional tensor is essentially a process of permuting and transforming the elements of the three-dimensional tensor. The elements in the three-dimensional tensor do not change; only the order of the elements changes. The pixel reorganization process is described below with reference to Figure 3. Figure 3 is a schematic diagram of pixel reorganization provided by an exemplary embodiment of the present disclosure. Assuming H=1, W=1, and C=1, the shape of the three-dimensional tensor before pixel reorganization can be [1, 1, 4], and the shape of the three-dimensional tensor after pixel reorganization can be [2, 2, 1]. The left side of Figure 3 is a schematic diagram of the three-dimensional tensor before pixel reorganization, where each cube represents an element in the three-dimensional tensor. The number of longitudinal elements, the number of transverse elements, and the number of channels in the three-dimensional tensor before pixel reorganization can be 1, 1, and 4. O The right side of Figure 3 is a schematic diagram of a three-dimensional tensor after pixel rebinding. Each cube represents an element in the three-dimensional tensor. The number of vertical elements, horizontal elements, and channels in the three-dimensional tensor after pixel rebinding can be 2, 2, and 1. As shown in Figure 3, the elements in the three-dimensional tensor remain unchanged before and after pixel rebinding; only the order of the elements changes. It should be understood that a change in the order of the elements will cause the spatial coordinates of the elements to change, and thus the linear offset of the elements to change. In view of this, the embodiments of the present disclosure propose the following concept: super-resolved image data (i.e., target image data) can be determined based on the image size of the super-resolved image (i.e., the image size of the image obtained through pixel rebinding in related art) and the pixel coordinates of each pixel in the image data before pixel rebinding (i.e., the initial super-resolved image data). This method avoids pixel rebinding on the image, thereby avoiding a large number of storage and read operations, reducing read and write overhead, and improving image processing efficiency. The technical solutions presented in this disclosure are described in detail below through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other. Identical or similar content will not be described repeatedly in different embodiments. FIG. 4 is a flowchart illustrating an image processing method provided by an exemplary embodiment of the present disclosure. Referring to FIG. 4 , the method may include:

[0010] S401. Determine initial super-resolved image data for an initial image. The execution subject of the disclosed embodiment may be an image processing device, or a processing chip within the image processing device. For example, the processing chip may be a GPU. For ease of understanding, the following description uses the image processing device as an example. The initial image may be an image to be super-resolved. The initial super-resolved image data may be image data obtained by super-resolving the initial image in the channel direction. The number of channels of the image corresponding to the initial super-resolved image data is greater than the number of channels of the initial image. In this embodiment, an initial super-resolved image size corresponding to the initial super-resolved image data may be pre-set, and the initial image may be super-resolved in the channel direction based on the initial super-resolved image size to obtain the initial super-resolved image data. In other words, the number of channels of the initial image may be expanded based on the initial super-resolved image size to obtain the initial super-resolved image data. For example, assuming that the image size of the initial image is [H, W, C], and the initial super-resolved image size is [H, W, 4C], O The number of channels of the initial image can be expanded to 4 times of the original number to obtain the initial super-resolution image data.

[0011] S402. Transform the pixel coordinates of each pixel in the initial super-resolved image data according to the target image size to obtain target pixel coordinates of each pixel in the initial super-resolved image data. The target image size may be the image size of the super-resolved image corresponding to the initial image. In this embodiment, the target image size may be set based on image processing requirements. For example, assuming the image size of the initial image is [H, W, C], and the super-resolved processing is required to double the number of vertical pixels of the initial image and double the number of horizontal pixels of the initial image, the target image size may be [2H, 2W, C]. oThe pixel coordinates of each pixel in the initial super-resolved image data may be the spatial coordinates of each pixel in the tensor corresponding to the initial super-resolved image data. For example, assuming the size of the initial super-resolved image is [H, W, 4C], the initial super-resolved image data may be represented by a three-dimensional tensor of size [H, W, 4C]. For any pixel in the initial super-resolved image data, the pixel coordinates may be the spatial coordinates of the pixel in the three-dimensional tensor. The target pixel coordinates may be the spatial coordinates of each pixel in the three-dimensional tensor corresponding to the super-resolved image of the initial image. In this embodiment, relative to the image size of the initial image, the number of vertical and horizontal elements of the initial super-resolved image size remains unchanged, while the number of channels increases. For example, the image size of the initial image may be [H, W, C], and the initial super-resolved image size may be [H, W, 4C]. O Compared to the image size of the initial image, the target image size has an increased number of vertical elements and a increased number of horizontal elements, while the number of channels remains unchanged. For example, the image size of the initial image can be [H, W, C], and the target image size can be [2H, 2W, C]. o The inventors discovered that the pixels included in the image data corresponding to the target image size are identical to the pixels included in the initial super-resolved image data; however, the arrangement order of the pixels in the image data corresponding to the target image size differs from the arrangement order in the initial super-resolved image data. If the initial super-resolved image size and the target image size are determined, the transformation relationship between the spatial coordinates corresponding to the initial super-resolved image size and the spatial coordinates corresponding to the target image size can be determined based on the initial super-resolved image size and the target image size. In this embodiment, the initial super-resolved image size and the target image size can be set according to image processing requirements, the coordinate transformation relationship can be determined based on the initial super-resolved image size and the target image size, and the pixel coordinates of each pixel in the initial super-resolved image data can be transformed based on the coordinate transformation relationship to obtain the target pixel coordinates of each pixel in the initial super-resolved image data. It should be noted that the specific implementation of determining the coordinate transformation relationship and transforming the pixel coordinates of each pixel in the initial super-resolved image data based on the coordinate transformation relationship can be seen in the embodiment of FIG. 5 and will not be further described here.

[0012] S403: Generate target image data based on the target pixel coordinates of each pixel in the initial super-resolved image data. The target image data is the image data of the super-resolved image corresponding to the initial image. The target pixel coordinates of each pixel in the initial super-resolved image data can be the spatial coordinates of each pixel in the three-dimensional tensor corresponding to the image data of the super-resolved image of the initial image. As can be seen from the embodiment of FIG1 , each element in the tensor can be stored sequentially in a storage space. In this embodiment, the pixels included in the target image data are the same as the pixels included in the initial super-resolved image data. The image processing device can determine the storage order of each pixel based on the target pixel coordinates and target image size of each pixel in the initial super-resolved image data, and can store each pixel in the initial super-resolved image data based on the storage order of each pixel to generate the target image data. It should be noted that the method for determining the storage order of each pixel based on the target pixel coordinates and target image size of each pixel in the initial super-resolved image data can be found in the embodiment of FIG1 and will not be further described here. In the image processing method provided in this embodiment, an image processing device can determine initial super-resolution image data of an initial image; transform the pixel coordinates of each pixel in the initial super-resolution image data according to the target image size to obtain target pixel coordinates for each pixel in the initial super-resolution image data; and generate target image data based on the target pixel coordinates for each pixel in the initial super-resolution image data. This method eliminates the need for pixel reassembly during super-resolution processing of the initial image, avoiding a large number of storage and read operations, thereby reducing read and write overhead and improving image processing efficiency. In another optional embodiment, the present disclosure further provides an image processing method, which is applied to an image processor and includes: determining initial super-resolved image data of an initial image, the initial super-resolved image data being image data after super-resolving the initial image in the channel direction, the initial image being the image to be super-resolved using artificial intelligence; transforming the pixel coordinates of each pixel in the initial super-resolved image data according to a target image size to obtain target pixel coordinates for each pixel in the initial super-resolved image data; and generating target image data based on the target pixel coordinates of each pixel in the initial super-resolved image data, the target image data being image data of a super-resolved image corresponding to the initial image, the image size of the super-resolved image being the target image size. Based on the above embodiment, the image processing method provided in the embodiment of the present disclosure may further determine the one-dimensional coordinates of each pixel in the intermediate image, and may generate the target image data based on the one-dimensional coordinates of each pixel. The image processing method provided in the embodiment of the present disclosure is further described below with reference to FIG. FIG. 5 is a schematic flow chart of another image processing method provided in an exemplary embodiment of the present disclosure. Referring to FIG. 5 , the method may include:

[0013] S501. Determine initial super-resolved image data of an initial image. The execution subject of the embodiment of the present disclosure may be an image processing device, or a processing chip provided in the image processing device. For example, the processing chip may be a GPU. For ease of understanding, the following description uses the image processing device as an example. It should be noted that the specific implementation of S501 can be found in S401 and will not be further described here.

[0014] S502. Determine the initial super-resolved image size corresponding to the initial super-resolved image data. The initial super-resolved image data can be represented by a tensor. The initial super-resolved image size corresponding to the initial super-resolved image data can be the size of the tensor corresponding to the initial super-resolved image data. In this embodiment, the image processing device can perform convolution processing on the initial image to expand the number of channels of the initial image without changing the number of longitudinal elements and the number of transverse elements corresponding to the initial image. Furthermore, the image processing device can perform convolution processing on the initial image to expand the number of channels of the initial image by four times. Specifically, the number of channels corresponding to the initial super-resolved image data can be four times the number of channels corresponding to the initial image, the number of longitudinal elements corresponding to the initial super-resolved image data can be the same as the number of longitudinal elements corresponding to the initial image, and the number of transverse elements corresponding to the initial super-resolved image data can be the same as the number of transverse elements corresponding to the initial image. For example, assuming that the initial image size corresponding to the initial image is [H, W, C], the initial super-resolved image size corresponding to the initial super-resolved image data can be [H, W, 4C]. O

[0015] S503. Determine a coordinate transformation relationship based on the initial super-resolution image size and the target image size. The target image size may be the image size of the super-resolution image corresponding to the initial image. It should be noted that, as shown in the embodiment of FIG2 , the pixel reorganization process included in the image super-resolution process may be a process of changing the arrangement order of elements in a tensor and changing the size of the tensor. As shown in FIG2 , after pixel reorganization of the tensor, the number of channels can be reduced to one-fourth of the original number, the number of vertical elements can be doubled, and the number of horizontal elements can also be doubled. If the size of the image before pixel reorganization is [H, W, 4C], the size of the image after pixel reorganization can be [2H, 2W, C]. o In this embodiment, pixel reorganization may not be performed during the image super-resolution process, but the relevant Specifically, the coordinate transformation relationship may include: i = (r / (2C))*H+p, j = (r% (2C) / C)*W+q, k = r% (2C)% (C)

[0016] S504. For any pixel in the initial super-resolved image data, determine the target pixel coordinates of the pixel based on the pixel coordinates in the initial super-resolved image data and the coordinate transformation relationship. In this embodiment, the target vertical coordinate can be determined based on the vertical coordinate in the initial super-resolved pixel coordinates and the vertical coordinate transformation relationship; the target horizontal coordinate can be determined based on the horizontal coordinate in the initial super-resolved pixel coordinates and the horizontal coordinate transformation relationship; and the target channel coordinate can be determined based on the channel coordinates in the initial super-resolved pixel coordinates and the channel coordinate transformation relationship. The determined target pixel coordinates include the target vertical coordinate, the target horizontal coordinate, and the target channel coordinate. Specifically, the target pixel coordinates of each pixel can be determined based on the 3D pixel coordinates of each pixel corresponding to the initial super-resolved image data and the coordinate transformation relationship in S503. For example, assuming H=1, W=1, C=1, then pe[0, 1), qe[0, 1), re [0, 4) : ie[0, 2), je [0, 2), ke [o, 1), where, if the three-dimensional pixel coordinates corresponding to the initial super-resolution image size are (0,0,0), then the target pixel coordinates may be (0,0,0); if the three-dimensional pixel coordinates corresponding to the initial super-resolution image size are (0,0,1), then the target pixel coordinates may be (0,1,0); if the three-dimensional pixel coordinates corresponding to the initial super-resolution image size are (0,0,2), then the target pixel coordinates may be (1,0,0); if the three-dimensional pixel coordinates corresponding to the initial super-resolution image size are (0,0,3), then the target pixel coordinates may be (1,1,0). Sorting is performed to obtain target image data. In this embodiment, the target image data may also be stored in a preset storage space. Specifically, each pixel in the initial super-resolved image may be stored in the preset storage space in ascending order of one-dimensional coordinates. Optionally, in this embodiment, a super-resolved image may be generated based on the target image data. In the image processing method provided in this embodiment, the image processing device may determine initial super-resolved image data of the initial image; may determine an initial super-resolved image size corresponding to the initial super-resolved image data; may determine a coordinate transformation relationship based on the initial super-resolved image size and the target image size; for any pixel in the initial super-resolved image data, may determine the target pixel coordinates of the pixel based on the pixel coordinates in the initial super-resolved image data and the coordinate transformation relationship; may determine the one-dimensional coordinates of each pixel in the initial super-resolved image based on the target pixel coordinates of each pixel in the initial super-resolved image; and may generate the target image data based on the pixel values ​​of each pixel in the initial super-resolved image data based on the one-dimensional coordinates of each pixel in the initial super-resolved image. The above method eliminates the need for pixel reorganization during image super-resolution processing, avoiding a large number of storage and read operations, thereby reducing read and write overhead and improving image processing efficiency. Based on any of the above embodiments, the image processing method provided by the present disclosure is described below with reference to FIG6 , using a specific example. Assuming H=2, W=2, and C=1, the initial super-resolution image size can be [2, 2, 4], and the target image size can be [4, 4, 1]. oAssuming that the three-dimensional pixel coordinates corresponding to the initial super-resolution image size can be represented by (p, q, r), the three-dimensional pixel coordinates corresponding to the target image size can be represented by (i, j, k). The vertical coordinate transformation relationship can be: i = (r / 2) *2+p; the horizontal coordinate transformation relationship can be: j= (r% (2) / 1) *2+q; the channel coordinate transformation relationship can be: k=r% (2) % (1); wherein, / is the floor operator after division, and % is the remainder operator after division. In addition, the step array corresponding to the target image size can be (4, 4, 1), and the coordinate mapping relationship can be: One-dimensional coordinate of pixel = i*4 +j +ko Figure 6 is a schematic diagram of image processing provided by an exemplary embodiment of the present disclosure. Referring to Figure 6, the tensor corresponding to the initial super-resolution image can include 4 channels, and each channel can include a 2*2 matrix. Channel 0 may include elements a0, b0, c0, and d0, channel 1 may include elements a1, b1, d, and d1, channel 2 may include elements a2, b2, c2, and d2, and channel 3 may include elements a3, b3, c3, and d3. The spatial coordinates of each pixel in the initial super-resolution image data may be as shown in Table 3: Table 3 As shown in Table 5, the order of the pixels in ascending order of one-dimensional coordinates is: a0, b0, a1, b1, c0, d0, c1, d1, a2, b2, a3, b3, c2, d2, c3, and d3. Referring to FIG6 , the target image data may be as shown in FIG6 . FIG7 is a schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present disclosure. Referring to FIG. 7 , the image processing device 10 includes a determination module 11, a transformation module 12, and a generation module 13. The determination module 11 is configured to determine initial super-resolved image data of an initial image, the initial super-resolved image data being image data obtained by super-resolving the initial image in the channel direction. The transformation module 12 is configured to transform the pixel coordinates of each pixel in the initial super-resolved image data according to a target image size to obtain target pixel coordinates for each pixel in the initial super-resolved image data. The generation module 12 is configured to generate target image data based on the target pixel coordinates of each pixel in the initial super-resolved image data, the target image data being image data of a super-resolved image corresponding to the initial image, the image size of the super-resolved image being the target image size. The image processing device provided in the present embodiment can implement the technical solutions shown in the above-described method embodiments. The implementation principles and beneficial effects thereof are similar and are not further described here. In one possible implementation, the transformation module 12 is specifically configured to determine an initial super-resolved image size corresponding to the initial super-resolved image data; determine a coordinate transformation relationship based on the initial super-resolved image size and the target image size, the coordinate transformation relationship being used to convert 3D pixel coordinates corresponding to the initial super-resolved image size into 3D pixel coordinates corresponding to the target image size; and determine, for any pixel in the initial super-resolved image data, a target pixel coordinate based on the pixel coordinate in the initial super-resolved image data and the coordinate transformation relationship. In one possible implementation, the coordinate transformation relationship includes a ordinate transformation relationship, a lateral transformation relationship, and a channel coordinate transformation relationship. The transformation module 12 is specifically configured to determine a target ordinate based on the ordinate in the initial super-resolved pixel coordinates and the ordinate transformation relationship; determine a target lateral coordinate based on the lateral coordinate in the initial super-resolved pixel coordinates and the lateral transformation relationship; and determine a target channel coordinate based on the channel coordinates in the initial super-resolved pixel coordinates and the channel coordinate transformation relationship. The determination of the target pixel coordinates includes the target ordinate, the target lateral, and the target channel coordinates.In one possible embodiment, the generation module 13 is specifically configured to determine the one-dimensional coordinates of each pixel in the initial super-resolved image based on the target pixel coordinates of each pixel in the initial super-resolved image; and generate the target image data based on the pixel values ​​of each pixel in the initial super-resolved image data based on the one-dimensional coordinates of each pixel in the initial super-resolved image data. In one possible embodiment, for any pixel in the initial super-resolved image, the generation module 13 is specifically configured to determine a coordinate mapping relationship, where the coordinate mapping relationship is used to map three-dimensional coordinates to one-dimensional coordinates; and determine the one-dimensional coordinates of the pixel based on the target pixel coordinates of the pixel and the coordinate mapping relationship. In one possible embodiment, the generation module 13 is specifically configured to sort the pixels in the initial super-resolved image in ascending order of one-dimensional coordinates to obtain the target image data. In one possible embodiment, the determination module 11 is specifically configured to perform convolution processing on the initial image to obtain the initial super-resolved image data. Figure 8 is a schematic structural diagram of another image processing device provided by an exemplary embodiment of the present disclosure. Referring to FIG8 , based on FIG7 , the image processing device 10 further includes a storage module 14, which is configured to store the target image data in a preset storage space. In one possible implementation, the generation module 13 is further configured to generate the super-resolved image based on the target image data. The image processing device provided in the embodiments of the present disclosure can implement the technical solutions described in the above-mentioned method embodiments. The implementation principles and beneficial effects are similar and will not be further described here. FIG9 is a schematic structural diagram of an image processing device provided in an exemplary embodiment of the present disclosure. Referring to FIG9 , the image processing device 20 may include a processor 21 and a memory 22. Exemplarily, the processor 21 and the memory 22 are interconnected via a bus 23. The memory 22 stores computer-executable instructions; the processor 21 executes the computer-executable instructions stored in the memory 22, causing the processor 21 to perform the method described in the above-mentioned method embodiments. Accordingly, an embodiment of the present disclosure provides an image processing system, comprising an image processor and a memory chip, wherein the image processor is configured to execute the method described in the above method embodiment to determine image data of a super-resolved image corresponding to an initial image, and store the image data of the super-resolved image in the memory chip.Accordingly, embodiments of the present disclosure provide a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions are used to implement the methods described in the above method embodiments. Accordingly, embodiments of the present disclosure may also provide a computer program product, including a computer program. When executed by a processor, the computer program may implement the methods described in the above method embodiments. Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code. The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, may be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, such that the instructions, when executed by the processor of the computer or other programmable data processing device, produce means for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram. These computer program instructions can also be stored in a computer-readable memory capable of directing the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram. These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, such that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.Memory may include non-persistent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM. O Memory is an example of computer-readable media. Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can implement information storage using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmitting medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves. It should also be noted that the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, product, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, an element specified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, product, or apparatus comprising the recited element. The foregoing description is merely an example of the present disclosure and is not intended to limit the present disclosure. Various modifications and variations of the present disclosure will be apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, and the like made within the spirit and principles of the present disclosure are intended to be encompassed by the claims of the present disclosure.

Claims

Claims 1. An image processing method, wherein: include: Initial super-resolved image data of an initial image is determined, where the initial super-resolved image data is image data obtained by super-resolving the initial image in a channel direction; pixel coordinates of each pixel in the initial super-resolved image data are transformed according to a target image size to obtain target pixel coordinates of each pixel in the initial super-resolved image data; and target image data is generated based on the target pixel coordinates of each pixel in the initial super-resolved image data, where the target image data is image data of a super-resolved image corresponding to the initial image, and the image size of the super-resolved image is the target image size.

2. An image processing method, wherein: The method is applied to an image processor and includes: determining initial super-resolution image data of an initial image, the initial super-resolution image data being image data after super-resolution processing is performed on the initial image in a channel direction, and the initial image being an image to be super-resolutioned using artificial intelligence; transforming pixel coordinates of each pixel in the initial super-resolution image data according to a target image size to obtain target pixel coordinates of each pixel in the initial super-resolution image data; and generating target image data according to the target pixel coordinates of each pixel in the initial super-resolution image data, the target image data being image data of a super-resolution image corresponding to the initial image, and the image size of the super-resolution image being the target image size.

3. The method according to claim 1 or 2, wherein: Transforming the pixel coordinates of each pixel in the initial super-resolved image data according to the target image size to obtain the target pixel coordinates of each pixel in the initial super-resolved image data includes: determining an initial super-resolved image size corresponding to the initial super-resolved image data; determining a coordinate transformation relationship according to the initial super-resolved image size and the target image size, the coordinate transformation relationship being used to convert the three-dimensional pixel coordinates corresponding to the initial super-resolved image size into three-dimensional pixel coordinates corresponding to the target image size; and determining, for any pixel in the initial super-resolved image data, the target pixel coordinates of the pixel according to the pixel coordinates of the pixel in the initial super-resolved image data and the coordinate transformation relationship.

4. The method according to claim 3, wherein: The coordinate transformation relationship includes a vertical coordinate transformation relationship, a horizontal coordinate transformation relationship, and a channel coordinate transformation relationship; determining the target pixel coordinates of the pixel according to the pixel coordinates of the pixel in the initial super-resolution image data and the coordinate transformation relationship, including: determining the target vertical coordinate according to the vertical coordinate in the initial super-resolution pixel coordinates and the vertical coordinate transformation relationship; determining the target horizontal coordinate according to the horizontal coordinate in the initial super-resolution pixel coordinates and the horizontal coordinate transformation relationship; determining the target channel coordinates according to the channel coordinates in the initial super-resolution pixel coordinates and the channel coordinate transformation relationship; determining the target pixel coordinates including the target vertical coordinate, the target horizontal coordinate, and the target channel coordinates 5. The method according to any one of claims 1 to 4, wherein: Generating target image data according to target pixel coordinates of each pixel in the initial super-resolved image data includes: determining one-dimensional coordinates of each pixel in the initial super-resolved image according to the target pixel coordinates of each pixel in the initial super-resolved image; and generating target image data according to the target pixel coordinates of each pixel in the initial super-resolved image data based on the target pixel coordinates of each pixel in the initial super-resolved image data. The pixel value of the pixel is used to generate the target image data.

6. The method according to claim 5, wherein: For any pixel in the initial super-resolved image; Determining the one-dimensional coordinates of the pixel according to the target pixel coordinates of the pixel in the initial super-resolution image includes: determining a coordinate mapping relationship, where the coordinate mapping relationship is used to map the three-dimensional coordinates to the one-dimensional coordinates; Determine the one-dimensional coordinates of the pixel according to the target pixel coordinates of the pixel and the coordinate mapping relationship.

7. The method according to claim 5 or 6, wherein: Generating the target image data based on the one-dimensional coordinates of each pixel in the initial super-resolved image and the pixel value of each pixel in the initial super-resolved image data includes: sorting the pixels in the initial super-resolved image in ascending order of the one-dimensional coordinates to obtain the target image data.

8. The method according to any one of claims 1 to 7, wherein: Determining initial super-resolution image data of an initial image includes: performing convolution processing on the initial image to obtain the initial super-resolution image data.

9. The method according to any one of claims 1 to 8, wherein: The method further includes: storing the target image data in a preset storage space.

10. The method according to any one of claims 1 to 9, wherein: The method further includes: generating the super-resolution image according to the target image data.

11. An image processing system, wherein: The image processing system includes an image processor and a memory chip, wherein the image processor is used to execute the method of any one of claims 1 to 10 to determine image data of a super-resolved image corresponding to an initial image, and store the image data of the super-resolved image in the memory chip.

12. An image processing device, wherein: include: memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the image processing device performs the method described in any one of claims 1 to 0.

13. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product, comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images