Image Upsampling Device, Method, Equipment and Storage Medium

Abstract

This application relates to the field of image processing technology, and provides an image upsampling device, method, equipment, and storage medium. The device includes: a first reading module, a second reading module, a sorting module, and an interpolation module; the first reading module is used to read the first row of the original image; the first row includes multiple first original pixel values; the second reading module is used to read the second row of the original image; the second row includes multiple second original pixel values; the sorting module is used to arrange a set of values according to a preset rule to obtain a pixel value queue; each first original pixel value and a second original pixel value are in the same column of the original image; the interpolation module is used to perform shifting and addition calculations on the pixel value queue to generate a target pixel value; the target pixel value is a part of the target image. The technical solution of this application does not require the use of SRAM, reducing the hardware area.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to an image upsampling device, method, equipment, and storage medium. Background Art

[0002] Upsampling refers to enlarging an image. The most common upsampling is with a sampling ratio of 2, that is, doubling both the width and height of the image. Usually, through an interpolation algorithm, new pixel values are interpolated based on the original pixel values of the original image.

[0003] However, related technologies usually use SRAM (Static Random Access Memory) to cache the original pixel values of the current row and perform interpolation operations after obtaining the original pixel values of the next row. Using SRAM to cache the original pixel values of the current row will greatly increase the hardware area. Summary of the Invention

[0004] Embodiments of this application provide an image upsampling device, method, equipment, and storage medium that do not require the use of SRAM, reducing the hardware area. The technical solution is as follows:

[0005] According to a first aspect of the embodiments of this application, an image upsampling device is provided. The device includes:

[0006] A first reading module, a second reading module, a sorting module, and an interpolation module;

[0007] The first reading module is configured to read the first row of the original image; the first row includes multiple first original pixel values;

[0008] The second reading module is configured to read the second row of the original image; the second row includes multiple second original pixel values;

[0009] The sorting module is configured to arrange a set of values according to a preset rule to obtain a pixel value queue; the set of values includes n first original pixel values of the first row and n second original pixel values obtained from the second row; the positions of the n first original pixel values are consecutive; the positions of the n second original pixel values are consecutive; each first original pixel value and a second original pixel value are in the same column of the original image; n is an integer greater than or equal to 1;

[0010] The interpolation module is configured to perform shift and addition calculations on the pixel value queue to generate a target pixel value; the target pixel value is a part of the target image.

[0011] In a possible implementation, the pixel value queue sequentially includes a first pixel value, a second pixel value, a third pixel value, and a fourth pixel value; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

[0012] In a possible implementation, the first reading module is further configured to read the first row of the original image;

[0013] The second reading module is further configured to read the first row of the original image.

[0014] In a possible implementation, the original image includes p rows; the first reading module is further configured to read the p-th row of the original image;

[0015] The second reading module is further configured to read the p-th row of the original image; p is an integer greater than or equal to 1.

[0016] In a possible implementation, the set of values includes two identical sets of the k-th first original pixel value and the (k + 1)-th first original pixel value in the first row; the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value; or the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value; k is an integer greater than or equal to 1.

[0017] In a possible implementation, the first reading module is further configured to read first garbage data; the first garbage data is located after the m-th first original pixel value in the first row; the first row includes m first original pixel values; m is an integer greater than or equal to k + 3;

[0018] The second reading module is further configured to read second garbage data; the second garbage data is located after the m-th second original pixel value in the second row; the second row includes m second original pixel values.

[0019] In a possible implementation, the device further includes a replacement module; the replacement module is configured to replace the first garbage data with a plurality of first preset pixel values to obtain a first intermediate row; and replace the second garbage data with a plurality of second preset pixel values to obtain a second intermediate row.

[0020] In a possible implementation, the sorting module is further configured to:

[0021] Supplement a preset number of third preset pixel values before the first first original pixel value in the first intermediate row to obtain a first filled row; supplement the preset number of fourth preset pixel values before the first second original pixel value in the second intermediate row to obtain a second filled row.

[0022] In a possible implementation, the sorting module is further configured to:

[0023] Arrange the set of values according to the preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value of the first filled row; and the k-th second original pixel value and the (k + 1)-th second original pixel value of the second filled row; the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value; or the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value.

[0024] Arrange the set of values according to the preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value of the first filled row; and the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value of the second filled row; the pixel value queue is the (k + 2)-th first original pixel value, the (k + 3)-th first original pixel value, the (k + 2)-th second original pixel value, and the (k + 3)-th second original pixel value; or the pixel value queue is the (k + 3)-th first original pixel value, the (k + 2)-th first original pixel value, the (k + 3)-th second original pixel value, and the (k + 2)-th second original pixel value.

[0025] In a possible implementation, the sorting module is further configured to:

[0026] Arrange the set of values according to the preset rule to obtain a queue of the pixel values; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value of the first padding row; and the k-th second original pixel value and the (k + 1)-th second original pixel value of the second padding row; the queue of pixel values is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value; or the queue of pixel values is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value;

[0027] Arrange the set of values according to the preset rule to obtain the next queue of the pixel values; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value of the first padding row; and the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value of the second padding row;

[0028] The queue of pixel values is the (k + 2)-th second original pixel value, the (k + 3)-th second original pixel value, the (k + 2)-th first original pixel value, and the (k + 3)-th first original pixel value; or the queue of pixel values is the (k + 3)-th second original pixel value, the (k + 2)-th second original pixel value, the (k + 3)-th first original pixel value, and the (k + 2)-th first original pixel value.

[0029] In a possible implementation, the interpolation module is further configured to:

[0030] Generate a first target row; the first target row includes a plurality of the target pixel values;

[0031] Generate a second target row; the second target row includes a plurality of the target pixel values;

[0032] The first target row and the second target row are part of the target image.

[0033] In a possible implementation, the interpolation module is further configured to:

[0034] Shift the first pixel value to obtain a first shift result; shift the third pixel value to obtain a second shift result;

[0035] Perform an addition calculation on the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result;

[0036] Shift the first addition result to obtain a third shift result;

[0037] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain one of the target pixel values; the target pixel value is a part of the first target row.

[0038] In a possible implementation, generating the first target row includes:

[0039] When the pixel value queue is the k-th first original pixel value, the k + 1-th first original pixel value, the k-th first original pixel value, and the k + 1-th first original pixel value,

[0040] Shift the k-th first original pixel value to obtain a first shift result; shift the k-th first original pixel value to obtain a second shift result;

[0041] Perform an addition calculation on the first shift result, the k-th first original pixel value, and the k + 1-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th first original pixel value, and the k + 1-th first original pixel value to obtain a second addition result;

[0042] Shift the first addition result to obtain a third shift result;

[0043] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain one of the target pixel values; the target pixel value is a part of the first target row.

[0044] In a possible implementation, generating the second target row includes:

[0045] When the pixel value queue is the k-th first original pixel value, the k + 1-th first original pixel value, the k-th second original pixel value, and the k + 1-th second original pixel value, shift the k-th first original pixel value to obtain a first shift result; shift the k-th second original pixel value to obtain a second shift result;

[0046] Perform an addition calculation on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a second addition result;

[0047] Shift the first addition result to obtain a third shift result;

[0048] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain one of the target pixel values; the target pixel value is part of the second target row.

[0049] In a possible implementation, generating the second target row further includes:

[0050] When the pixel queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value, shift the (k + 1)-th first original pixel value to obtain a first shift result; shift the (k + 1)-th second original pixel value to obtain a second shift result;

[0051] Perform an addition calculation on the first shift result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a second addition result;

[0052] Shift the first addition result to obtain a third shift result;

[0053] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain one of the target pixel values.

[0054] In a possible implementation, generating the second target row further includes:

[0055] When the pixel queue is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value, shift the k-th second original pixel value to obtain a first shift result; shift the k-th first original pixel value to obtain a second shift result;

[0056] Perform an addition calculation on the first shifted result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shifted result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result;

[0057] Shift the first addition result to obtain a third shifted result;

[0058] Perform an addition calculation on the third shifted result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0059] In a possible implementation manner, generating the second target row further includes:

[0060] When the pixel queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value, shift the (k + 1)-th second original pixel value to obtain a first shifted result; shift the (k + 1)-th first original pixel value to obtain a second shifted result;

[0061] Perform an addition calculation on the first shifted result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shifted result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a second addition result;

[0062] Shift the first addition result to obtain a third shifted result;

[0063] Perform an addition calculation on the third shifted result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0064] According to a second aspect of the embodiments of the present application, an upsampling method is provided. The method includes:

[0065] Read the first row of the original image; the first row includes a plurality of first original pixel values;

[0066] Read the second row of the original image; the second row includes a plurality of second original pixel values;

[0067] Arrange the set of values according to a preset rule to obtain a pixel value queue; the set of values includes the n first original pixel values of the first row and the n second original pixel values obtained from the second row; the positions of the n first original pixel values are consecutive; the positions of the n second original pixel values are consecutive; each of the first original pixel values and one of the second original pixel values are in the same column of the original image; n is an integer greater than or equal to 1.

[0068] Perform a shift and addition calculation on the pixel value queue to generate a target pixel value; the target pixel value is a part of the target image.

[0069] According to the third aspect of the embodiments of the present application, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store at least one segment of program, and the at least one segment of program is loaded and executed by the processor to perform the upsampling method.

[0070] According to the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores at least one segment of program, and the at least one segment of program is loaded and executed by a processor to implement the upsampling method.

[0071] In the embodiments of the present application, an image upsampling device is provided. The original image is read by two reading modules, and SRAM is not required, saving hardware area. In addition, the original pixel values read by the reading modules are sorted by a sorting module to output a pixel value queue, so that the interpolation module can perform fixed calculations on each pixel value queue. Moreover, no multiplication calculation is required, so a multiplier is not needed, saving hardware area and simplifying the calculation process, thereby saving hardware computing resources. Description of the Drawings

[0072] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0073] Figure 1 is a schematic diagram of an implementation environment provided according to the embodiments of the present application;

[0074] Figure 2 is a schematic diagram of the structure of an image upsampling device provided according to the embodiments of the present application;

[0075] Figure 3 is a schematic diagram of the comparison between a target image and an original image provided according to the embodiments of the present application;

[0076] Figure 4 is the first positional relationship between a target pixel and four surrounding original pixels provided by an embodiment of the present application;

[0077] Figure 5 is the second positional relationship between a target pixel and four surrounding original pixels provided by an embodiment of the present application;

[0078] Figure 6 is the third positional relationship between a target pixel and four surrounding original pixels provided by an embodiment of the present application;

[0079] Figure 7 is the fourth positional relationship between a target pixel and four surrounding original pixels provided by an embodiment of the present application;

[0080] Figure 8 is a schematic flowchart of an upsampling method provided by an embodiment of the present application;

[0081] Figure 9 is a schematic structural diagram of a terminal provided by an embodiment of the present application;

[0082] Figure 10 is a schematic structural diagram of a server provided by an embodiment of the present application. Detailed implementation manners

[0083] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0084] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application.

[0085] In the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor are the quantity and execution order limited. It should also be understood that although the following description uses terms such as first and second to describe various elements, these elements should not be limited by the terms.

[0086] These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, the first action can be referred to as the second action, and similarly, the second action can also be referred to as the first action. Both the first action and the second action can be actions, and in some cases, they can be separate and different actions.

[0087] Among them, "at least one" means one or more than one. For example, at least one action can be one action, two actions, three actions, etc., any integer greater than or equal to one. And "a plurality of" means two or more than two. For example, a plurality of actions can be two actions, three actions, etc., any integer greater than or equal to two.

[0088] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application. This implementation environment may include a terminal 101 and a server 102.

[0089] In the terminal 101, an image upsampling device is provided; the image upsampling device is electrically connected to a DDR (Double Data Rate SDRAM, double data rate synchronous dynamic random access memory) to facilitate the image upsampling device to collect image data.

[0090] The terminal 101 can be a smart phone with an image upsampling device, a wearable device, a personal computer, a laptop computer, a tablet computer, a smart TV, a vehicle-mounted terminal, etc.

[0091] The server 102 can be a single server, a server cluster composed of multiple servers, or, alternatively, a cloud processing center.

[0092] The terminal 101 is connected to the server 102 through a wired or wireless network.

[0093] In some embodiments, a wireless network or a wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network. In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some of the links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0094] In the related art, usually through an interpolation algorithm, new pixel values are interpolated based on the original pixel values of the original image, so as to enlarge the original image. The following are two common interpolation algorithms:

[0095] Nearest Neighbor Interpolation is a simple interpolation method. Its principle is to determine the new pixel value according to the known pixel value closest to the target pixel position. In short, it selects the known pixel point closest to the target point and uses its pixel value as the pixel value of the new pixel.

[0096] Bilinear Interpolation is more complex. It takes into account the four nearest pixel points around the target pixel point and uses the distances and relative positions between these pixel points to perform a linear weighted average calculation. By interpolating these weights, bilinear interpolation can provide a smoother and more continuous image result, helping to reduce jagged edges and distortion.

[0097] However, in the process of upsampling in related technologies, SRAM is usually used to cache the original pixel values of the current row, and interpolation operations are performed after obtaining the original pixel values of the next row. This results in an excessive circuit area; moreover, a full-functional multiplier is required for the calculation part, which also leads to an excessive area. In addition, when interpolating, the original image is expanded by one circle and then interpolation is performed. By doing so, it is usually necessary to expand the original image by one circle in advance, which will increase the processing time of the digital circuit.

[0098] To solve the above technical problems, an embodiment of the present application provides an image upsampling device.

[0099] Figure 2 FIG. 200 is a schematic structural diagram of an image upsampling device 200 provided according to an embodiment of the present application. The device includes:

[0100] A first reading module 201, a second reading module 202, a sorting module 203, and an interpolation module 204.

[0101] In some embodiments, the first reading module 201 and the second reading module 202 are respectively electrically connected to a DDR. The DDR is used to store the original image. The first reading module 201 and the second reading module 202 respectively read the original pixel values in the original image from the DDR. For example: both the first reading module 201 and the second reading module 202 are DMA modules (Direct Memory Access). The original image includes p rows and m columns. Wherein, p is an integer greater than or equal to 1; m is an integer greater than or equal to k + 3.

[0102] In an embodiment of the present application, any one of the bicubic interpolation algorithm, the nearest neighbor interpolation algorithm, the bilinear interpolation algorithm, etc. can be selected when performing interpolation. Taking the bilinear interpolation algorithm as an example below, the upsampling process of the present application will be exemplarily described.

[0103] In some embodiments, for the convenience of hardware calculation, the sorting module sorts four original pixel values in a group of values to obtain a pixel value queue. Optionally, the pixel value queue sequentially includes a first pixel value, a second pixel value, a third pixel value, and a fourth pixel value; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

[0104] Figure 3 FIG. 223 is a schematic diagram of comparing a target image and an original image provided according to an embodiment of the present application.

[0105] The following combines Figure 3An exemplary description of the process of reading the original image is given.

[0106] It can be understood that the original image includes a plurality of original pixel points. Each original pixel point corresponds to an original pixel value. The target image includes a plurality of target pixel points. Each target pixel point corresponds to a target pixel value. When the sampling magnification is 2, the width and height of the target image are twice the width and height of the original image.

[0107] In some embodiments, since the first target row of the target image needs to be generated based on the first row of the original image and supplementary pixel values, supplementary pixel values are required. To solve this problem, a first reading module is used to read the first row of the original image. A second reading module is also used to read the first row of the original image. The first row of the original image read by the second reading module is used as the supplementary pixel value. Thus, the first target row of the target image can be generated based on the first row of the original image and the second row of the original image. For example: The target image includes 2p rows and 2m columns.

[0108] In some embodiments, for the second target row of the target image, the first reading module is used to read the first row of the original image; the first row includes a plurality of first original pixel values. The second reading module is used to read the second row of the original image; the second row includes a plurality of second original pixel values.

[0109] In one example, since the original image needs to be read multiple times to generate the target image. The above-mentioned first row can be understood as the previous row, and the second row can be understood as the next row, that is, the reading step needs to be executed cyclically multiple times. Optionally, for the (s + 1)-th target row of the target image, the first reading module is used to read the s-th row of the original image; the s-th row includes a plurality of first original pixel values. The second reading module is used to read the (s + 1)-th row of the original image; the (s + 1)-th row includes a plurality of second original pixel values. s is an integer greater than or equal to 1 and less than (p - 1). That is, in the case where no supplementary pixel values are required, the first reading module and the second reading module read the original image simultaneously, and the second reading module and the first reading module read two adjacent rows respectively.

[0110] In some embodiments, for the last target row of the target image, that is, the 2p-th target row, it needs to be generated based on the last row (the p-th row) of the original image and supplementary pixel values, so supplementary pixel values are also required. To solve this problem, in the embodiments of the present application, the original image includes p rows; the first reading module is also used to read the p-th row of the original image. The second reading module is also used to read the p-th row of the original image; p is an integer greater than or equal to 1. The last row of the original image read by the second reading module is used as the supplementary pixel value. Thus, the 2p-th target row can be generated based on the last row of the original image and the supplementary pixel value.

[0111] In one example, since the bus bit width is 256 bits, that is, the first reading module and the second reading module read 256 bits per clock cycle. Each first original pixel value and second original pixel value includes two-channel data, and the size of each channel data is 16 bits, so the size of each first original pixel value and second original pixel value is 32 bits. When the size of the first original pixel values included in each row of the original image exceeds 256 bits, in order to read 256 bits per clock cycle, the first reading module and the second reading module need to read garbage data. Specifically, the first reading module is further configured to read first garbage data; the first garbage data is located after the m-th first original pixel value in the first row; the first row includes m first original pixel values. The second reading module is further configured to read second garbage data; the second garbage data is located after the m-th second original pixel value in the second row; the second row includes m second original pixel values. Among them, the specific values of the first garbage data and the second garbage data can be configured according to actual needs. For example, taking the first reading module reading the first row as an example, assuming the original image includes 3 rows and 9 columns, then the size of the first original pixel values included in the first row is 288 bits. Then the first reading module needs two clock cycles to finish reading the first row. However, the first reading module can read 512 bits in two clock cycles. Therefore, the first reading module needs to read 224 bits of first garbage data. Similarly, the second reading module needs to read 224 bits of second garbage data.

[0112] In some embodiments, the apparatus further includes a replacement module; the replacement module is configured to replace the first garbage data with a plurality of first preset pixel values to obtain a first intermediate row; and replace the second garbage data with a plurality of second preset pixel values to obtain a second intermediate row. For example, the first preset pixel value is the m-th first original pixel value in the first row. The second preset pixel value is the m-th second original pixel value in the second row. Similarly, the first preset pixel value corresponding to the s-th row is the m-th first original pixel value in the s-th row. The second preset pixel value corresponding to the s + 1-th row is the m-th second original pixel value in the s + 1-th row.

[0113] In some embodiments, a sorting module is configured to arrange a set of values according to a preset rule to obtain a pixel value queue for a set of values. The set of values includes n first original pixel values in the first row and n second original pixel values in the second row. The positions of the n first original pixel values are consecutive. The positions of the n second original pixel values are consecutive. Each first original pixel value and a second original pixel value are in the same column of the original image. n is an integer greater than or equal to 1. For example, when using the bilinear interpolation algorithm for interpolation, n is 2. It should be understood that the preset rule indicates the arrangement order of the n first original pixel values and the n second original pixel values. "The positions of the first original pixel values are consecutive" can be understood as: the positions of the original pixel points corresponding to the first original pixel values in the original image are consecutive.

[0114] In the first example, to generate the first target column of the target image, at least one column of pixel values needs to be supplemented before the first column of the original image. To solve this problem, in the embodiments of the present application, the sorting module is further configured to: supplement a preset number of third preset pixel values before the first first original pixel value in the first intermediate row to obtain a first filled row; supplement a preset number of fourth preset pixel values before the first second original pixel value in the second intermediate row to obtain a second filled row, so as to facilitate the generation of the first two target pixel values of the first target column of the target image. Optionally, the first target pixel value of the first target column is generated based on the first first original pixel value in the first intermediate row, the third preset pixel values, the second second original pixel value, and the fourth preset pixel values. Among them, the preset pixel values supplemented in different filled rows may be different. Optionally, for the convenience of hardware processing, the preset number is determined according to the bus bit width. For example, 256 bits of third preset pixel values are supplemented before the first first original pixel value in the first intermediate row. That is, 8 third preset pixel values need to be supplemented. Similarly, 256 bits of fourth preset pixel values are supplemented before the first second original pixel value in the second intermediate row. That is, 8 fourth preset pixel values need to be supplemented. Optionally, the third preset pixel value is the first first original pixel value. The fourth preset pixel value is the first second original pixel value. That is, the embodiments of the present application can copy the last column and the first column of the original image, and there is no need to expand the original image by one circle in advance, shortening the processing time of the digital circuit.

[0115] In the second example, the sorting module is further configured to: arrange a set of values according to a preset rule to obtain a pixel value queue; the set of values includes two identical sets of the k-th first original pixel value and the (k + 1)-th first original pixel value in the first row, where k is an integer greater than or equal to 1. It should be noted that when the first reading module reads the first row and the second reading module also reads the first row, the pixel queue values include two cases. The first case is: the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value. The second case is: the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value. Among them, the second case can be understood as swapping the order of the k-th first original pixel value and the (k + 1)-th first original pixel value.

[0116] Optionally, the sorting module outputs a pixel value queue within the same clock cycle.

[0117] Figures 4 to 7 It is several position relationships between a target pixel point and four surrounding original pixel points provided by the embodiments of the present application.

[0118] The following combines Figures 4 to 7 to exemplarily illustrate the principle of the preset rule.

[0119] The reason why the embodiments of the present application need to sort the four original pixel values in a set of values is that there are four position relationships between the target pixel value and the four surrounding original pixel values. According to the bilinear interpolation algorithm, different position relationships correspond to different calculation processes. In the embodiments of the present application, in order to reduce the calculation of the interpolation module, by arranging the order of the four original pixel values, the four pixel values in any pixel value queue are subjected to the same calculation.

[0120] For example: Figure 4 In, the target pixel point is closest to the k-th first original pixel point. Figure 5 In, the target pixel point is closest to the (k + 1)-th first original pixel point. Figure 6 In, the target pixel point is closest to the k-th second original pixel point. Figure 7 In, the target pixel point is closest to the (k + 1)-th second original pixel point.

[0121] In the third example, the sorting module is further configured to: arrange a set of values according to a preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value of the first filled row; and the k-th second original pixel value and the (k + 1)-th second original pixel value of the second filled row. In this case, the sorting module inputs two pixel value queues in sequence. The first pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value. The second pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value.

[0122] In the fourth example, the sorting module is further configured to: arrange a set of values according to a preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value of the first filled row; and the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value of the second filled row. In this case, the sorting module outputs two pixel value queues in sequence. The first pixel value queue is the (k + 2)-th first original pixel value, the (k + 3)-th first original pixel value, the (k + 2)-th second original pixel value, and the (k + 3)-th second original pixel value. The second pixel value queue is the (k + 3)-th first original pixel value, the (k + 2)-th first original pixel value, the (k + 3)-th second original pixel value, and the (k + 2)-th second original pixel value.

[0123] In the fifth example, the sorting module is further configured to: arrange a set of values according to a preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value of the first filled row; and the k-th second original pixel value and the (k + 1)-th second original pixel value of the second filled row. In this case, the sorting module outputs two pixel value queues in sequence. The first pixel value queue is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value. The second pixel value queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value.

[0124] In the sixth example, the sorting module is further configured to: arrange a set of values according to a preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value of the first padding row; and the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value of the second padding row. In this case, the sorting module outputs two pixel value queues in sequence. The first pixel value queue is the (k + 2)-th second original pixel value, the (k + 3)-th second original pixel value, the (k + 2)-th first original pixel value, and the (k + 3)-th first original pixel value. The second pixel value queue is the (k + 3)-th second original pixel value, the (k + 2)-th second original pixel value, the (k + 3)-th first original pixel value, and the (k + 2)-th first original pixel value.

[0125] In some embodiments, the interpolation module is configured to perform a shift and an addition calculation on the pixel value queue to obtain a target pixel value; the target pixel value is a part of the target image.

[0126] Exemplarily, the interpolation module is further configured to: generate a first target row; the first target row includes multiple target pixel values. Generate a second target row; the second target row includes multiple target pixel values. The first target row and the second target row are parts of the target image.

[0127] That is, after the interpolation module completes the generation of the first target row, it then generates the second target row.

[0128] In some embodiments, the interpolation module is further configured to:

[0129] Shift the first pixel value to obtain a first shift result; shift the third pixel value to obtain a second shift result;

[0130] Perform an addition calculation on the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result;

[0131] Shift the first addition result to obtain a third shift result;

[0132] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value; the target pixel value is a part of the first target row.

[0133] It should be understood that the bilinear interpolation algorithm calculates the pixel value queue based on preset weights. For example: the weight of the first pixel value is a, and the weight of the second pixel value is (1 - a); the weight of the third pixel value is b, and the weight of the fourth pixel value is (1 - b). The weights of the first pixel value and the second pixel value are c, and the weights of the third pixel value and the fourth pixel value are (1 - c), where a, b, and c are numbers greater than 0 and less than 1.

[0134] That is to say, a target pixel value can be calculated through the following formula:

[0135] Target pixel value = c × (first pixel value × a + second pixel value × (1 - a)) + (1 - c) × (third pixel value × b + fourth pixel value × (1 - b)).

[0136] When c = 0.75; a = 0.75; 1 - a = 0.25; b = 0.75; 1 - b = 0.25; 1 - c = 0.25.

[0137] Then the target pixel value = 1 / 16 × (3 × (3 × first pixel value + second pixel value) + (3 × third pixel value + fourth pixel value)).

[0138] However, the above algorithm requires the use of a multiplier, thus consuming a relatively large hardware area. To solve this problem, the embodiments of the present application convert the above calculation method into shift and addition calculations, so that multiplication calculations are not required, and thus the multiplier is not needed, thereby saving the hardware area.

[0139] For example: in the embodiments of the present application, the target pixel value can be calculated through the following formula:

[0140] Since the target pixel value = 1 / 16 × (3 × (3 × first pixel value + second pixel value) + (3 × third pixel value + fourth pixel value)).

[0141] Then the target pixel value = 1 / 16 × ((3 × (first pixel value << 1 + first pixel value + second pixel value) + (third pixel value << 1 + third pixel value + fourth pixel value)).

[0142] Target pixel value = (first pixel value << 1 + first pixel value + second pixel value) << 1 + (first pixel value << 1 + (first pixel value << 1 + first pixel value + second pixel value) + (third pixel value << 1 + third pixel value + fourth pixel value) >> 4. Wherein, "<< " represents left shift; ">> " represents right shift.

[0143] It can be understood that the first pixel value is a binary number. "The first pixel value << 1 + the first pixel value + the second pixel value" means that the first pixel value is shifted left by one bit, and then an addition calculation is performed with the first pixel value and the second pixel value. It should be noted that after rounding the result of the right shift calculation, the target pixel value is obtained.

[0144] In the first example, a first target row is generated, including:

[0145] When the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value, the k-th first original pixel value is shifted to obtain a first shift result; the k-th first original pixel value is shifted to obtain a second shift result.

[0146] An addition calculation is performed on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; an addition calculation is performed on the second shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result;

[0147] The first addition result is shifted to obtain a third shift result.

[0148] An addition calculation is performed on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is part of the first target row.

[0149] Optionally, the interpolation module performs a pipelining operation. The first stage of the pipeline shifts the k-th first original pixel value to obtain a first shift result. An addition calculation is performed on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result. At the same time, the first stage of the pipeline also shifts the k-th first original pixel value to obtain a second shift result. An addition calculation is performed on the second shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result. The second stage of the pipeline shifts the first addition result to obtain a third shift result. An addition calculation is performed on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result. The third stage of the pipeline shifts the fourth addition result to obtain a target pixel value.

[0150] In the second example, a second target row is generated, including:

[0151] When the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value, shift the k-th first original pixel value to obtain a first shift result; shift the k-th second original pixel value to obtain a second shift result;

[0152] Perform an addition calculation on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a second addition result;

[0153] Shift the first addition result to obtain a third shift result;

[0154] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value; the target pixel value is part of the second target row.

[0155] In the third example, generating the second target row further includes:

[0156] When the pixel queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value, shift the (k + 1)-th first original pixel value to obtain a first shift result; shift the (k + 1)-th second original pixel value to obtain a second shift result;

[0157] Perform an addition calculation on the first shift result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a second addition result;

[0158] Shift the first addition result to obtain a third shift result;

[0159] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0160] In the fourth example, generating the second target row further includes:

[0161] When the pixel queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value, shift the (k + 1)-th second original pixel value to obtain a first shift result; shift the (k + 1)-th first original pixel value to obtain a second shift result;

[0162] Perform an addition calculation on the first shifted result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shifted result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a second addition result;

[0163] Shift the first addition result to obtain a third shifted result;

[0164] Perform an addition calculation on the third shifted result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0165] In some embodiments, the embodiments of the present application further include a removal module for removing garbage pixel values; the garbage pixel values are generated based on supplementary pixel values. That is, after the interpolation module generates a target pixel value, it transmits the target pixel value to the removal module, and the removal module removes the garbage pixel values and stores the valid pixel values in the DDR to obtain a target image. Among them, the valid pixel values can also be understood as the target pixel values in the target image. For example: Before generating the first target pixel value of the first target row, 15 garbage pixel values are generated. Then these 15 garbage pixel values need to be removed and not stored in the DDR. For the garbage pixel values generated after the m-th target pixel value, they are directly not stored in the DDR, and thus a target image with 2p rows and 2m columns can be obtained.

[0166] Optionally, the ISP (Image Signal Processor) stores the original image in the DDR. Taking the original pixel value of an original pixel point as an example for explanation. The original pixel value may be a signed number. The removal module takes the inverse of the highest bit of each signed number. For example: After taking the inverse, the value range of the original pixel value changes from [-128, 127] to [0, 255], so that the signed 16-bit can be regarded as an unsigned 16-bit. Finally, take the inverse of the highest bit of the target pixel value. That is, add 128 to the original pixel value and subtract 128 from the target pixel value. Thus, the embodiments of the present application can support the upsampling of both signed numbers and unsigned numbers with the calculation resources of unsigned numbers. And the inversion operation hardly wastes hardware resources. It should be noted that if the original pixel value is an unsigned number, no inversion operation is required.

[0167] The above embodiment is an upsampling method with a sampling ratio of 2. In addition, the embodiments of the present application can also perform upsampling with a sampling ratio that is a power of 2, such as 4, 8, and 16. Taking a sampling ratio of 4 as an example for explanation. When the sampling ratio is 4, two consecutive upsamplings with a sampling ratio of 2 are performed. That is, the above process is repeated 2 times. Similarly, upsampling with other sampling ratios can be achieved.

[0168] It should be noted that when the image upsampling device provided in the above embodiment executes the corresponding steps, only the division of the above functional modules is used as an example for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0169] In the embodiments of the present application, the original image is read by two reading modules, and SRAM is not required, saving hardware area. In addition, the original pixel values read by the reading modules are sorted by a sorting module to output a pixel value queue, so that the interpolation module can perform fixed calculations on each pixel value queue. Moreover, no multiplication calculation is required, so a multiplier is not needed, saving hardware area while simplifying the calculation process and thus saving hardware computing resources.

[0170] Figure 8 It is a schematic flowchart of an upsampling method provided according to an embodiment of the present application. As Figure 8 shown, in the embodiments of the present application, it is described by taking an application to a terminal with an image upsampling device as an example. The method includes the following steps:

[0171] In step 801, the first row of the original image is read; the first row includes multiple first original pixel values.

[0172] In step 802, the second row of the original image is read; the second row includes multiple second original pixel values.

[0173] In step 803, for a set of values, the set of values is arranged according to a preset rule to obtain a pixel value queue.

[0174] Among them, a set of values includes n first original pixel values in the first row and n second original pixel values obtained from the second row; the positions of the n first original pixel values are consecutive; the positions of the n second original pixel values are consecutive; each first original pixel value and a second original pixel value are in the same column of the original image; n is an integer greater than or equal to 1.

[0175] In step 804, the pixel value queue is shifted and added to generate a target pixel value.

[0176] Wherein, the target pixel value is a part of the target image.

[0177] In some embodiments, the pixel value queue sequentially includes a first pixel value, a second pixel value, a third pixel value, and a fourth pixel value; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

[0178] In some embodiments, the method further includes: reading the first row of the original image; reading the first row of the original image.

[0179] In some embodiments, the method further includes: the original image includes p rows; reading the p-th row of the original image; reading the p-th row of the original image; p is an integer greater than or equal to 1.

[0180] In some embodiments, the method further includes: a set of values includes the same two sets of the k-th first original pixel value and the (k + 1)-th first original pixel value in the first row; the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value; or the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value; k is an integer greater than or equal to 1.

[0181] In some embodiments, the method further includes: reading the first garbage data; the first garbage data is located after the m-th first original pixel value in the first row; the first row includes m first original pixel values; m is an integer greater than or equal to k + 3;

[0182] The second reading module is further configured to read the second garbage data; the second garbage data is located after the m-th second original pixel value in the second row; the second row includes m second original pixel values.

[0183] In some embodiments, the method further includes: a replacement module; the replacement module is configured to replace the first garbage data with a plurality of first preset pixel values to obtain a first intermediate row; replace the second garbage data with a plurality of second preset pixel values to obtain a second intermediate row.

[0184] In some embodiments, the method further includes: supplementing a preset number of third preset pixel values before the first first original pixel value in the first intermediate row to obtain a first filled row; supplementing a preset number of fourth preset pixel values before the first second original pixel value in the second intermediate row to obtain a second filled row.

[0185] In some embodiments, the method further includes: arranging a set of values according to a preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value of the first filling row; and the k-th second original pixel value and the (k + 1)-th second original pixel value of the second filling row; the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value; or the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value.

[0186] Arranging a set of values according to a preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value of the first filling row; and the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value of the second filling row; the pixel value queue is the (k + 2)-th first original pixel value, the (k + 3)-th first original pixel value, the (k + 2)-th second original pixel value, and the (k + 3)-th second original pixel value; or the pixel value queue is the (k + 3)-th first original pixel value, the (k + 2)-th first original pixel value, the (k + 3)-th second original pixel value, and the (k + 2)-th second original pixel value.

[0187] In some embodiments, the method further includes: arranging a set of values according to a preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value of the first filling row; and the k-th second original pixel value and the (k + 1)-th second original pixel value of the second filling row; the pixel value queue is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value; or the pixel value queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value.

[0188] Arranging a set of values according to a preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value of the first filling row; and the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value of the second filling row;

[0189] The pixel value queue is the (k + 2)-th second original pixel value, the (k + 3)-th second original pixel value, the (k + 2)-th first original pixel value, and the (k + 3)-th first original pixel value; or the pixel value queue is the (k + 3)-th second original pixel value, the (k + 2)-th second original pixel value, the (k + 3)-th first original pixel value, and the (k + 2)-th first original pixel value.

[0190] In some embodiments, the method further includes:

[0191] Generating a first target row; the first target row includes a plurality of target pixel values;

[0192] Generating a second target row; the second target row includes a plurality of target pixel values;

[0193] The first target row and the second target row are part of a target image.

[0194] In some embodiments, the method further includes:

[0195] Shifting a first pixel value to obtain a first shift result; shifting a third pixel value to obtain a second shift result;

[0196] Performing an addition calculation on the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; performing an addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result;

[0197] Shifting the first addition result to obtain a third shift result;

[0198] Performing an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shifting the fourth addition result to obtain a target pixel value; the target pixel value is part of the first target row.

[0199] In some embodiments, the method further includes:

[0200] When the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value, shifting the k-th first original pixel value to obtain a first shift result; shifting the k-th first original pixel value to obtain a second shift result;

[0201] Performing an addition calculation on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; performing an addition calculation on the second shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result;

[0202] Shifting the first addition result to obtain a third shift result;

[0203] Performing an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shifting the fourth addition result to obtain a target pixel value; the target pixel value is part of the first target row.

[0204] In some embodiments, the method further includes:

[0205] When the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value, shift the k-th first original pixel value to obtain a first shift result; shift the k-th second original pixel value to obtain a second shift result;

[0206] Perform an addition calculation on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a second addition result;

[0207] Shift the first addition result to obtain a third shift result;

[0208] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value; the target pixel value is a part of the second target row.

[0209] In some embodiments, the method further includes:

[0210] When the pixel queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value, shift the (k + 1)-th first original pixel value to obtain a first shift result; shift the (k + 1)-th second original pixel value to obtain a second shift result;

[0211] Perform an addition calculation on the first shift result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a second addition result;

[0212] Shift the first addition result to obtain a third shift result;

[0213] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0214] In some embodiments, the method further includes:

[0215] When the pixel queue is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value, shift the k-th second original pixel value to obtain a first shift result; shift the k-th first original pixel value to obtain a second shift result;

[0216] Perform an addition calculation on the first shift result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result;

[0217] Shift the first addition result to obtain a third shift result;

[0218] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0219] In some embodiments, the method further includes:

[0220] When the pixel queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value, shift the (k + 1)-th second original pixel value to obtain a first shift result; shift the (k + 1)-th first original pixel value to obtain a second shift result;

[0221] Perform an addition calculation on the first shift result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a second addition result;

[0222] Shift the first addition result to obtain a third shift result;

[0223] Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

[0224] In addition, the upsampling method provided in the above embodiments and the embodiments of the image upsampling device belong to the same concept. The specific implementation process can be seen in the device embodiments and will not be elaborated here.

[0225] In the embodiments of the present application, the original image is read by two reading modules without the need for SRAM, saving hardware area. In addition, the original pixel values read by the reading modules are sorted by a sorting module to output a pixel value queue, facilitating the interpolation module to perform fixed calculations on each pixel value queue. Moreover, no multiplication calculation is required, thus eliminating the need for a multiplier, saving hardware area and simplifying the calculation process, thereby saving hardware computing resources.

[0226] The embodiments of the present application also provide a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the above method is implemented.

[0227] Taking the computer device as an example of a terminal, Figure 9 FIG. is a schematic structural diagram of a terminal provided by an embodiment of the present application. Refer to Figure 9 , the terminal 900 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer, or a desktop computer. The terminal 900 may also be referred to by other names such as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, etc.

[0228] Generally, the terminal 900 includes: a processor 901 and a memory 902.

[0229] The processor 901 may include one or more processing cores, such as a quad-core processor, a penta-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0230] The memory 902 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 902 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 is used to store at least one program code, and the at least one program code is used to be executed by the processor 901 to implement the process executed by the terminal in the method embodiments provided in this application for the above-mentioned method.

[0231] In some embodiments, the terminal 900 may further optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902, and the peripheral device interface 903 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 903 through a bus, signal lines, or a circuit board. Specifically, the peripheral device includes at least one of a display screen 904, a camera assembly 905, an audio circuit 906, and a power supply 907.

[0232] The peripheral device interface 903 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on a separate chip or circuit board, and the embodiments of the present application do not limit this.

[0233] The display screen 904 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 904 is a touch display screen, the display screen 904 also has the ability to collect touch signals on or above the surface of the display screen 904. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 904 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be one display screen 904, which is set on the front panel of the terminal 900; in some other embodiments, there can be at least two display screens 904, which are respectively set on different surfaces of the terminal 900 or in a folding design; in some other embodiments, the display screen 904 can be a flexible display screen, which is set on the curved surface or the folding surface of the terminal 900. Even, the display screen 904 can also be set as an irregular non-rectangular shape, that is, an irregular-shaped screen. The display screen 904 can be prepared using materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0234] The camera assembly 905 is used to collect images or videos. In some embodiments, the camera assembly 905 includes a front camera and a rear camera. Generally, the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera respectively, to realize the function of background blurring by fusing the main camera and the depth-of-field camera, the function of panoramic shooting by fusing the main camera and the wide-angle camera, and the VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 905 can also include a flash. The flash can be a single-color temperature flash or a two-color temperature flash. The two-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0235] The audio circuit 906 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals for input to the processor 901 for processing. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signals from the processor 901 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for uses such as ranging. In some embodiments, the audio circuit 906 may further include a headphone jack.

[0236] The power supply 907 is used to supply power to each component in the terminal 900. The power supply 907 may be alternating current, direct current, a disposable battery, or a rechargeable battery. When the power supply 907 includes a rechargeable battery, the rechargeable battery may support wired charging or wireless charging. The rechargeable battery may also be used to support fast charging technology.

[0237] Those skilled in the art can understand that Figure 9 the structure shown in

[0238] does not limit the terminal 900, and may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements. Figure 10 Taking a computer device as a server as an example,

[0239] Embodiments of the present application also provide a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the method described above. Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact-disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0240] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc.

[0241] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image upsampling device, characterized in that, Including: A first reading module, a second reading module, a sorting module, and an interpolation module; The first reading module is configured to read the first row of the original image; the first row includes a plurality of first original pixel values; The second reading module is configured to read the second row of the original image; the second row includes a plurality of second original pixel values; The sorting module is configured to arrange a set of values according to a preset rule to obtain a pixel value queue; the pixel value queue sequentially includes a first pixel value, a second pixel value, a third pixel value, and a fourth pixel value; the set of values includes n first original pixel values of the first row and n second original pixel values obtained from the second row; the positions of the n first original pixel values are consecutive; the positions of the n second original pixel values are consecutive; each first original pixel value and a second original pixel value are in the same column of the original image; n is an integer greater than or equal to 1; The interpolation module is configured to perform weighted summation on the first pixel value, the second pixel value, the third pixel value, and the fourth pixel value to generate a target pixel value; performing weighted summation on the first pixel value, the second pixel value, the third pixel value, and the fourth pixel value includes performing shift and addition calculations on the first pixel value, the second pixel value, the third pixel value, and the fourth pixel value; the target pixel value is a part of the target image.

2. The device according to claim 1, characterized in that, The first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

3. The device according to claim 2, characterized in that, The first reading module is further configured to read the first row of the original image; The second reading module is further configured to read the first row of the original image.

4. The device according to claim 2, characterized in that, The original image includes p rows; the first reading module is further configured to read the p-th row of the original image; The second reading module is further configured to read the p-th row of the original image; p is an integer greater than or equal to 1.

5. The device according to claim 3, characterized in that, The set of values includes two identical sets of the k-th and (k + 1)-th first original pixel values of the first row; the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value; or the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value; k is an integer greater than or equal to 1.

6. The device according to claim 5, wherein The first reading module is further configured to read first garbage data; the first garbage data is located after the m-th first original pixel value of the first row; the first row includes m first original pixel values; m is an integer greater than or equal to k + 3; The second reading module is further configured to read second garbage data; the second garbage data is located after the m-th second original pixel value in the second row; the second row includes m second original pixel values.

7. The device according to claim 6, characterized in that, The device further includes a replacement module; the replacement module is configured to replace the first garbage data with a plurality of first preset pixel values to obtain a first intermediate row; and replace the second garbage data with a plurality of second preset pixel values to obtain a second intermediate row.

8. The device according to claim 7, characterized in that, The sorting module is further configured to: Supplement a preset number of third preset pixel values before the first first original pixel value in the first intermediate row to obtain a first filled row; and supplement the preset number of fourth preset pixel values before the first second original pixel value in the second intermediate row to obtain a second filled row.

9. The device according to claim 8, characterized in that, The sorting module is further configured to: Arrange the set of values according to the preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value in the first filled row; And the k-th second original pixel value and the (k + 1)-th second original pixel value in the second filled row; the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value; or the pixel value queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value. Arrange the set of values according to the preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th first original pixel value and the (k + 3)-th first original pixel value in the first filled row; And the (k + 2)-th second original pixel value and the (k + 3)-th second original pixel value in the second filled row; the pixel value queue is the (k + 2)-th first original pixel value, the (k + 3)-th first original pixel value, the (k + 2)-th second original pixel value, and the (k + 3)-th second original pixel value; or the pixel value queue is the (k + 3)-th first original pixel value, the (k + 2)-th first original pixel value, the (k + 3)-th second original pixel value, and the (k + 2)-th second original pixel value.

10. The device according to claim 8, characterized in that, The sorting module is further configured to: Arrange the set of values according to the preset rule to obtain a pixel value queue; the set of values includes the k-th first original pixel value and the (k + 1)-th first original pixel value in the first filled row; and the k-th and (k + 1)-th second original pixel values of the second filling row; the pixel value queue is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value; or the pixel value queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value; For the set of values, arrange the set of values according to the preset rule to obtain the next pixel value queue; the set of values includes the (k + 2)-th and (k + 3)-th first original pixel values of the first filling row; and the (k + 2)-th and (k + 3)-th second original pixel values of the second filling row; the pixel value queue is the (k + 2)-th second original pixel value, the (k + 3)-th second original pixel value, the (k + 2)-th first original pixel value, and the (k + 3)-th first original pixel value; or the pixel value queue is the (k + 3)-th second original pixel value, the (k + 2)-th second original pixel value, the (k + 3)-th first original pixel value, and the (k + 2)-th first original pixel value.

11. The device according to claim 10, characterized in that, The interpolation module is further configured to: generate a first target row; the first target row includes a plurality of the target pixel values; generate a second target row; the second target row includes a plurality of the target pixel values; The first target row and the second target row are part of the target image.

12. The device according to claim 11, characterized in that The interpolation module is further configured to: shift the first pixel value to obtain a first shift result; shift the third pixel value to obtain a second shift result; perform an addition calculation on the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result; shift the first addition result to obtain a third shift result; perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain one of the target pixel values; The target pixel value is part of the first target row.

13. The device according to claim 12, characterized in that, The generating the first target row includes: when the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th first original pixel value, shift the k-th first original pixel value to obtain a first shift result; shift the k-th first original pixel value to obtain a second shift result; Perform an addition calculation on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result; Shift the first addition result to obtain a third shift result; Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value; the target pixel value is part of the first target row.

14. The device according to claim 12, characterized in that, The generating the second target row includes: When the pixel value queue is the k-th first original pixel value, the (k + 1)-th first original pixel value, the k-th second original pixel value, and the (k + 1)-th second original pixel value, shift the k-th first original pixel value to obtain a first shift result; shift the k-th second original pixel value to obtain a second shift result; Perform an addition calculation on the first shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a second addition result; Shift the first addition result to obtain a third shift result; Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value; the target pixel value is part of the second target row.

15. The device according to claim 12, characterized in that, The generating the second target row further includes: When the pixel queue is the (k + 1)-th first original pixel value, the k-th first original pixel value, the (k + 1)-th second original pixel value, and the k-th second original pixel value, shift the (k + 1)-th first original pixel value to obtain a first shift result; shift the (k + 1)-th second original pixel value to obtain a second shift result; Perform an addition calculation on the first shift result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a second addition result; Shift the first addition result to obtain a third shift result; Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

16. The device according to claim 12, characterized in that, The generating the second target row further includes: When the pixel queue is the k-th second original pixel value, the (k + 1)-th second original pixel value, the k-th first original pixel value, and the (k + 1)-th first original pixel value, shift the k-th second original pixel value to obtain a first shift result; shift the k-th first original pixel value to obtain a second shift result; Perform an addition calculation on the first shift result, the k-th second original pixel value, and the (k + 1)-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the k-th first original pixel value, and the (k + 1)-th first original pixel value to obtain a second addition result; Shift the first addition result to obtain a third shift result; Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

17. The device according to claim 12, wherein Generating the second target row further includes: When the pixel queue is the (k + 1)-th second original pixel value, the k-th second original pixel value, the (k + 1)-th first original pixel value, and the k-th first original pixel value, shift the (k + 1)-th second original pixel value to obtain a first shift result; shift the (k + 1)-th first original pixel value to obtain a second shift result; Perform an addition calculation on the first shift result, the (k + 1)-th second original pixel value, and the k-th second original pixel value to obtain a first addition result; perform an addition calculation on the second shift result, the (k + 1)-th first original pixel value, and the k-th first original pixel value to obtain a second addition result; Shift the first addition result to obtain a third shift result; Perform an addition calculation on the third shift result, the first addition result, and the second addition result to obtain a fourth addition result; shift the fourth addition result to obtain a target pixel value.

18. An upsampling method, characterized in that, Including: Read the first row of the original image; the first row includes a plurality of first original pixel values; Read the second row of the original image; the second row includes a plurality of second original pixel values; For a set of values, arrange the set of values according to a preset rule to obtain a pixel value queue; the pixel value queue sequentially includes a first pixel value, a second pixel value, a third pixel value, and a fourth pixel value; the set of values includes n first original pixel values of the first row and n second original pixel values obtained from the second row; the positions of the n first original pixel values are consecutive; the positions of the n second original pixel values are consecutive; each first original pixel value and a second original pixel value are in the same column of the original image; n is an integer greater than or equal to 1; Perform a weighted sum of the first pixel value, the second pixel value, the third pixel value, and the fourth pixel value to generate a target pixel value; the performing a weighted sum of the first pixel value, the second pixel value, the third pixel value, and the fourth pixel value includes performing a shift and an addition calculation on the first pixel value, the second pixel value, the third pixel value, and the fourth pixel value; the target pixel value is part of a target image.

19. A computer device, characterized in that, The computer device includes a processor and a memory, the memory is used to store at least one segment of program, and the at least one segment of program is loaded and executed by the processor to perform the upsampling method as described in claim 18.

20. A computer-readable storage medium, characterized in that, At least one segment of program is stored in the computer-readable storage medium, and the at least one segment of program is loaded and executed by a processor to implement the upsampling method as described in claim 18.

Images