Image up-sampling device, method and equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and provides an image up-sampling device, method and equipment and a storage medium, and the device comprises a first reading module, a second reading module, a sorting module and an interpolation module. The first reading module is used for reading a first line of the original image; the first row comprises a plurality of first original pixel values; the second reading module is used for reading a second line of the original image; the second row comprises a plurality of second original pixel values; the sorting module is used for sorting a group of values according to a preset rule for the group of values to obtain a pixel value queue; each first original pixel value and one second original pixel value are in the same column of the original image; the interpolation module is used for carrying out shifting and additive calculation on the pixel value queue to generate a target pixel value; the target pixel value is part of the target image. According to the technical scheme, an SRAM does not need to be used, and the hardware area is reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image upsampling device, method, equipment and storage medium. Background Art

[0002] Upsampling refers to enlarging an image. The most common upsampling is a sampling ratio of 2, which doubles the width and height of the image. New pixel values ​​are usually generated based on the original pixel values ​​of the original image through interpolation algorithms.

[0003] However, related technologies usually use SRAM (Static Random Access Memory) to cache the original pixel values ​​of the current row, and perform interpolation operations after obtaining the original pixel values ​​of the next row. Using SRAM to cache the original pixel values ​​of the current row will greatly increase the hardware area. Summary of the invention

[0004] The embodiment of the present application provides an image upsampling device, method, equipment and storage medium, which does not require the use of SRAM and reduces the hardware area. The technical solution is as follows: According to a first aspect of an embodiment of the present application, there is provided an image upsampling device, the device comprising: A first reading module, a second reading module, a sorting module and an interpolation module; The first reading module is used to read a first line of the original image; the first line includes a plurality of first original pixel values; The second reading module is used to read a second line of the original image; the second line includes a plurality of second original pixel values; The sorting module is used for arranging a group of values ​​according to a preset rule to obtain a pixel value queue; the group of values ​​includes n first original pixel values ​​in the first row and n second original pixel values ​​in the second row; the positions of the n first original pixel values ​​are continuous; the positions of the n second original pixel values ​​are continuous; each of the first original pixel values ​​and one of the second original pixel values ​​are in the same column of the original image; n is an integer greater than or equal to 1; The interpolation module is used to perform shift and addition calculations on the pixel value queue to generate a target pixel value; the target pixel value is a part of the target image.

[0005] In a possible implementation, the pixel value queue includes a first pixel value, a second pixel value, a third pixel value and a fourth pixel value in sequence; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

[0006] In a possible implementation, the first reading module is further configured to read the first row of the original image; The second reading module is further used to read the first line of the original image.

[0007] In a possible implementation, the original image includes p lines; the first reading module is further configured to read the p-th line of the original image; The second reading module is further used to read the p-th row of the original image; p is an integer greater than or equal to 1.

[0008] In a possible implementation, the group of values ​​includes two identical groups of the kth first original pixel value and the k+1th first original pixel value of the first row; the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, and the k+1th first original pixel value; or the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th first original pixel value, and the kth first original pixel value; k is an integer greater than or equal to 1.

[0009] In a possible implementation, the first reading module is further used to read first garbage data; the first garbage data is located after the mth first original pixel value in the first row; the first row includes m first original pixel values; m is an integer greater than or equal to k+3; The second reading module is further used to read second junk data; the second junk data is located after the mth second original pixel value in the second row; the second row includes m second original pixel values.

[0010] In a possible implementation, the device further includes a replacement module; the replacement module is used to replace the first junk data with multiple first preset pixel values ​​to obtain a first intermediate row; and replace the second junk data with multiple second preset pixel values ​​to obtain a second intermediate row.

[0011] In a possible implementation, the sorting module is further used to: A preset number of third preset pixel values ​​are added before the first first original pixel value of the first intermediate row to obtain a first filling row; and a preset number of fourth preset pixel values ​​are added before the first second original pixel value of the second intermediate row to obtain a second filling row.

[0012] In a possible implementation, the sorting module is further used to: For the group of values, the group of values ​​are arranged according to the preset rule to obtain a pixel value queue; the group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row; the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value and the k+1th second original pixel value; or the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value and the kth second original pixel value; For the group of values, the group of values ​​are arranged according to the preset rules to obtain the next pixel value queue; the group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row; the pixel value queue is the k+2th first original pixel value, the k+3th first original pixel value, the k+2th second original pixel value and the k+3th second original pixel value; or the pixel value queue is the k+3th first original pixel value, the k+2th first original pixel value, the k+3th second original pixel value and the k+2th second original pixel value.

[0013] In a possible implementation, the sorting module is further used to: For the group of values, the group of values ​​are arranged according to the preset rule to obtain a pixel value queue; the group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row; the pixel value queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value and the k+1th first original pixel value; or the pixel value queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value and the kth first original pixel value; For the group of values, arrange the group of values ​​according to the preset rule to obtain the next pixel value queue; the group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row; The pixel value queue is the k+2th second original pixel value, the k+3th second original pixel value, the k+2th first original pixel value and the k+3th first original pixel value; or the pixel value queue is the k+3th second original pixel value, the k+2th second original pixel value, the k+3th first original pixel value and the k+2th first original pixel value.

[0014] In a possible implementation, the interpolation module is further used to: Generate a first target row; the first target row includes a plurality of the target pixel values; Generate a second target row; the second target row includes a plurality of the target pixel values; The first target line and the second target line are part of the target image.

[0015] In a possible implementation, the interpolation module is further used to: Shifting the first pixel value to obtain a first shift result; shifting the third pixel value to obtain a second shift result; Adding the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; adding the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

[0016] In a possible implementation manner, generating the first target row includes: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, and the k+1th first original pixel value, Shifting the kth first original pixel value to obtain a first shift result; shifting the kth first original pixel value to obtain a second shift result; Performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

[0017] In a possible implementation manner, generating the second target row includes: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value and the k+1th second original pixel value, shifting the kth first original pixel value to obtain a first shift result; shifting the kth second original pixel value to obtain a second shift result; performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the second target row.

[0018] In a possible implementation manner, the generating the second target row further includes: When the pixel queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value, and the kth second original pixel value, shifting the k+1th first original pixel value to obtain a first shift result; shifting the k+1th second original pixel value to obtain a second shift result; performing addition calculation on the first shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; and the fourth addition result is shifted to obtain the target pixel value.

[0019] In a possible implementation manner, the generating the second target row further includes: When the pixel queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value and the k+1th first original pixel value, shifting the kth second original pixel value to obtain a first shift result; shifting the kth first original pixel value to obtain a second shift result; Adding the first shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a first addition result; adding the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; and the fourth addition result is shifted to obtain the target pixel value.

[0020] In a possible implementation manner, the generating the second target row further includes: When the pixel queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value and the kth first original pixel value, shifting the k+1th second original pixel value to obtain a first shift result; shifting the k+1th first original pixel value to obtain a second shift result; Adding the first shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a first addition result; adding the second shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; and the fourth addition result is shifted to obtain the target pixel value.

[0021] According to a second aspect of an embodiment of the present application, an upsampling method is provided, the method comprising: Reading a first row of an original image; the first row includes a plurality of first original pixel values; Reading a second line of the original image; the second line includes a plurality of second original pixel values; For a group of values, the group of values ​​are arranged according to a preset rule to obtain a pixel value queue; the group of values ​​includes n first original pixel values ​​in the first row and n second original pixel values ​​in the second row; the positions of the n first original pixel values ​​are continuous; the positions of the n second original pixel values ​​are continuous; each of the first original pixel values ​​and one of the second original pixel values ​​are in the same column of the original image; n is an integer greater than or equal to 1; The pixel value queue is shifted and added to generate a target pixel value; the target pixel value is a part of the target image.

[0022] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes an upsampling method.

[0023] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement an upsampling method.

[0024] In an embodiment of the present application, an image upsampling device is provided, which reads the original image through two reading modules, does not require SRAM, and saves hardware area. In addition, the original pixel values ​​read by the reading module are sorted by the sorting module, and the pixel value queue is output, so that the interpolation module performs fixed calculations on each pixel value queue. And there is no need for multiplication calculation, so there is no need for a multiplier, which saves hardware area and simplifies the calculation process, thereby saving hardware computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application; Figure 2 is a structural schematic diagram of an image upsampling device provided according to an embodiment of the present application; Figure 3 is a schematic diagram of comparing a target image and an original image provided according to an embodiment of the present application; Figure 4 This is the first positional relationship between a target pixel and four surrounding original pixels provided in an embodiment of the present application; Figure 5 This is the second positional relationship between a target pixel and four surrounding original pixels provided in an embodiment of the present application; Figure 6 This is the third positional relationship between a target pixel and four surrounding original pixels provided in an embodiment of the present application; Figure 7 This is the fourth positional relationship between a target pixel and four surrounding original pixels provided in an embodiment of the present application; Figure 8 is a flow chart of an upsampling method provided according to an embodiment of the present application; Fig. 9 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application; Fig.10 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0028] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0029] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0030] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0031] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0032] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.

[0033] In the terminal 101, an image up-sampling device is provided; the image up-sampling device is electrically connected to a DDR (Double Data Rate SDRAM, double data rate synchronous dynamic random access memory) so that the image up-sampling device can collect image data.

[0034] The terminal 101 may be a smart phone with an image upsampling device, a wearable device, a personal computer, a laptop computer, a tablet computer, a smart TV, a car terminal, etc.

[0035] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0036] The terminal 101 is connected to the server 102 via a wired or wireless network.

[0037] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to any combination of local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, wired or wireless network, private network or virtual private network. In some embodiments, the data exchanged through the network is represented by technology and / or format including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above data communication technology.

[0038] In the related art, an interpolation algorithm is usually used to interpolate and generate new pixel values ​​according to the original pixel values ​​of the original image, thereby enlarging the original image. Common interpolation algorithms are as follows: Nearest Neighbor Interpolation is a simple interpolation method. Its principle is to determine the new pixel value based on the known pixel value closest to the target pixel position. In short, it selects the known pixel point closest to the target point and uses its pixel value as the pixel value of the new pixel.

[0039] Bilinear interpolation is more complex. It considers the four nearest pixels around the target pixel, and uses the distance and relative position between these pixels to perform a linear weighted average calculation. By interpolating these weights, bilinear interpolation can provide smoother and more continuous image results, helping to reduce jagged edges and distortion.

[0040] However, in the process of upsampling, the related technology usually uses SRAM to cache the original pixel values ​​of the current row, and then performs interpolation after obtaining the original pixel values ​​of the next row. This results in a large circuit area; and the calculation part requires a full-function multiplier, which also results in a large area. In addition, when interpolating, the original image is expanded once before interpolation. To do so, it is usually necessary to expand the original image once in advance, which will increase the processing time of the digital circuit.

[0041] In order to solve the above technical problem, an embodiment of the present application provides an image upsampling device.

[0042] Figure 2 is a structural schematic diagram of an image upsampling device 200 provided according to an embodiment of the present application, the device comprising: A first reading module 201 , a second reading module 202 , a sorting module 203 and an interpolation module 204 .

[0043] In some embodiments, the first reading module 201 and the second reading module 202 are electrically connected to the DDR respectively. The DDR is used to store the original image. The first reading module 201 and the second reading module 202 read the original pixel values ​​in the original image from the DDR respectively. For example, the first reading module 201 and the second reading module 202 are both DMA modules (Direct Memory Access). The original image includes p rows and m columns. Wherein, p is an integer greater than or equal to 1; m is an integer greater than or equal to k+3.

[0044] In the embodiment of the present application, any one of the bicubic interpolation algorithm, the nearest neighbor interpolation algorithm, and the bilinear interpolation algorithm can be selected during interpolation. The upsampling process of the present application is exemplarily described below using the bilinear interpolation algorithm as an example.

[0045] In some embodiments, in order to facilitate hardware calculation, the sorting module sorts the four original pixel values ​​in a set of values ​​to obtain a pixel value queue. Optionally, the pixel value queue includes a first pixel value, a second pixel value, a third pixel value, and a fourth pixel value in sequence; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

[0046] Figure 3 It is a schematic diagram of comparing a target image and an original image provided according to an embodiment of the present application.

[0047] Combine the following Figure 3 The process of reading the original image is exemplified.

[0048] It is understandable that the original image includes a plurality of original pixels. Each original pixel corresponds to an original pixel value. The target image includes a plurality of target pixels. Each target pixel corresponds to a target pixel value. When the sampling rate is 2, the width and height of the target image are twice the width and height of the original image.

[0049] In some embodiments, since the first target row of the target image can only be generated based on the first row of the original image and the supplementary pixel value, the supplementary pixel value is required. In order to solve this problem, the first reading module is used to read the first row of the original image. The second reading module is also used to read the first row of the original image. The first row of the original image read by the second reading module is used as the supplementary pixel value. Thus, the first target row of the target image can be generated based on the first row of the original image and the second row of the original image. For example, the target image includes 2p rows and 2m columns.

[0050] In some embodiments, for the second target row of the target image, the first reading module is used to read the first row of the original image; the first row includes a plurality of first original pixel values. The second reading module is used to read the second row of the original image; the second row includes a plurality of second original pixel values.

[0051] In one example, the target image can be generated because the original image needs to be read multiple times. The above-mentioned first row can be understood as the previous row, and the second row can be understood as the next row, that is, the reading step needs to be performed multiple times in a loop. Optionally, for the s+1th target row of the target image, the first reading module is used to read the sth row of the original image; the sth row includes multiple first original pixel values. The second reading module is used to read the s+1th row of the original image; the s+1th row includes multiple second original pixel values. s is an integer greater than or equal to 1 and less than (p-1). That is, when there is no need to supplement the pixel value, the first reading module and the second reading module read the original image at the same time, and the second reading module and the first reading module read two adjacent rows respectively.

[0052] In some embodiments, for the last target row of the target image, that is, the 2pth target row, it is necessary to generate based on the last row (pth row) of the original image and the supplementary pixel value, so the supplementary pixel value is also required. In order to solve this problem, in an embodiment of the present application, the original image includes p rows; the first reading module is also used to read the pth row of the original image. The second reading module is also used to read the pth row of the original image; p is an integer greater than or equal to 1. The last row of the original image read by the second reading module is used as the supplementary pixel value. Therefore, the 2pth target row can be generated based on the last row of the original image and the supplementary pixel value.

[0053] In one example, since the bus width is 256 bits, the first reading module and the second reading module read 256 bits per clock cycle. Each first original pixel value and the second original pixel value include two channel data, and the size of each channel data is 16 bits, so the size of each first original pixel value and the size of the second original pixel value is 32 bits. When the size of the first original pixel value included in each row of the original image exceeds 256 bits, in order to read 256 bits per clock cycle, the first reading module and the second reading module need to read garbage data. Specifically, the first reading module is also used to read the first garbage data; the first garbage data is located after the mth first original pixel value of the first row; the first row includes m first original pixel values. The second reading module is also used to read the second garbage data; the second garbage data is located after the mth second original pixel value of the second row; the second row includes m second original pixel values. Among them, the specific values ​​of the first garbage data and the second garbage data can be configured according to actual needs. For example, taking the first row read by the first reading module as an example, assuming that the original image includes 3 rows and 9 columns, the size of the first original pixel value included in the first row is 288 bits, and the first reading module needs two clock cycles to read the first row. However, the first reading module can read 512 bits in two clock cycles, so the first reading module needs to read 224 bits of the first garbage data. Similarly, the second reading module needs to read 224 bits of the second garbage data.

[0054] In some embodiments, the device further includes a replacement module; the replacement module is used to replace the first junk data with multiple first preset pixel values ​​to obtain a first intermediate row; replace the second junk data with multiple second preset pixel values ​​to obtain a second intermediate row. For example, the first preset pixel value is the mth first original pixel value of the first row. The second preset pixel value is the mth second original pixel value of the second row. Similarly, the first preset pixel value corresponding to the sth row is the mth first original pixel value of the sth row. The second preset pixel value corresponding to the s+1th row is the mth second original pixel value of the s+1th row.

[0055] In some embodiments, the sorting module is used to arrange a group of values ​​according to a preset rule for a group of values ​​to obtain a pixel value queue; the group of values ​​includes n first original pixel values ​​in the first row and n second original pixel values ​​in the second row; the positions of the n first original pixel values ​​are continuous; the positions of the n second original pixel values ​​are continuous; each first original pixel value and a second original pixel value are in the same column of the original image; n is an integer greater than or equal to 1. For example: when the bilinear interpolation algorithm is used for interpolation, n is 2. It should be understood that the preset rule indicates the arrangement order of the n first original pixel values ​​and the n second original pixel values. "The positions of the first original pixel values ​​are continuous" can be understood as: the positions of the original pixel points corresponding to the first original pixel values ​​in the original image are continuous.

[0056] In the first example, to generate the first target column of the target image, it is necessary to supplement at least one column of pixel values ​​before the first column of the original image. In order to solve this problem, in an embodiment of the present application, the sorting module is also used to: supplement a preset number of third preset pixel values ​​before the first first original pixel value of the first middle row to obtain a first filling row; supplement a preset number of fourth preset pixel values ​​before the first second original pixel value of the second middle row to obtain a second filling row, so as to generate the first two target pixel values ​​of the first target column of the target image. Optionally, the first target pixel value of the first target column is generated based on the first first original pixel value, the third preset pixel value, the second second original pixel value and the fourth preset pixel value of the first middle row. Among them, the preset pixel values ​​supplemented by different filling rows may be different. Optionally, in order to facilitate hardware processing, the preset number is determined according to the bus bit width. For example, a 256-bit third preset pixel value is supplemented before the first first original pixel value of the first middle row. That is, 8 third preset pixel values ​​need to be supplemented. Similarly, a 256-bit fourth preset pixel value is supplemented before the first second original pixel value of the second middle row. That is, 8 fourth preset pixel values ​​need to be supplemented. Optionally, the third preset pixel value is the first first original pixel value. The fourth preset pixel value is the first second original pixel value. That is, in the embodiment of the present application, the last column and the first column of the original image can be copied, and there is no need to expand the original image in advance, thereby shortening the processing time of the digital circuit.

[0057] In the second example, the sorting module is also used to: for a group of values, arrange a group of values ​​according to a preset rule to obtain a pixel value queue; a group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first row of the same two groups, and k is an integer greater than or equal to 1. It should be noted that when the first reading module reads the first row and the second reading module also reads the first row, the pixel queue value includes two cases. The first case is: the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, the k+1th first original pixel value. The second case is: the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th first original pixel value and the kth first original pixel value. Among them, the second case can be understood as exchanging the order of the kth first original pixel value and the k+1th first original pixel value.

[0058] Optionally, the sorting module outputs a queue of pixel values ​​within the same clock cycle.

[0059] Figures 4 to 7 These are several positional relationships between a target pixel and four surrounding original pixels provided in the embodiments of the present application.

[0060] Combine the following Figures 4 to 7 The principle of the preset rules is illustrated by way of example.

[0061] The reason why the embodiment of the present application needs to sort the four original pixel values ​​in a set of values ​​is that the target pixel value has four positional relationships with the four surrounding original pixel values. According to the bilinear interpolation algorithm, different positional relationships correspond to different calculation processes. In order to reduce the calculation of the interpolation module, the embodiment of the present application arranges the order of the four original pixel values ​​so that the four pixel values ​​in any pixel value queue are subjected to the same calculation.

[0062] For example: Figure 4 In the example, the target pixel is closest to the kth first original pixel. Figure 5 In , the target pixel is closest to the k+1th first original pixel. Figure 6 In the example, the target pixel is closest to the kth second original pixel. Figure 7 In , the target pixel is closest to the k+1th second original pixel.

[0063] In a third example, the sorting module is further used to: for a group of values, arrange a group of values ​​according to a preset rule to obtain a pixel value queue; a group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row. In this case, the sorting module inputs two pixel value queues in sequence. The first pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value, and the k+1th second original pixel value. The second pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value, and the kth second original pixel value.

[0064] In the fourth example, the sorting module is further used to: for a group of values, arrange a group of values ​​according to a preset rule to obtain the next pixel value queue; a group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row. In this case, the sorting module outputs two pixel value queues in sequence. The first pixel value queue is the k+2th first original pixel value, the k+3th first original pixel value, the k+2th second original pixel value, and the k+3th second original pixel value. The second pixel value queue is the k+3th first original pixel value, the k+2th first original pixel value, the k+3th second original pixel value, and the k+2th second original pixel value.

[0065] In the fifth example, the sorting module is further used to: for a group of values, arrange a group of values ​​according to a preset rule to obtain a pixel value queue; a group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row. In this case, the sorting module outputs two pixel value queues in sequence. The first pixel value queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value, and the k+1th first original pixel value. The second pixel value queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value, and the kth first original pixel value.

[0066] In the sixth example, the sorting module is further used to: for a group of values, arrange a group of values ​​according to a preset rule to obtain the next pixel value queue; a group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row. In this case, the sorting module outputs two pixel value queues in sequence. The first pixel value queue is the k+2th second original pixel value, the k+3th second original pixel value, the k+2th first original pixel value, and the k+3th first original pixel value. The second pixel value queue is the k+3th second original pixel value, the k+2th second original pixel value, the k+3th first original pixel value, and the k+2th first original pixel value.

[0067] In some embodiments, the interpolation module is used to shift and perform addition calculations on the pixel value queue to obtain a target pixel value; the target pixel value is a part of the target image.

[0068] Exemplarily, the interpolation module is further used to: generate a first target line; the first target line includes a plurality of target pixel values; generate a second target line; the second target line includes a plurality of target pixel values; the first target line and the second target line are part of the target image.

[0069] That is, after the interpolation module completes the generation of the first target row, it generates the second target row.

[0070] In some embodiments, the interpolation module is further configured to: Shifting the first pixel value to obtain a first shift result; shifting the third pixel value to obtain a second shift result; Performing addition calculation on the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; performing addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

[0071] It should be understood that the bilinear interpolation algorithm calculates the pixel value queue based on preset weights. For example: the weight of the first pixel value is a, the weight of the second pixel value is (1-a); the weight of the third pixel value is b, and the weight of the fourth pixel value is (1-b). The weights of the first pixel value and the second pixel value are c, the weights of the third pixel value and the fourth pixel value are (1-c), and a, b, and c are numbers greater than 0 and less than 1.

[0072] That is, a target pixel value can be calculated by the following formula: The target pixel value=c×(first pixel value×a+second pixel value×(1-a))+(1-c)×(third pixel value×b+fourth pixel value×(1-b)).

[0073] When c=0.75; a=0.75; 1-a=0.25; b=0.75; 1-b=0.25; 1-c=0.25.

[0074] Then the target pixel value = 1 / 16×(3×(3×first pixel value+second pixel value)+(3×third pixel value+fourth pixel value)).

[0075] However, the above algorithm needs to use a multiplier, which consumes a large hardware area. In order to solve this problem, the embodiment of the present application converts the above calculation method into shift and addition calculation, so that multiplication calculation is not required, and multipliers are not required, thereby saving hardware area.

[0076] For example, in the embodiment of the present application, the target pixel value can be calculated by the following formula: Since the target pixel value = 1 / 16×(3×(3×first pixel value+second pixel value)+(3×third pixel value+fourth pixel value)).

[0077] Then the target pixel value = 1 / 16×((3×(first pixel value<<1+first pixel value+second pixel value)+(third pixel value<<1+third pixel value+fourth pixel value)).

[0078] Target pixel value = (first pixel value << 1 + first pixel value + second pixel value) << 1 + (first pixel value << 1 + (first pixel value << 1 + first pixel value + second pixel value) + (third pixel value << 1 + third pixel value + fourth pixel value) >> 4. Among them, “<<” means left shift; ">>” means right shift.

[0079] It is understandable that the first pixel value is a binary number. "First pixel value << 1 + first pixel value + second pixel value" means that the first pixel value is shifted left by one bit, and then added to the first pixel value and the second pixel value. It should be noted that the target pixel value is obtained after the result of the right shift calculation is rounded.

[0080] In the first example, the first target row is generated, including: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, and the k+1th first original pixel value, the kth first original pixel value is shifted to obtain a first shift result; the kth first original pixel value is shifted to obtain a second shift result.

[0081] Performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; The first addition result is shifted to obtain a third shift result.

[0082] The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

[0083] Optionally, the interpolation module performs a pipeline operation. The first-stage pipeline shifts the kth first original pixel value to obtain a first shift result. The first shift result, the kth first original pixel value and the k+1th first original pixel value are added to obtain a first addition result. At the same time, the first-stage pipeline also shifts the kth first original pixel value to obtain a second shift result. The second shift result, the kth first original pixel value and the k+1th first original pixel value are added to obtain a second addition result. The second-stage pipeline shifts the first addition result to obtain a third shift result. The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result. The third-stage pipeline shifts the fourth addition result to obtain a target pixel value.

[0084] In the second example, a second target line is generated, including: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value, and the k+1th second original pixel value, the kth first original pixel value is shifted to obtain a first shift result; the kth second original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the second target row.

[0085] In the third example, a second target row is generated and also includes: When the pixel queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value, and the kth second original pixel value, the k+1th first original pixel value is shifted to obtain a first shift result; the k+1th second original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value.

[0086] In the fourth example, a second target row is generated, further comprising: When the pixel queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value, and the kth first original pixel value, the k+1th second original pixel value is shifted to obtain a first shift result; the k+1th first original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value.

[0087] In some embodiments, the embodiments of the present application further include a removal module, which is used to remove junk pixel values; junk pixel values ​​are generated based on supplementary pixel values. That is, after the interpolation module generates a target pixel value, it transmits the target pixel value to the removal module, and the removal module removes the junk pixel value and stores the valid pixel value in the DDR, thereby obtaining a target image. Among them, the valid pixel value can also be understood as the target pixel value in the target image. For example: before the first target pixel value of the first target row is generated, 15 junk pixel values ​​are generated. Then these 15 junk pixel values ​​need to be removed and not stored in the DDR. For the junk pixel values ​​generated after the mth target pixel value, they are not directly stored in the DDR, and thus a target image with 2p rows and 2m columns can be obtained.

[0088] Optionally, the ISP (Image Signal Processor) stores the original image in the DDR. Take the original pixel value of an original pixel as an example for explanation. The original pixel value may be a signed number. The removal module inverts the highest bit of each signed number. For example: after inversion, the value range of the original pixel value is changed from [-128, 127] to [0, 255], so that the signed 16-bit can be regarded as an unsigned 16-bit. Finally, the highest bit of the target pixel value is inverted. That is, the original pixel value is added with 128, and the target pixel value is subtracted by 128. Therefore, the embodiments of the present application can use the computing resources of unsigned numbers to support the upsampling of signed and unsigned numbers at the same time. And the inversion operation will hardly waste hardware resources. It should be noted that if the original pixel value is an unsigned number, no inversion operation is required.

[0089] The above embodiment is an upsampling method with a sampling magnification of 2. In addition, the embodiment of the present application can also perform upsampling with a sampling magnification of 4, 8, 16, and other powers of 2. The sampling magnification of 4 is used as an example for explanation. When the sampling magnification is 4, upsampling with a sampling magnification of 2 is performed twice in succession. That is, the above process is repeated twice. Similarly, upsampling of other sampling magnifications can be achieved.

[0090] It should be noted that: when the image upsampling device provided in the above embodiment executes the corresponding steps, it only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0091] In the embodiment of the present application, the original image is read by two reading modules, and no SRAM is required, which saves the hardware area. In addition, the original pixel values ​​read by the reading module are sorted by the sorting module, and the pixel value queue is output, so that the interpolation module performs fixed calculations on each pixel value queue. And there is no need for multiplication calculation, so there is no need for a multiplier, which saves the hardware area and simplifies the calculation process, thereby saving hardware computing resources.

[0092] Figure 8 is a flow chart of an upsampling method provided according to an embodiment of the present application, such as Figure 8 As shown, in the embodiment of the present application, the application is described by taking the application on a terminal having an image upsampling device as an example. The method comprises the following steps: In step 801, the first row of the original image is read; the first row includes a plurality of first original pixel values.

[0093] In step 802, a second row of the original image is read; the second row includes a plurality of second original pixel values.

[0094] In step 803, for a group of values, a group of values ​​are arranged according to a preset rule to obtain a pixel value queue.

[0095] Among them, a set of values ​​includes n first original pixel values ​​in the first row and n second original pixel values ​​in the second row; the positions of the n first original pixel values ​​are continuous; the positions of the n second original pixel values ​​are continuous; each first original pixel value and a second original pixel value are in the same column of the original image; n is an integer greater than or equal to 1.

[0096] In step 804, the pixel value queue is shifted and added to generate a target pixel value.

[0097] Among them, the target pixel value is a part of the target image.

[0098] In some embodiments, the pixel value queue includes a first pixel value, a second pixel value, a third pixel value and a fourth pixel value in sequence; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

[0099] In some embodiments, the method further includes: reading the first line of the original image; reading the first line of the original image.

[0100] In some embodiments, the method further includes: the original image includes p lines; reading the p-th line of the original image; reading the p-th line of the original image; p is an integer greater than or equal to 1.

[0101] In some embodiments, the method further includes: a group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the same two groups of first rows; the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, the k+1th first original pixel value; or the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th first original pixel value and the kth first original pixel value; k is an integer greater than or equal to 1.

[0102] In some embodiments, the method further includes: reading first garbage data; the first garbage data is located after the mth first original pixel value in the first row; the first row includes m first original pixel values; m is an integer greater than or equal to k+3; The second reading module is further used to read second junk data; the second junk data is located after the mth second original pixel value in the second row; the second row includes m second original pixel values.

[0103] In some embodiments, the method further includes: a replacement module; a replacement module, used to replace the first junk data with multiple first preset pixel values ​​to obtain a first intermediate row; and replace the second junk data with multiple second preset pixel values ​​to obtain a second intermediate row.

[0104] In some embodiments, the method further includes: supplementing a preset number of third preset pixel values ​​before the first first original pixel value of the first intermediate row to obtain a first filling row; and supplementing a preset number of fourth preset pixel values ​​before the first second original pixel value of the second intermediate row to obtain a second filling row.

[0105] In some embodiments, the method further includes: for a group of values, arranging the group of values ​​according to a preset rule to obtain a pixel value queue; the group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row; the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value and the k+1th second original pixel value; or the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value and the kth second original pixel value; For a group of values, arrange the group of values ​​according to a preset rule to obtain the next pixel value queue; a group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row; the pixel value queue is the k+2th first original pixel value, the k+3th first original pixel value, the k+2th second original pixel value and the k+3th second original pixel value; or the pixel value queue is the k+3th first original pixel value, the k+2th first original pixel value, the k+3th second original pixel value and the k+2th second original pixel value.

[0106] In some embodiments, the method further includes: for a group of values, arranging the group of values ​​according to a preset rule to obtain a pixel value queue; the group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row; the pixel value queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value and the k+1th first original pixel value; or the pixel value queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value and the kth first original pixel value; For a group of values, arrange the group of values ​​according to a preset rule to obtain a next pixel value queue; the group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row; The pixel value queue is the k+2th second original pixel value, the k+3th second original pixel value, the k+2th first original pixel value and the k+3th first original pixel value; or the pixel value queue is the k+3th second original pixel value, the k+2th second original pixel value, the k+3th first original pixel value and the k+2th first original pixel value.

[0107] In some embodiments, the method further comprises: generating a first target row; the first target row comprising a plurality of target pixel values; generating a second target row; the second target row comprising a plurality of target pixel values; The first target line and the second target line are part of the target image.

[0108] In some embodiments, the method further comprises: Shifting the first pixel value to obtain a first shift result; shifting the third pixel value to obtain a second shift result; Performing addition calculation on the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; performing addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

[0109] In some embodiments, the method further comprises: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, and the k+1th first original pixel value, the kth first original pixel value is shifted to obtain a first shift result; the kth first original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

[0110] In some embodiments, the method further comprises: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value, and the k+1th second original pixel value, the kth first original pixel value is shifted to obtain a first shift result; the kth second original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the second target row.

[0111] In some embodiments, the method further comprises: When the pixel queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value, and the kth second original pixel value, the k+1th first original pixel value is shifted to obtain a first shift result; the k+1th second original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value.

[0112] In some embodiments, the method further comprises: When the pixel queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value, and the k+1th first original pixel value, the kth second original pixel value is shifted to obtain a first shift result; the kth first original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value.

[0113] In some embodiments, the method further comprises: When the pixel queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value, and the kth first original pixel value, the k+1th second original pixel value is shifted to obtain a first shift result; the k+1th first original pixel value is shifted to obtain a second shift result; Performing addition calculation on the first shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value.

[0114] In addition, the upsampling method provided in the above embodiment and the image upsampling device embodiment belong to the same concept, and the specific implementation process thereof is detailed in the device embodiment and will not be repeated here.

[0115] In the embodiment of the present application, the original image is read by two reading modules, and no SRAM is required, which saves the hardware area. In addition, the original pixel values ​​read by the reading module are sorted by the sorting module, and the pixel value queue is output, so that the interpolation module performs fixed calculations on each pixel value queue. And there is no need for multiplication calculation, so there is no need for a multiplier, which saves the hardware area and simplifies the calculation process, thereby saving hardware computing resources.

[0116] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above method when executing the computer program.

[0117] Taking computer equipment as the terminal as an example, Fig. 9 is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Fig. 9 The terminal 900 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The terminal 900 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0118] Typically, the terminal 900 includes a processor 901 and a memory 902 .

[0119] The processor 901 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0120] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one program code, which is used to be executed by the processor 901 to implement the process provided by the method embodiment of the present application for the terminal execution in the above method.

[0121] In some embodiments, the terminal 900 may further optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 904, a camera assembly 905, an audio circuit 906 and a power supply 907.

[0122] The peripheral device interface 903 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0123] The display screen 904 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 904 is a touch display screen, the display screen 904 also has the ability to collect touch signals on the surface or above the surface of the display screen 904. The touch signal can be input as a control signal to the processor 901 for processing. At this time, the display screen 904 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 904 can be one, which is set on the front panel of the terminal 900; in other embodiments, the display screen 904 can be at least two, which are respectively set on different surfaces of the terminal 900 or are folded; in other embodiments, the display screen 904 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 900. Even, the display screen 904 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 904 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0124] The camera assembly 905 is used to capture images or videos. In some embodiments, the camera assembly 905 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 905 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0125] The audio circuit 906 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 906 may also include a headphone jack.

[0126] The power supply 907 is used to power various components in the terminal 900. The power supply 907 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 907 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0127] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal 900, and the terminal 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0128] Take the computer device as a server as an example. Fig.10 It is a structural diagram of a server provided in an embodiment of the present application. The server 1000 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1001 and one or more memories 1002, wherein the one or more memories 1002 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 1001 to implement the above-mentioned upsampling method. Of course, the server 1000 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 1000 may also include other components for implementing device functions, which will not be repeated here.

[0129] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above method. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0130] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0131] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image upsampling device, characterized in that: include: A first reading module, a second reading module, a sorting module and an interpolation module; The first reading module is used to read a first line of the original image; the first line includes a plurality of first original pixel values; The second reading module is used to read a second line of the original image; the second line includes a plurality of second original pixel values; The sorting module is used for arranging a group of values ​​according to a preset rule to obtain a pixel value queue; the group of values ​​includes n first original pixel values ​​in the first row and n second original pixel values ​​in the second row; the positions of the n first original pixel values ​​are continuous; the positions of the n second original pixel values ​​are continuous; each of the first original pixel values ​​and one of the second original pixel values ​​are in the same column of the original image; n is an integer greater than or equal to 1; The interpolation module is used to perform shift and addition calculations on the pixel value queue to generate a target pixel value; The target pixel value is a part of a target image.

2. The device according to claim 1, characterized in that The pixel value queue includes a first pixel value, a second pixel value, a third pixel value and a fourth pixel value in sequence; the first pixel value and the second pixel value are in the same row of the original image; the third pixel value and the fourth pixel value are in the same row of the original image; the first pixel value and the third pixel value are in the same column of the original image; the second pixel value and the fourth pixel value are in the same column of the original image.

3. The device according to claim 2, characterized in that The first reading module is further used to read the first line of the original image; The second reading module is further used to read the first line of the original image.

4. The device according to claim 2, characterized in that The original image includes p lines; the first reading module is further used to read the p-th line of the original image; The second reading module is further used to read the p-th row of the original image; p is an integer greater than or equal to 1.

5. The device according to claim 3, characterized in that The group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the same two groups of the first rows; the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, and the k+1th first original pixel value; or the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th first original pixel value and the kth first original pixel value; k is an integer greater than or equal to 1.

6. The device according to claim 5, characterized in that The first reading module is further used to read first garbage data; the first garbage data is located after the mth first original pixel value in the first row; the first row includes m first original pixel values; m is an integer greater than or equal to k+3; The second reading module is further used to read second junk data; the second junk data is located after the mth second original pixel value in the second row; the second row includes m second original pixel values.

7. The device according to claim 6, characterized in that The device further includes a replacement module, which is used to replace the first junk data with a plurality of first preset pixel values ​​to obtain a first intermediate row; and replace the second junk data with a plurality of second preset pixel values ​​to obtain a second intermediate row.

8. The device according to claim 7, characterized in that The sorting module is further used for: A preset number of third preset pixel values ​​are added before the first first original pixel value of the first intermediate row to obtain a first filling row; and a preset number of fourth preset pixel values ​​are added before the first second original pixel value of the second intermediate row to obtain a second filling row.

9. The device according to claim 8, characterized in that The sorting module is further used for: For the group of values, the group of values ​​are arranged according to the preset rule to obtain a pixel value queue; the group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row; the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value and the k+1th second original pixel value; or the pixel value queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value and the kth second original pixel value; For the group of values, arrange the group of values ​​according to the preset rule to obtain the next pixel value queue; the group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; And the k+2th second original pixel value and the k+3th second original pixel value of the second filling row; the pixel value queue is the k+2th first original pixel value, the k+3th first original pixel value, the k+2th second original pixel value and the k+3th second original pixel value; or the pixel value queue is the k+3th first original pixel value, the k+2th first original pixel value, the k+3th second original pixel value and the k+2th second original pixel value.

10. The device according to claim 8, characterized in that The sorting module is further used for: For the group of values, the group of values ​​are arranged according to the preset rule to obtain a pixel value queue; the group of values ​​includes the kth first original pixel value and the k+1th first original pixel value of the first filling row; and the kth second original pixel value and the k+1th second original pixel value of the second filling row; the pixel value queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value and the k+1th first original pixel value; or the pixel value queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value and the kth first original pixel value; For the group of values, arrange the group of values ​​according to the preset rule to obtain the next pixel value queue; the group of values ​​includes the k+2th first original pixel value and the k+3th first original pixel value of the first filling row; and the k+2th second original pixel value and the k+3th second original pixel value of the second filling row; The pixel value queue is the k+2th second original pixel value, the k+3th second original pixel value, the k+2th first original pixel value and the k+3th first original pixel value; or the pixel value queue is the k+3th second original pixel value, the k+2th second original pixel value, the k+3th first original pixel value and the k+2th first original pixel value.

11. The device according to claim 10, characterized in that The interpolation module is further used for: Generate a first target row; the first target row includes a plurality of the target pixel values; generating a second target row; the second target row comprising a plurality of the target pixel values; The first target line and the second target line are part of the target image.

12. The device according to claim 11, characterized in that The interpolation module is further used for: Shifting the first pixel value to obtain a first shift result; shifting the third pixel value to obtain a second shift result; Adding the first shift result, the first pixel value, and the second pixel value to obtain a first addition result; performing addition calculation on the second shift result, the third pixel value, and the fourth pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; performing addition calculation on the third shift result, the first addition result and the second addition result to obtain a fourth addition result; Shifting the fourth addition result to obtain the target pixel value; The target pixel value is part of the first target row.

13. The device according to claim 12, characterized in that The generating the first target row includes: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth first original pixel value, and the k+1th first original pixel value, Shifting the kth first original pixel value to obtain a first shift result; shifting the kth first original pixel value to obtain a second shift result; Performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the first target row.

14. The device according to claim 12, characterized in that The generating the second target row comprises: When the pixel value queue is the kth first original pixel value, the k+1th first original pixel value, the kth second original pixel value and the k+1th second original pixel value, shifting the kth first original pixel value to obtain a first shift result; shifting the kth second original pixel value to obtain a second shift result; performing addition calculation on the first shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; the fourth addition result is shifted to obtain a target pixel value; the target pixel value is a part of the second target row.

15. The device according to claim 12, characterized in that The generating the second target row further includes: When the pixel queue is the k+1th first original pixel value, the kth first original pixel value, the k+1th second original pixel value, and the kth second original pixel value, shifting the k+1th first original pixel value to obtain a first shift result; shifting the k+1th second original pixel value to obtain a second shift result; performing addition calculation on the first shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a first addition result; performing addition calculation on the second shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; and the fourth addition result is shifted to obtain the target pixel value.

16. The device according to claim 12, characterized in that The generating the second target row further includes: When the pixel queue is the kth second original pixel value, the k+1th second original pixel value, the kth first original pixel value and the k+1th first original pixel value, shifting the kth second original pixel value to obtain a first shift result; shifting the kth first original pixel value to obtain a second shift result; Adding the first shift result, the kth second original pixel value, and the k+1th second original pixel value to obtain a first addition result; adding the second shift result, the kth first original pixel value, and the k+1th first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; and the fourth addition result is shifted to obtain the target pixel value.

17. The device according to claim 12, characterized in that The generating the second target row further includes: When the pixel queue is the k+1th second original pixel value, the kth second original pixel value, the k+1th first original pixel value and the kth first original pixel value, shifting the k+1th second original pixel value to obtain a first shift result; shifting the k+1th first original pixel value to obtain a second shift result; Adding the first shift result, the k+1th second original pixel value, and the kth second original pixel value to obtain a first addition result; adding the second shift result, the k+1th first original pixel value, and the kth first original pixel value to obtain a second addition result; Shifting the first addition result to obtain a third shift result; The third shift result, the first addition result and the second addition result are added to obtain a fourth addition result; and the fourth addition result is shifted to obtain the target pixel value.

18. An upsampling method, characterized in that: include: Reading a first row of an original image; the first row includes a plurality of first original pixel values; Reading a second line of the original image; the second line includes a plurality of second original pixel values; For a group of values, the group of values ​​are arranged according to a preset rule to obtain a pixel value queue; the group of values ​​includes n first original pixel values ​​in the first row and n second original pixel values ​​in the second row; the positions of the n first original pixel values ​​are continuous; the positions of the n second original pixel values ​​are continuous; each of the first original pixel values ​​and one of the second original pixel values ​​are in the same column of the original image; n is an integer greater than or equal to 1; Shifting and adding the pixel value queue to generate a target pixel value; The target pixel value is a part of a target image.

19. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor to execute the upsampling method according to claim 18.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the upsampling method as claimed in claim 18.

Images