Accelerated calculation method and system for RAW graph interpolation, electronic equipment and storage medium

Abstract

The invention relates to an accelerated calculation method and system for RAW graph interpolation, electronic equipment and a storage medium. The RAW graph interpolation accelerated calculation method comprises the following steps: acquiring a to-be-processed image, loading the to-be-processed image, and outputting a plurality of pixel data; performing calculation processing on the plurality of pixel data, and outputting a plurality of color channel data; fitting the plurality of color channel data, and outputting a pixel table; performing extraction processing on the pixel table, and outputting color data of a plurality of pixel points; and sequentially writing the color data of the plurality of pixel points into a memory. According to the scheme provided by the invention, the calculation efficiency of the RGB value of each pixel can be improved, so that the consumed time is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to an accelerated calculation method, system, electronic device and storage medium for RAW image interpolation. Background Art

[0002] Currently, Raw image interpolation uses an interpolation algorithm to calculate the RGB value of each pixel. This is because in a Raw image, each pixel usually contains only a portion of the color information. The goal of the interpolation algorithm is to restore the complete RGB value from this limited information for subsequent image processing and display.

[0003] In the related art, only the RGB value of each pixel can be calculated separately, which is very inefficient and time-consuming. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide an accelerated calculation method, system, electronic device and storage medium for RAW image interpolation, which can improve the calculation efficiency of the RGB value of each pixel, thereby reducing the time consumption.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] The first aspect of the present application provides an accelerated calculation method for RAW image interpolation, including: obtaining an image to be processed, loading and processing the image to be processed, and outputting multiple pixel data; calculating and processing the multiple pixel data, and outputting multiple color channel data; fitting the multiple color channel data, and outputting a pixel table; extracting and processing the pixel table, and outputting multiple pixel point color data; and writing the multiple pixel point color data into a memory in sequence.

[0007] The step of acquiring the image to be processed, loading and processing the image to be processed, and outputting a plurality of pixel data includes: acquiring the image to be processed, analyzing and processing the image to be processed, and outputting an original single-channel Raw image; and continuously loading pixel points at adjacent positions in the original single-channel Raw image, and outputting a plurality of the pixel data.

[0008] The step of computing and processing the plurality of pixel data to output a plurality of color channel data comprises:

[0009] The pixel data are respectively averaged using a first algorithm, and a plurality of color channel data are output.

[0010] The extracting and processing the pixel table to output a plurality of pixel point color data includes: generating a multi-data summing instruction according to the pixel table, and executing the multi-data summing instruction to output a plurality of the pixel point color data.

[0011] The second aspect of the present application provides an accelerated computing system for RAW image interpolation, including: a loading module, used to obtain an image to be processed, load and process the image to be processed, and output multiple pixel data; a computing module, used to compute and process the multiple pixel data, and output multiple color channel data; a fitting module, used to fit the multiple color channel data, and output a pixel table; an extraction module, used to extract and process the pixel table, and output multiple pixel point color data; a writing module, used to write multiple pixel point color data into a memory in sequence.

[0012] The loading module is also used to obtain the image to be processed, analyze and process the image to be processed, and output an original single-channel Raw image;

[0013] Pixel points at adjacent positions in the original single-channel Raw image are continuously loaded to output a plurality of pixel data.

[0014] The calculation module is also used to perform mean value calculation on the pixel data respectively using a first algorithm, and output a plurality of color channel data.

[0015] The extraction module is also used to generate a multiple data summation instruction according to the pixel table, and execute the multiple data summation instruction to output a plurality of pixel point color data.

[0016] A third aspect of the present application provides an electronic device, including:

[0017] Processor; and

[0018] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0019] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0020] Compared with the prior art, the present invention has at least the following advantages:

[0021] After acquiring the image to be processed, the present application can continuously load and output a plurality of pixel data, and can perform subsequent interpolation calculation processing on the plurality of pixel data within one cycle, so that the interpolation efficiency of the original Raw image is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below.

[0023] Figure 1 A method flow chart of an accelerated calculation method for RAW image interpolation in one embodiment of the present invention;

[0024] Figure 2 A method flow chart of another implementation of the accelerated calculation method for RAW image interpolation in one embodiment of the present invention;

[0025] Figure 3 A functional module diagram of an accelerated computing system for RAW image interpolation in one embodiment of the present invention;

[0026] Figure 4 FIG. 4 is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0028] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0029] Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, or a detachable connection or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0030] Currently, raw image interpolation uses an interpolation algorithm to calculate the RGB value of each pixel. This is because in a raw image, each pixel usually contains only a portion of the color information. The goal of the interpolation algorithm is to restore the complete RGB value from this limited information for subsequent image processing and display. However, at this stage, only the RGB value of each pixel can be calculated separately, which is very inefficient and time-consuming.

[0031] In response to the above problems, the embodiments of the present application provide an accelerated calculation method, system, electronic device and storage medium for RAW image interpolation, which can improve the calculation efficiency of the RGB value of each pixel, thereby reducing the time consumption.

[0032] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0033] Figure 1 It is a flowchart of an accelerated calculation method for RAW image interpolation shown in an embodiment of the present application.

[0034] See also Figure 1 , an accelerated calculation method for RAW image interpolation, comprising:

[0035] Step S101: Acquire an image to be processed, load and process the image to be processed, and output a plurality of pixel data.

[0036] It should be noted that the computer terminal first captures the image to be processed, which is the original single-channel Raw image. Then the original single-channel Raw image is loaded and 8 consecutive 16-bit pixel data are output, where the pixel data is single-channel pixel data.

[0037] Step S102: Calculate and process a plurality of pixel data to output a plurality of color channel data.

[0038] It should be noted that multiple pixel data will be processed for mean value calculation at the same time, and finally multiple color channel data will be output. The color channel data here refers to the bilinear interpolation of each pixel point.

[0039] Step S103: Fit multiple color channel data and output a pixel table.

[0040] It should be noted that the calculated data of multiple color channels are fitted to form a pixel table.

[0041] Step S104: extracting and processing the pixel table to output a plurality of pixel point color data.

[0042] It should be noted that, since the computer can only display 8-bit color data, this step will extract the high-order eight-bit pixel color data in the pixel table and discard the low-order eight-bit pixel color data.

[0043] Step S105, writing the color data of multiple pixels into the memory in sequence.

[0044] It should be noted that the color data of multiple pixels will be written into the memory according to the arrangement of the computer.

[0045] Figure 2 for Figure 1 A more detailed method flow, an accelerated calculation method for RAW image interpolation, including:

[0046] Step S201, obtaining an image to be processed, analyzing and processing the image to be processed, and outputting an original single-channel Raw image;

[0047] Continuously load the pixels at adjacent positions in the original single-channel Raw image and output multiple pixel data.

[0048] It should be noted that the loading method is to load continuously through adjacent positions, and load 8 consecutive 16-bit single-channel pixel data m11, m12 to m43 at multiple adjacent positions, as shown in Table 1 below:

[0049] m11 A1 B1 C1 D1 E1 F1 G1 H1 m12 B1 C1 D1 E1 F1 G1 H1 I1 m13 C1 D1 E1 F1 G1 H1 I1 J1 m21 A2 B2 C2 D2 E2 F2 G2 H2 m22 B2 C2 D2 E2 F2 G2 H2 I2 m23 C2 D2 E2 F2 G2 H2 I2 J2 m31 A3 B3 C3 D3 E3 F3 G3 H3 m32 B3 C3 D3 E3 F3 G3 H3 I3 m33 C3 D3 E3 F3 G3 H3 I3 J3 m41 A4 B4 C4 D4 E4 F4 G4 H4 m42 B4 C4 D4 E4 F4 G4 H4 I4 m43 C4 D4 E4 F4 G4 H4 I4 J4

[0050] Table 1

[0051] Among them, A1, C1, E1, G1, I1, A3, C3, E3, G3, I3 are red pixels, B1, D1, F1, H1, J1, A2, C2, E2, G2, I2, A4, C4, E4, G4, I4, B3, D3, F3, H3, J3 are green pixels, and B2, D2, F2, H2, J2, B4, D4, F4, H4, J4 are blue pixels.

[0052] Step S202: Calculate the mean of each pixel data using the first algorithm, and output multiple color channel data.

[0053] Step S203: Fit multiple color channel data and output a pixel table.

[0054] It should be noted that the first algorithm here uses instructions such as _mm_avg_epu16 to calculate the mean of 8 pixels at the same time and calculate the bilinear interpolation mean data element. Based on the above Table 1, the red, green and blue values ​​of the pixels centered on the 8 pixel positions corresponding to m22 are bilinearly interpolated.

[0055] For example, based on Table 1, the following method is to calculate the blue bilinear interpolation of the pixel position corresponding to m22. Specifically, the formula (m21+m23) / 2 is used to obtain the following Table 2:

[0056]

[0057] Table 2

[0058] Among them, the green bilinear interpolation of (A2+C2) / 2, (C2+E2) / 2, (E2+G2) / 2 and (G2+I2) / 2 is discarded. Further, the odd columns are taken as m22, and the even columns are taken as avgm21m23, and the blue bilinear interpolation of the pixel position corresponding to m22 is obtained by interleaving and mixing. As shown in the following Table 3:

[0059] B2 (B2+D2) / 2 D2 (D2+F2) / 2 F2 (F2+H2) / 2 H2 (H2+J2) / 2

[0060] Table 3

[0061] For another example, the following method is to calculate the green bilinear interpolation of the pixel position corresponding to m22, and the following Table 4 is calculated by the formula (m12+m32+m21+m23) / 4:

[0062]

[0063] Table 4

[0064] By using the formula (m22+m13) / 2, we can calculate the following table 5:

[0065]

[0066] Table 5

[0067] Combining Table 4 and Table 5, the odd columns take avgm12m32m21m23, and the even columns take avgm22m13, and the green bilinear interpolation of the pixel position corresponding to m22 is obtained by interleaving and mixing, as shown in Table 6.

[0068]

[0069] Table 6

[0070] For another example, the following method is to calculate the red bilinear interpolation of the pixel position corresponding to m22, and the formula (m11+m13+m31+m33) / 4 is used to calculate Table 7:

[0071]

[0072] Table 7

[0073] By using the formula (m11+m13+m31+m33) / 4, we can calculate Table 8:

[0074]

[0075] Table 8

[0076] Combining Table 7 and Table 8, the odd columns take avgm11m13m31m33, and the even columns take avgm12m32, and the red bilinear interpolation of the pixel position corresponding to m22 is obtained by interleaving and mixing, as shown in Table 9:

[0077]

[0078] Table 9

[0079] In this way, the bilinear interpolation mean of 8 pixels can be calculated simultaneously in the above manner, which is highly efficient and time-saving.

[0080] Step S204: Generate a multi-data summation instruction according to the pixel table, and execute the multi-data summation instruction to output color data of multiple pixel points.

[0081] It should be noted that the interpolated data is 16-bit color value data. Since the computer can only display 8-bit color data, the extra bits of the interpolated red, green and blue values ​​are removed, leaving the upper 8 bits of data, and the upper bits are filled with 0. Therefore, the multi-data summation instruction will only leave the upper 8 bits of data for summation.

[0082] Specifically, based on the above Tables 1 to 9, the interpolated data, Tables 10 and 11, are obtained.

[0083]

[0084] Table 10

[0085]

[0086]

[0087] Table 11

[0088] According to the way of out-of-order processing of Table 10 and Table 11, Table 12 is obtained:

[0089]

[0090] Table 12

[0091] Among them, Table 12 uses the out-of-order instruction to adjust the 128-bit data arrangement order of avgRed8(m22).

[0092] It can be understood that Table 13 respectively uses the out-of-order instruction to adjust the arrangement order of the 128-bit data of avgGreen8(m22).

[0093]

[0094] Table 13

[0095] Table 14 is obtained by summing Table 13 and Table 12.

[0096]

[0097]

[0098] Table 14 and Table 15 show the order of adjusting the 128-bit data of avgBlue8(m22) using out-of-order instructions.

[0099]

[0100] Table 15 is obtained by summing Table 14 and Table 15 to obtain Table 16.

[0101]

[0102] Table 16 Finally, Table 16 is sorted to generate multiple pixel color data.

[0103]

[0104]

[0105] Table 17

[0106] Step S205: write the color data of multiple pixels into the memory in sequence.

[0107] It should be noted that the above Table 17 is written into the memory.

[0108] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides an accelerated computing system for RAW image interpolation, an electronic device and corresponding embodiments.

[0109] Figure 3 It is a functional module diagram of the accelerated computing system for RAW image interpolation shown in an embodiment of the present application.

[0110] See also Figure 3 , an accelerated computing system for RAW image interpolation, including: a loading module 100 for acquiring an image to be processed, loading and processing the image to be processed, and outputting a plurality of pixel data; a computing module 200 for computing and processing a plurality of pixel data, and outputting a plurality of color channel data; a fitting module 300 for fitting a plurality of color channel data, and outputting a pixel table; an extraction module 400 for extracting and processing the pixel table, and outputting a plurality of pixel point color data; a writing module 500 for sequentially writing a plurality of pixel point color data into a memory.

[0111] In one embodiment, the loading module 100 is further used to obtain an image to be processed, analyze and process the image to be processed, and output an original single-channel Raw image; continuously load pixel points at adjacent positions in the original single-channel Raw image, and output multiple pixel data.

[0112] In one implementation, the calculation module 200 is further configured to perform mean value calculation on the pixel data respectively using the first algorithm, and output a plurality of color channel data.

[0113] In one embodiment, the extraction module 400 is further configured to generate a multi-data summation instruction according to the pixel table, and execute the multi-data summation instruction to output a plurality of pixel point color data.

[0114] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0115] Figure 4 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0116] See also Figure 4 , the electronic device 1000 includes a memory 1010 and a processor 1020 .

[0117] The processor 1020 may be a central processing unit (CPU), or an integrated circuit composed of other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor may be any conventional processor that can run the Linux kernel.

[0118] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0119] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.

[0120] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0121] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0122] The scheme of the present application has been described in detail above with reference to the accompanying drawings. In the above embodiments, the descriptions of the various embodiments have their own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art should also be aware that the actions and modules involved in the specification are not necessarily required for the present application. In addition, it can be understood that the steps in the method of the embodiment of the present application can be adjusted in order, merged and deleted according to actual needs, and the modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0123] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An accelerated calculation method for RAW image interpolation, characterized in that: include: Acquire an image to be processed, load and process the image to be processed, and output a plurality of pixel data; Calculate and process the plurality of pixel data to output a plurality of color channel data; Fitting the plurality of color channel data and outputting a pixel table; Extract and process the pixel table and output color data of multiple pixels; The color data of the plurality of pixels are sequentially written into the memory.

2. The accelerated calculation method for RAW image interpolation according to claim 1, characterized in that: The step of acquiring the image to be processed, loading and processing the image to be processed, and outputting a plurality of pixel data includes: Acquire the image to be processed, analyze and process the image to be processed, and output an original single-channel Raw image; Pixel points at adjacent positions in the original single-channel Raw image are continuously loaded to output a plurality of pixel data.

3. The accelerated calculation method for RAW image interpolation according to claim 1, characterized in that: The step of computing and processing the plurality of pixel data to output a plurality of color channel data comprises: The pixel data are respectively averaged using a first algorithm, and a plurality of color channel data are output.

4. The accelerated calculation method for RAW image interpolation according to claim 1, characterized in that: The step of extracting and processing the pixel table and outputting a plurality of pixel point color data includes: A multiple data summation instruction is generated according to the pixel table, and the multiple data summation instruction is executed to output a plurality of pixel point color data.

5. An accelerated computing system for RAW image interpolation, characterized in that: include: A loading module, used for acquiring an image to be processed, loading and processing the image to be processed, and outputting a plurality of pixel data; A calculation module, used for calculating and processing the plurality of pixel data and outputting a plurality of color channel data; A fitting module, used for fitting the plurality of color channel data and outputting a pixel table; An extraction module is used to extract and process the pixel table and output color data of multiple pixels; The writing module is used to write the color data of the plurality of pixels into the memory in sequence.

6. The accelerated computing system for RAW image interpolation according to claim 5, characterized in that: The loading module is also used to obtain the image to be processed, analyze and process the image to be processed, and output an original single-channel Raw image; Pixel points at adjacent positions in the original single-channel Raw image are continuously loaded to output a plurality of pixel data.

7. The accelerated computing system for RAW image interpolation according to claim 5, characterized in that: The calculation module is also used to perform mean value calculation on the pixel data respectively using a first algorithm, and output a plurality of color channel data.

8. The accelerated computing system for RAW image interpolation according to claim 5, characterized in that: The extraction module is also used to generate a multiple data summation instruction according to the pixel table, and execute the multiple data summation instruction to output a plurality of pixel point color data.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 4.

Images