Image generation method, device, computer-readable storage medium and electronic device

Abstract

Embodiments of the present disclosure disclose an image generation method, apparatus, computer-readable storage medium, and electronic device. Among them, the method includes: performing image restoration processing on the original image data to obtain a restored image including at least one pass-through component channel; performing color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel; determining a target pass-through component channel from at least one pass-through component channel; merging the target pass-through component channel and the corrected color component channel to generate a corrected image in the first color space; converting the corrected image from the first color space to the second color space to obtain a corrected image in the second color space. Embodiments of the present disclosure retain the full value of the target pass-through component during color correction, and the brightness value of the finally obtained image does not suffer losses, and color noise brought by other color values is reduced, thereby effectively improving the clarity of the image.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to an image generation method, apparatus, computer-readable storage medium, and electronic device. Background Art

[0002] A CFA (Color Filter Array) is a complex of color filters. It can remove some components in the spectrum so that each pixel only retains one color component, and it is commonly used before a CCD (Charge Coupled Device) sensor.

[0003] Traditional image signal processing (ISP, Image Signal Processing) methods perform conversion processing based on an RGGB CFA. That is, each 2×2 array unit in the CFA includes one R (red) filter, two G (green) filters, and one B (blue) filter. The ISP system converts the four electrical signals collected through these four filters into the color value (such as an RGB value) of one pixel of the image.

[0004] With the development of image processing technologies in the field of environmental perception, in order to obtain image sensing signals with a higher signal-to-noise ratio, various optimized CFAs have emerged. Among them, an RCCC CFA means that each 2×2 array unit in the CFA includes one R filter and three C (Clarify) filters. The C filter does not filter any color light, that is, 75% of each 2×2 array unit is a pass-through filter, and the remaining 25% is a filter for sensing red light. The RCCC CFA can not only obtain a high signal-to-noise ratio but also detect red objects (such as signal lamp detection) by sensing the intensity of red light. Therefore, the RCCC CFA can be applied in fields such as autonomous driving.

[0005] Currently, the method for processing RCCC data is usually the same as the method for processing RGGB data. Therefore, there are certain losses in the details and brightness of the generated image. Summary of the Invention

[0006] In order to solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide an image generation method, apparatus, computer-readable storage medium, and electronic device.

[0007] Embodiments of the present disclosure provide an image generation method, which includes: performing image restoration processing on original image data to obtain a restored image including at least one pass-through component channel; performing color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel; determining a target pass-through component channel from at least one pass-through component channel; combining the target pass-through component channel and the corrected color component channel to generate a corrected image in a first color space; and converting the corrected image from the first color space to a second color space to obtain a corrected image in the second color space.

[0008] According to another aspect of the embodiments of the present disclosure, there is provided an image generation device, which includes: a restoration module configured to perform image restoration processing on original image data to obtain a restored image including at least one pass-through component channel; a correction module configured to perform color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel; a determination module configured to determine a target pass-through component channel from at least one pass-through component channel; a generation module configured to combine the target pass-through component channel and the corrected color component channel to generate a corrected image in a first color space; and a conversion module configured to convert the corrected image from the first color space to a second color space to obtain a corrected image in the second color space.

[0009] According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for being executed by a processor to implement the above image generation method.

[0010] According to another aspect of the embodiments of the present disclosure, there is provided an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; and the processor configured to read the executable instructions from the memory and execute the instructions to implement the above image generation method.

[0011] Based on the image generation method, apparatus, computer-readable storage medium, and electronic device provided in the above embodiments of the present disclosure, by performing image restoration processing on the original image data, a restored image including at least one pass-through component channel is obtained. Then, color correction is performed on the image data in a preset image channel in the restored image to obtain a corrected color component channel. Next, a target pass-through component channel is determined from the at least one pass-through component channel, and the target pass-through component channel and the corrected color component channel are combined to generate a corrected image in the first color space. Finally, the corrected image is converted from the first color space to the second color space to obtain a corrected image in the second color space. When performing color correction in the embodiments of the present disclosure, other color component channels do not participate in the extraction process of the target pass-through component, that is, the full value of the target pass-through component is retained, so that the brightness value of the finally obtained image does not suffer loss, and color noise brought by other color values is reduced, thereby effectively improving the clarity of the image and displaying richer detail information.

[0012] The technical solutions of the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] By describing the embodiments of the present disclosure in more detail with reference to the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure, and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation to the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0014] Figure 1 It is a system diagram applicable to the present disclosure.

[0015] Figure 2 It is a schematic flowchart of an image generation method provided by an exemplary embodiment of the present disclosure.

[0016] Figure 3 It is a schematic flowchart of an image generation method provided by another exemplary embodiment of the present disclosure.

[0017] Figure 4 It is a schematic flowchart of an image generation method provided by another exemplary embodiment of the present disclosure.

[0018] Figure 5 It is a schematic flowchart of an image generation method provided by another exemplary embodiment of the present disclosure.

[0019] Figure 6 It is a schematic structural diagram of an image generation apparatus provided by an exemplary embodiment of the present disclosure.

[0020] Figure 7It is a schematic structural diagram of an image generation device provided by another exemplary embodiment of the present disclosure.

[0021] Figure 8 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed implementation manners

[0022] Next, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. It should be understood that the present disclosure is not limited by the exemplary embodiments described herein.

[0023] It should be noted that: Unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and values set forth in these embodiments do not limit the scope of the present disclosure.

[0024] Those skilled in the art can understand that terms such as "first", "second", etc. in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and do not represent any specific technical meaning nor indicate an inevitable logical order between them.

[0025] It should also be understood that in the embodiments of the present disclosure, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0026] It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, without clear limitation or contrary indication in the context, it can generally be understood as one or more.

[0027] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the associated objects before and after.

[0028] It should also be understood that the present disclosure emphasizes the differences between various embodiments, and their similarities or similarities can be referred to each other. For the sake of brevity, they will not be described one by one.

[0029] At the same time, it should be understood that for the sake of description, the dimensions of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0030] The following description of at least one exemplary embodiment is actually only illustrative and in no way a limitation on the present disclosure and its application or use.

[0031] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods, and devices should be considered as part of the specification.

[0032] It should be noted that like reference numerals and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

[0033] Embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems, and so on.

[0034] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system / server can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0035] Application Overview

[0036] In a typical image signal processing method, the RGB2YUV method is required to convert an image from the RGB color space to the YUV (where Y represents luminance, and U and V represent chrominance, which is used to describe the color and saturation of the image) color space. The conversion formulas can be as follows:

[0037] Y = 0.299 * R + 0.587 * G + 0.114 * B

[0038] U = -0.168 * R - 0.328 * G + 0.5 * B + 128

[0039] V = 0.5 * R - 0.418 * G - 0.078 * B + 128

[0040] If the above formula is used to calculate for the RCCC CFA, that is, replacing the G and B components in the above formula with the C component, the following problems exist:

[0041] Since Y = 0.299*R + 0.587*C + 0.114*C, therefore, in the generation process of Y, the contribution of the C channel component is only about 70%, resulting in a brightness loss. Before performing the RGB2YUV calculation, the gain values compensated for the R channel component by the white balance module and the color restoration module included in the ISP system will be introduced to participate in the operation during the RGB2YUV calculation, causing significant color noise.

[0042] Embodiments of the present disclosure aim to solve the above problems. After performing image restoration processing on the original image data to obtain a restored image including a pass-through component channel, when performing color correction and image space conversion, other color component channels do not participate in the extraction process of the target pass-through component, that is, the full value of the target pass-through component is retained, so that the brightness value of the finally obtained image does not suffer loss and the color noise brought by other color values is reduced.

[0043] Exemplary System

[0044] Figure 1 Exemplary system architecture 100 of an image generation method or an image generation device to which embodiments of the present disclosure can be applied is shown.

[0045] As Figure 1 shown, system architecture 100 may include a terminal device 101, a network 102, a server 103, and an image sensor 104. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0046] The image sensor 104 includes a color filter array, and the color filter array includes a pass-through filter, so that the original image data collected by the image sensor 104 includes pass-through component data.

[0047] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications may be installed on the terminal device 101, such as an image processing application, a video surveillance application, etc.

[0048] The terminal device 101 may be various electronic devices, including but not limited to mobile terminals such as in-vehicle terminals, mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0049] The terminal device 101 can receive the raw image data collected by the image sensor 104, process the raw image data, and obtain a corrected image in the second color space. Alternatively, the terminal device 101 can send the raw image data to the server 103.

[0050] The server 103 can be a server that provides various services, such as a background image processing server that processes the raw image data uploaded by the terminal device 101. The background image processing server can generate a corrected image in the second color space based on the raw image data.

[0051] It should be noted that the image generation method provided by the embodiments of the present disclosure can be executed by the server 103 or the terminal device 101. Correspondingly, the image generation device can be set in the server 103 or the terminal device 101.

[0052] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. In the case where the raw image data does not need to be obtained remotely, the above system architecture may not include a network and only includes a server or a terminal device.

[0053] Exemplary Method

[0054] Figure 2 is a schematic flowchart of an image generation method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device (such as Figure 1 the terminal device 101 or the server 103 shown), as Figure 2 shown, the method includes the following steps:

[0055] Step 201, perform image restoration processing on the raw image data to obtain a restored image including at least one pass-through component channel.

[0056] In this embodiment, the electronic device can perform image restoration processing on the raw image data to obtain a restored image including at least one pass-through component channel. Among them, the raw image data is data collected by an image sensor as Figure 1 shown. The raw image data includes a preset number of groups of pixel data, and each group of pixel data includes at least one pass-through component data.

[0057] As an example, the image sensor includes an RCCC color filter array, that is, each 2×2-sized array unit in the color filter array includes one R filter and three C (Clarify) filters. The image sensor senses the light passing through these four filters to obtain an electrical signal, and further obtains a corresponding set of pixel data, and this set of pixel data includes three clarify component data.

[0058] The original image data is single-channel data, and the pixel data it includes is distributed within this single channel. The single-channel original image data cannot be normally displayed in the form of an image. The electronic device can convert the original image data into an image including multiple channels that can be normally displayed, and these channels include at least one clarify component channel. The electronic device can perform image restoration processing on the original image data according to methods such as image signal processing (ISP, Image Signal Processing). For example, methods including white balance processing method and demosaicing processing method included in the ISP method can be used to perform image restoration processing on the original image data.

[0059] Generally, for the method of performing image restoration processing on the original image data, the method of generating an RGB image can be adopted. For example, each set of pixel data of the original image data used to generate the RGB image is composed of RGGB data, and each set of pixel data of the original image data in this embodiment is composed of RCCC data. Therefore, the CCC data can be used to generate the corresponding clarify channel components C1 and C2 according to the method of processing GGB data to generate G and B channel components, that is, the restored image can include three channel components of R, C1, and C2.

[0060] Step 202: Perform color correction on the image data in the preset image channel in the restored image to obtain a corrected color component channel.

[0061] In this embodiment, the electronic device can perform color correction on the image data in the preset image channel in the restored image to obtain a corrected color component channel. Among them, the preset image channel can be all the channels included in the restored image, or on the basis of all the channels included in the restored image, at least one channel is added, and each data in the added channel is a constant (such as 1).

[0062] The execution process of normal color correction is to compare the original image captured by the camera with a reference image in advance, so as to calculate a color correction matrix (CCM, Color Correction Matrix). Using the obtained color correction matrix, the color of the restored image can be corrected to obtain an image with a color close to the real scene. In this embodiment, since the restored image is not an RGB color image, a color correction matrix can be set in advance. For each pixel of the restored image, the respective channel components included in the pixel form a vector, and the color correction matrix is multiplied by the vector to obtain the corrected color component channels corresponding to the pixel.

[0063] Step 203: Determine a target pass-through component channel from at least one pass-through component channel.

[0064] In this embodiment, the electronic device can determine a target pass-through component channel from at least one pass-through component channel. Among them, the target pass-through component channel can be any one of the above at least one pass-through component channels. As an example, the restored image includes three channels: R, C1, and C2. The C1 channel can be set as the target pass-through component channel, or the C2 channel can be set as the target pass-through component channel. Optionally, a pass-through component channel extraction matrix can be set in advance, and the matrix is multiplied by the vector formed by the respective components included in each pixel of the restored image to obtain the target pass-through component corresponding to each pixel. The set of target pass-through components of each pixel is the target pass-through component channel. Optionally, the channel corresponding to the specified target channel identifier can also be determined as the target pass-through component channel.

[0065] Step 204: Merge the target pass-through component channel and the corrected color component channels to generate a corrected image in the first color space.

[0066] In this embodiment, the electronic device can merge the target pass-through component channel and the corrected color component channels to generate a corrected image in the first color space. Generally, the target pass-through component channel and the corrected color component channels can be directly superimposed to generate a corrected image including the target pass-through component channel and the corrected color component channels.

[0067] As an example, if the target pass-through component channel is C1 and the corrected color component channels include G′ and B′, the corrected image can include three channels: C1, G′, and B′. It should be noted that G′ and B′ here represent color component channels obtained by a method similar to that for color correction of RGB images, and have nothing to do with the actual colors represented by the color component channels.

[0068] Step 205: Convert the corrected image from the first color space to the second color space to obtain the corrected image in the second color space.

[0069] In this embodiment, the electronic device can convert the corrected image from the first color space to the second color space to obtain the corrected image in the second color space.

[0070] In a conventional ISP method based on color images, the color space of the corrected image is the RGB color space, and it is necessary to convert the corrected image from the RGB color space to the YUV color space. In this embodiment, the first color space of the corrected image is the above-mentioned C1G′B′ color space, and the second color space is the YUV color space. A color space conversion matrix can be set according to a method similar to converting the RGB color space to the YUV color space to convert the C1G′B′ color space to the YUV color space.

[0071] The method provided in the above embodiments of the present disclosure performs image restoration processing on the original image data to obtain a restored image including at least one pass-through component channel, then performs color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel, then determines a target pass-through component channel from at least one pass-through component channel, combines the target pass-through component channel and the corrected color component channel to generate a corrected image in the first color space, and finally converts the corrected image from the first color space to the second color space to obtain the corrected image in the second color space. When performing color correction in the embodiments of the present disclosure, other color component channels do not participate in the extraction process of the target pass-through component, that is, the full value of the target pass-through component is retained, so that the brightness value of the finally obtained image is not lost, and the color noise brought by other color values is reduced, thereby effectively improving the clarity of the image and displaying richer detail information.

[0072] In some optional implementation manners, as Figure 3 shown, step 202 includes:

[0073] Step 2021: Perform color correction on the channel components included in the restored image based on a preset first correction parameter set to obtain a corrected color component channel.

[0074] Among them, the first correction parameter set is used to perform weighted summation on the component values included in each pixel of the restored image to obtain the corrected color component value of each pixel.

[0075] As an example, as shown in the following formula (1), it shows the calculation process of performing color correction on the channel components included in the restored image.

[0076]

[0077] The 3×4 matrix in Equation (1) is a color correction matrix, and the first set of correction parameters is included in the color correction matrix, that is, c in Equation (1). 21 、c 22 、c 23 、c 24 、c 31 、c 32 、c 33 、c 34 are the first set of correction parameters. The vector [R C1 C2] in Equation (1) represents the component values included in a pixel of the restored image. According to the matrix multiplication rule, G′ and B′ are the corrected color component values of this pixel. Each parameter value in the first set of correction parameters can be preset. Usually, a reference image and a restored image can be preset, and by repeatedly modifying each parameter value in the first set of correction parameters, the indicators (such as contrast, saturation, etc.) of the image obtained after color correction using the first set of correction parameters are made close to the reference image, thereby obtaining the first set of correction parameters.

[0078] In this embodiment, the color correction is performed on the components included in the restored image using the first set of correction parameters, which can more specifically perform weighted summation on the channel components included in the restored image, so that the corrected color component channels are more accurate, which helps to improve the quality of the corrected image.

[0079] In some optional implementation manners, as Figure 3 shown, step 203 includes:

[0080] Step 2031, based on a preset second set of correction parameters, perform channel extraction on the channel components included in the restored image to obtain a target pass-through component channel included in at least one pass-through component channel.

[0081] Among them, the second set of correction parameters is used to perform weighted summation on the component values included in each pixel of the restored image, so as to extract the target pass-through component channel from at least one pass-through component channel included in the restored image.

[0082] As an example, as shown in the above Equation (1), the second set of correction parameters is included in the color correction matrix, that is, 0, 1, 0, 0 in Equation (1) are the second set of correction parameters. The vector [R C1 C2] in Equation (1) represents the component values included in a pixel of the restored image. According to the matrix multiplication rule, C1 can be extracted from [R C1 C2] as the component value included in the target pass-through component channel. Each parameter value in the second set of correction parameters can also be preset, that is, the position of the numerical value 1 in the second set of correction parameters shown in Equation (1) can be adjusted to determine which pass-through component channel is extracted as the target pass-through component channel.

[0083] In this embodiment, by using the second correction parameter set to extract a target pass-through component channel from at least one pass-through component channel included in the restored image, it is possible to flexibly and efficiently extract the target pass-through component channel. Combined with the above first correction parameter set, and by using matrix multiplication to calculate the channel components included in the restored image, it is possible to efficiently generate a corrected image including the target pass-through component channel.

[0084] In some alternative implementation manners, as Figure 4 shown, step 2021 includes:

[0085] Step 20211: Add a color bias component channel on the basis of the original component channels included in the restored image.

[0086] Each component value included in the color bias component channel can usually be set to a constant. As an example, as shown in the above formula (1), the vector [R C1 C2 1] represents the component values of each channel included in a pixel of the restored image, and the numerical value 1 therein is the color bias component value corresponding to a pixel.

[0087] Step 20212: Perform color correction on the original channels included in the restored image based on the bias term elements and scale term elements included in the first correction parameter set to obtain color bias data and color base data.

[0088] As shown in the above formula (1), the first correction parameter set is included in the color correction matrix. The bias term elements include c 21 , c 22 , c 23 , c 31 , c 32 , c 33 , and the scale term elements include c 24 , c 34 . According to the matrix multiplication rule, the color base data includes c 21 *R + c 22 *C1 + c 23 *C2 and c 31 *R + c 32 *C1 + c 33 *C2. The color bias data includes c 24 *1 and c 34 *1.

[0089] Step 20213: Generate corrected color component channels based on the color bias data and the color base data.

[0090] Specifically, the color bias data and the color base data corresponding to the same corrected color component channel can be added together, or the color bias data and the color base data can be weighted and summed according to a preset weight to obtain the corrected color component channel. As shown in the above formula (1), according to the matrix multiplication principle, in the corrected image, G' corresponding to a pixel in the corrected color component channel is the sum of the corresponding color base data and color bias data, and B' corresponding to a pixel is the sum of the corresponding color base data and color bias data.

[0091] In this embodiment, by adding a color bias component channel on the basis of the original component channels included in the restored image, and setting the bias term element and the scale term element in the first correction parameter set, and performing color correction based on the bias term element and the scale term element to generate the corrected color component channel, it is possible to flexibly set the bias term for the corrected color component channel, which helps to further improve the quality of the generated corrected image.

[0092] In some optional implementation manners, as Figure 5 shown, step 205 includes:

[0093] Step 2051, perform color space conversion on the corrected color component channels of the corrected image to obtain the converted color component channels.

[0094] Specifically, a preset color space conversion matrix can be used to perform color space conversion on the corrected color component values of each pixel in the corrected image to obtain the converted color component values of each pixel. The values of the elements included in the color space conversion matrix can be preset in advance.

[0095] Step 2052, extract the target pass-through component channel from the corrected image.

[0096] Optionally, the target pass-through component channel can be directly extracted from the corrected image, or a pass-through component channel extraction matrix can be preset in advance, and the matrix is multiplied by the vector composed of each component included in each pixel of the corrected image to obtain the target pass-through component corresponding to each pixel. The set of the target pass-through components of each pixel is the target pass-through component channel.

[0097] Step 2053, merge the target pass-through component channel and the converted color component channels to generate the corrected image in the second color space.

[0098] As an example, if the target pass-through component channel is C1 and the second color space is the YUV color space, then C1 can be used as the Y channel, and the U channel and the V channel are the converted color component channels. Then, the corrected image in the second color space can include three channels: C1, U, and V.

[0099] In this embodiment, by performing color space conversion on the corrected color component channels and extracting the target pass-through component channels from the corrected image, the converted color component channels and the target pass-through component channels are combined to generate the corrected image in the second color space, so that the finally obtained corrected image includes the full amount value of the target pass-through component channels, minimizing the brightness loss of the corrected image in the second color space, and thus effectively improving the clarity of the corrected image, and the corrected image can display richer detail information.

[0100] In some alternative implementation manners, step 2051 may be performed as follows:

[0101] Based on the first parameter subset included in the preset color space conversion parameter set, perform color space conversion on the channel components included in the corrected image to obtain the converted color component channels.

[0102] Among them, the color space conversion parameter set is used to convert the corrected image from the first color space to the second color space. The first parameter subset is used to perform weighted summation on the component values included in each pixel of the corrected image to obtain the converted color component value of each pixel.

[0103] As an example, as shown in the following formula (2), it shows the calculation process of performing color space conversion on the channel components included in the corrected image.

[0104]

[0105] The color space conversion parameter set is in the form of a matrix, that is, the 3×3 matrix in formula (2) is the color space conversion matrix, the first parameter subset is included in the color space conversion matrix, that is, a, b, c, d, e, f in formula (2) are the first parameter subset, and the vector [C1 G′B′] in formula (2) represents the component values included in a pixel of the corrected image in the first color space. According to the matrix multiplication rule, U and V are the converted color component values of this pixel. Each parameter value in the first parameter subset can be set in advance. Optionally, according to the parameters commonly used to convert the RGB color space to the YUV color space currently, the parameter values corresponding to the U and V channels can be used as the parameter values included in the first parameter subset.

[0106] In this embodiment, by setting the first parameter subset included in the color space conversion parameter set to perform color space conversion on the corrected image, it is possible to more specifically perform weighted summation on the channel components included in the corrected image, so that the converted color component channels are more accurate, which helps to improve the quality of the corrected image in the second color space.

[0107] In some alternative implementation manners, the above step 2052 may be performed as follows:

[0108] Based on the second parameter subset included in the color space conversion parameter set, channel extraction is performed on the channel components included in the corrected image to obtain the target pass-through component channel.

[0109] Among them, the second parameter subset is used to perform weighted summation on the component values included in each pixel of the corrected image, so as to extract the target pass-through component channel from each component channel included in the corrected image.

[0110] As an example, as shown in the above formula (2), the second parameter subset is included in the color space conversion matrix, that is, 1, 0, 0 in formula (2) are the second parameter subset, and the vector [C1 G′B′] in formula (2) represents the component values included in a pixel of the corrected image. According to the matrix multiplication rule, the target pass-through component value C1 can be extracted from [C1 G′B′], and the target pass-through component channel is the Y channel of the YUV color space.

[0111] In this embodiment, by using the second parameter subset, the target pass-through component channel can be flexibly and efficiently extracted from the channel components included in the corrected image. Combined with the above first parameter subset, the channel components included in the corrected image are calculated using matrix multiplication, and the corrected image in the second color space including the target pass-through component channel can be efficiently generated.

[0112] Exemplary Device

[0113] Figure 6 It is a schematic structural diagram of an image generation device provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, such as Figure 6 As shown, the image generation device includes: a restoration module 601, configured to perform image restoration processing on the original image data to obtain a restored image including at least one pass-through component channel; a correction module 602, configured to perform color correction on the image data in a preset image channel of the restored image to obtain corrected color component channels; a determination module 603, configured to determine a target pass-through component channel from the at least one pass-through component channel; a generation module 604, configured to merge the target pass-through component channel and the corrected color component channels to generate a corrected image in the first color space; a conversion module 605, configured to convert the corrected image from the first color space to a second color space to obtain a corrected image in the second color space.

[0114] In this embodiment, the restoration module 601 can perform image restoration processing on the original image data to obtain a restored image including at least one pass-through component channel. Among them, the original image data is data collected by an image sensor as Figure 1 shown. The original image data includes a preset number of groups of pixel data, and each group of pixel data includes at least one pass-through component data.

[0115] As an example, the image sensor includes an RCCC color filter array, that is, each 2×2-sized array unit in the color filter array includes one R filter and three C (Clarify) filters. The image sensor senses the light passing through these four filters to obtain an electrical signal, and then obtains a corresponding set of pixel data. This set of pixel data includes three through-transmission component data.

[0116] The original image data is single-channel data, and the pixel data it includes is distributed within this single channel. The single-channel original image data cannot be normally displayed in the form of an image. The restoration module 601 can convert the original image data into an image including multiple channels that can be normally displayed. These channels include at least one through-transmission component channel. The restoration module 601 can perform image restoration processing on the original image data according to, for example, an image signal processing method (ISP, Image Signal Processing). For example, methods included in the ISP method such as white balance processing method and demosaicing processing method can be used to perform image restoration processing on the original image data.

[0117] In this embodiment, the correction module 602 can perform color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel. Among them, the preset image channel can be all the channels included in the restored image, or on the basis of all the channels included in the restored image, at least one channel can be added, and each data in the added channel is a constant (for example, 1).

[0118] The execution process of the usual color correction is to compare the original image captured by the camera with a reference image in advance, so as to calculate and obtain a color correction matrix (CCM, Color Correction Matrix). Using the obtained color correction matrix, the restored image can be color-corrected to obtain an image close to the color of the real scene. In this embodiment, since the restored image is not an RGB color image, a color correction matrix can be set in advance. For each pixel in the restored image, the respective channel components included in the pixel form a vector. Multiply the color correction matrix by this vector to obtain the corrected color component channel corresponding to the pixel.

[0119] In this embodiment, the determination module 603 may determine a target pass-through component channel from at least one pass-through component channel. Wherein, the target pass-through component channel may be any one of the at least one pass-through component channel. As an example, the restored image includes three channels, namely R, C1, and C2. The C1 channel may be set as the target pass-through component channel, or the C2 channel may be set as the target pass-through component channel. Optionally, a pass-through component channel extraction matrix may be preset, and the matrix is multiplied by a vector formed by each component included in each pixel of the restored image to obtain the target pass-through component corresponding to each pixel. The set of target pass-through components of each pixel is the target pass-through component channel. Optionally, the channel corresponding to the specified target channel identifier may also be determined as the target pass-through component channel.

[0120] In this embodiment, the generation module 604 may merge the target pass-through component channel and the corrected color component channel to generate a corrected image in the first color space. Generally, the target pass-through component channel and the corrected color component channel may be directly superimposed to generate a corrected image including the target pass-through component channel and the corrected color component channel.

[0121] As an example, if the target pass-through component channel is C1 and the corrected color component channels include G' and B', the corrected image may include three channels, namely C1, G', and B'. It should be noted that G' and B' here represent color component channels obtained by a method similar to the color correction of an RGB image, and have nothing to do with the actual colors represented by the color component channels.

[0122] In this embodiment, the conversion module 605 may convert the corrected image from the first color space to the second color space to obtain a corrected image in the second color space.

[0123] In a general ISP method based on a color image, the color space of the corrected image is the RGB color space, and it is necessary to convert the corrected image from the RGB color space to the YUV color space. In this embodiment, the first color space of the corrected image is the above C1G'B' color space, and the second color space is the YUV color space. A color space conversion matrix may be set according to a method similar to converting the RGB color space to the YUV color space to convert the C1G'B' color space to the YUV color space.

[0124] Refer to Figure 7 , Figure 7 which is a schematic structural diagram of an image generation device provided in another exemplary embodiment of the present disclosure.

[0125] In some alternative implementation manners, the correction module 602 is further configured to: perform color correction on the channel components included in the restored image based on a preset first correction parameter set, to obtain the corrected color component channels.

[0126] In some alternative implementation manners, the determination module 603 is further configured to: perform channel extraction on the channel components included in the restored image based on a preset second correction parameter set, to obtain the target pass-through component channel included in the at least one pass-through component channel.

[0127] In some alternative implementation manners, the correction module 602 includes: an addition unit 6021, configured to add a color offset component channel based on the original component channels included in the restored image; a correction unit 6022, configured to perform color correction on the original channels included in the restored image based on the offset term elements and ratio term elements included in the first correction parameter set, to obtain color offset data and color base data; a generation unit 6023, configured to generate the corrected color component channels based on the color offset data and the color base data.

[0128] In some alternative implementation manners, the conversion module 605 includes: a conversion unit 6051, configured to perform color space conversion on the corrected color component channels of the corrected image, to obtain the converted color component channels; an extraction unit 6052, configured to extract the target pass-through component channel from the corrected image; a merging unit 6053, configured to merge the target pass-through component channel and the converted color component channels to generate the corrected image in a second color space.

[0129] In some alternative implementation manners, the conversion unit 6051 is further configured to: perform color space conversion on the channel components included in the corrected image based on a first parameter subset included in a preset color space conversion parameter set, to obtain the converted color component channels.

[0130] In some alternative implementation manners, the extraction unit 6052 is further configured to: perform channel extraction on the channel components included in the corrected image based on a second parameter subset included in the color space conversion parameter set, to obtain the target pass-through component channel.

[0131] The image generation device provided in the above embodiments of the present disclosure performs image restoration processing on the original image data to obtain a restored image including at least one pass-through component channel, then performs color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel, then determines a target pass-through component channel from the at least one pass-through component channel, combines the target pass-through component channel and the corrected color component channel to generate a corrected image in the first color space, and finally converts the corrected image from the first color space to the second color space to obtain a corrected image in the second color space. When performing color correction in the embodiments of the present disclosure, other color component channels do not participate in the extraction process of the target pass-through component, that is, the full value of the target pass-through component is retained, so that the brightness value of the finally obtained image does not suffer losses, and the color noise brought by other color values is reduced, thereby effectively improving the clarity of the image and displaying richer detail information.

[0132] Exemplary Electronic Device

[0133] Next, reference will be made to Figure 8 to describe an electronic device according to an embodiment of the present disclosure. The electronic device may be any one or both of a terminal device 101 and a server 103 as shown in Figure 1 or a stand-alone device independent of them, and the stand-alone device may communicate with the terminal device 101 and the server 103 to receive the input signals collected from them.

[0134] Figure 8 FIG. shows a block diagram of an electronic device according to an embodiment of the present disclosure.

[0135] As shown in Figure 8 the electronic device 800 includes one or more processors 801 and a memory 802.

[0136] The processor 801 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 800 to perform desired functions.

[0137] The memory 802 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 801 may run the program instructions to implement the image generation method of various embodiments of the present disclosure above and / or other desired functions. Various contents such as original image data, corrected images, etc. may also be stored in the computer-readable storage media.

[0138] In one example, the electronic device 800 may further include: an input device 803 and an output device 804, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0139] For example, when the electronic device is the terminal device 101 or the server 103, the input device 803 may be devices such as an image sensor, a mouse, a keyboard, etc., for inputting original image data, various commands, etc. When the electronic device is a stand-alone device, the input device 803 may be a communication network connector for receiving the input original image data, various commands, etc. from the terminal device 101 and the server 103.

[0140] The output device 804 may output various information to the outside, including the generated corrected image. The output device 804 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0141] Of course, for simplicity, Figure 8 only some of the components related to the present disclosure in the electronic device 800 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 800 may further include any other appropriate components.

[0142] Exemplary Computer Program Product and Computer Readable Storage Medium

[0143] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, and the computer program instructions, when run by a processor, cause the processor to execute the steps in the image generation method according to various embodiments of the present disclosure described in the "Exemplary Method" section above of this specification.

[0144] The computer program product can be written in any combination of one or more programming languages for executing the program code of the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0145] In addition, an embodiment of the present disclosure can also be a computer-readable storage medium storing computer program instructions, which, when run by a processor, cause the processor to execute the steps in the image generation method according to various embodiments of the present disclosure described in the "Exemplary Method" section above of this specification.

[0146] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0147] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for illustrative purposes and for ease of understanding, rather than limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details to implement.

[0148] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

[0149] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the phrase "and / or", and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0150] The methods and apparatuses of this disclosure can be implemented in many ways. For example, the methods and apparatuses of this disclosure can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of the steps for the methods is for illustration purposes only, and the steps of the methods of this disclosure are not limited to the specific order described above, unless otherwise specifically stated. In addition, in some embodiments, this disclosure can also be implemented as a program recorded in a recording medium, and these programs include machine-readable instructions for implementing the methods according to this disclosure. Therefore, this disclosure also covers the recording medium storing the programs for executing the methods according to this disclosure.

[0151] It should also be noted that in the apparatuses, equipment, and methods of this disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this disclosure.

[0152] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0153] The above description has been given for purposes of illustration and description. In addition, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. An image generation method, comprising: Performing image restoration processing on original image data to obtain a restored image including at least one pass-through component channel, where the at least one pass-through component channel is a channel component generated based on at least one pass-through component data in any set of pixel data included in the original image data; Performing color correction on the image data in a preset image channel of the restored image to obtain a corrected color component channel, where the preset image channel includes all channels included in the restored image; Determining a target pass-through component channel from the at least one pass-through component channel; Merging the target pass-through component channel and the corrected color component channel to generate a corrected image in a first color space; Converting the corrected image from the first color space to a second color space to obtain a corrected image in the second color space.

2. The method according to claim 1, wherein The performing color correction on the image data in a preset image channel of the restored image to obtain a corrected color component channel includes: Performing color correction on the channel components included in the restored image based on a preset first correction parameter set to obtain the corrected color component channel.

3. The method according to claim 2, wherein The determining a target pass-through component channel from the at least one pass-through component channel includes: Performing channel extraction on the channel components included in the restored image based on a preset second correction parameter set to obtain the target pass-through component channel included in the at least one pass-through component channel.

4. The method according to claim 2, wherein, The performing color correction on the channel components included in the restored image based on a preset first correction parameter set to obtain the corrected color component channel includes: Adding a color offset component channel based on the original component channels included in the restored image; Performing color correction on the original channels included in the restored image based on the offset term elements and ratio term elements included in the first correction parameter set to obtain color offset data and color base data; Generating the corrected color component channel based on the color offset data and the color base data.

5. The method according to claim 1, wherein, The converting the corrected image from the first color space to a second color space to obtain a corrected image in the second color space includes: Performing color space conversion on the corrected color component channel of the corrected image to obtain a converted color component channel; Extracting the target pass-through component channel from the corrected image; Merging the target pass-through component channel and the converted color component channel to generate a corrected image in the second color space.

6. The method according to claim 5, wherein, The performing color space conversion on the corrected color component channel of the corrected image to obtain a converted color component channel includes: Performing color space conversion on the channel components included in the corrected image based on a first parameter subset included in a preset color space conversion parameter set to obtain the converted color component channel.

7. The method according to claim 6, wherein The extracting the target pass-through component channel from the corrected image includes: Performing channel extraction on the channel components included in the corrected image based on a second parameter subset included in the color space conversion parameter set to obtain the target pass-through component channel.

8. An image generation device, comprising: a restoration module configured to perform image restoration processing on original image data to obtain a restored image including at least one pass-through component channel, where the at least one pass-through component channel is a channel component generated based on at least one pass-through component data in any set of pixel data included in the original image data; a correction module configured to perform color correction on the image data in a preset image channel in the restored image to obtain a corrected color component channel, where the preset image channel includes all channels included in the restored image; a determination module configured to determine a target pass-through component channel from the at least one pass-through component channel; a generation module configured to merge the target pass-through component channel and the corrected color component channel to generate a corrected image in a first color space; a conversion module configured to convert the corrected image from the first color space to a second color space to obtain a corrected image in the second color space.

9. A computer-readable storage medium storing a computer program, where the computer program is configured to be executed by a processor to implement the method according to any one of claims 1-7 above.

10. An electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method according to any one of claims 1-7 above.

Images