Image signal processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a method and device for processing image signals, an electronic device and a storage medium. The method comprises: determining a reference parameter item according to first channel information of a first channel in a preset photosensitive unit; the reference parameter item is a second derivative of the first channel information; determining the reliability of the reference parameter item; in response to the reference parameter item being unreliable, determining second channel information of a second channel in the preset photosensitive unit according to a correction parameter item; the correction parameter item is used to represent the second derivative of the second channel. Using the method of the present disclosure, the reliability of the reference parameter item corresponding to the first channel information can be investigated, and when the reference parameter item is unreliable, the second channel information is determined in combination with the corrected parameter item. The smoothness of the image texture edge is improved, and the problem of unsmooth image texture edge caused by the unreliable reference parameter item is solved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to a method, apparatus, electronic device, and storage medium for processing image signals. Background Technology

[0002] In image processing, the photosensitive device of an electronic device needs to receive red (R), green (G), and blue (B) color information from light, and then synthesize a color image based on these three colors. Under the action of a color filter array, each photosensitive unit in the photosensitive device receives light of one color, and the resulting photosensitive data is the photosensitive data corresponding to that color. For any given photosensitive unit, after it senses one color data, the color data of the remaining color channels of that unit need to be interpolated to obtain the three color data for synthesizing a color image.

[0003] In related technologies, when using color interpolation algorithms for color interpolation, there is a problem of inconsistency between the interpolation resolution and the smoothness of the obtained image texture edges. For example, pursuing high resolution may result in uneven edges, or ensuring smooth edges may result in a decrease in resolution. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image signal processing method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of the present disclosure, a method for processing image signals is provided, comprising:

[0006] Based on the first channel information of the first channel in the preset photosensitive unit, a reference parameter item is determined; the reference parameter item is the second derivative of the first channel information.

[0007] Determine the confidence level of the reference parameter item;

[0008] In response to the unreliability of the reference parameter, the second channel information of the second channel in the preset photosensitive unit is determined according to the correction parameter; the correction parameter is used to characterize the second derivative of the second channel.

[0009] Optionally, the reference parameter item includes a first sub-item and a second sub-item;

[0010] The step of determining reference parameter items based on the first channel information of the first channel in the preset photosensitive unit includes:

[0011] Based on the first channel information, determine the second derivative of the first channel in the first direction, and use it as the first sub-item;

[0012] Based on the information of the first channel, determine the second derivative of the first channel in the second direction, and use it as the second sub-item;

[0013] Wherein, the first direction and the second direction are perpendicular, and the first direction and the second direction are the arrangement directions of the photosensitive units in the photosensitive device array.

[0014] Optionally, determining the confidence level of the reference parameter item includes:

[0015] The credibility of the first sub-item is determined based on the first sub-item and the first threshold range;

[0016] The credibility of the second sub-item is determined based on the second sub-item and the second threshold range.

[0017] Optionally, the method includes: determining a target threshold range:

[0018] In two photosensitive units adjacent to the preset photosensitive unit in the target direction, the second derivative of the second channel of each photosensitive unit in the target direction is determined respectively.

[0019] The target threshold range is determined based on the second derivative of the second channel in the target direction.

[0020] Wherein, the target threshold range is either the first threshold range or the second threshold range, and the target direction is either the first direction or the second direction.

[0021] Optionally, in response to the reference parameter item being unreliable, the method includes:

[0022] Based on the second derivative of the second channel of each of the two photosensitive units adjacent to the preset photosensitive unit in the target direction in the target direction, the correction parameter item corresponding to the second channel in the target direction is determined.

[0023] Optionally, the method includes:

[0024] In response to the reliability of the reference parameter item, the second channel information is determined based on the reference parameter item.

[0025] According to a second aspect of the present disclosure, an image signal processing apparatus is provided, comprising:

[0026] The first determining module is used to determine a reference parameter item based on the first channel information of the first channel in the preset photosensitive unit; the reference parameter item is the second derivative of the first channel information.

[0027] The second determining module is used to determine the confidence level of the reference parameter item;

[0028] The third determining module is used to determine the second channel information of the second channel in the preset photosensitive unit based on the correction parameter item in response to the unreliability of the reference parameter item; the correction parameter item is used to characterize the second derivative of the second channel.

[0029] Optionally, the reference parameter item includes a first sub-item and a second sub-item;

[0030] The first determining module is used for:

[0031] Based on the first channel information, determine the second derivative of the first channel in the first direction, and use it as the first sub-item;

[0032] Based on the information of the first channel, determine the second derivative of the first channel in the second direction, and use it as the second sub-item;

[0033] Wherein, the first direction and the second direction are perpendicular, and the first direction and the second direction are the arrangement directions of the photosensitive units in the photosensitive device array.

[0034] Optionally, the second determining module is used to:

[0035] The credibility of the first sub-item is determined based on the first sub-item and the first threshold range;

[0036] The credibility of the second sub-item is determined based on the second sub-item and the second threshold range.

[0037] Optionally, the device includes: a fourth determining module, configured to determine a target threshold range, the fourth determining module being configured to:

[0038] In two photosensitive units adjacent to the preset photosensitive unit in the target direction, the second derivative of the second channel of each photosensitive unit in the target direction is determined respectively.

[0039] The target threshold range is determined based on the second derivative of the second channel in the target direction.

[0040] Wherein, the target threshold range is either a first threshold range or a second threshold range, and the target direction is either the first direction or the second direction.

[0041] Optionally, in response to the unreliability of the reference parameter item, the third determining module is used to:

[0042] Based on the second derivative of the second channel of each of the two photosensitive units adjacent to the preset photosensitive unit in the target direction in the target direction, the correction parameter item corresponding to the second channel in the target direction is determined.

[0043] Optionally, the third determining module is used to:

[0044] In response to the reliability of the reference parameter item, the second channel information is determined based on the reference parameter item.

[0045] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0046] processor;

[0047] Memory used to store the processor's executable instructions;

[0048] The processor is configured to perform the image signal processing method as described in any of the preceding claims.

[0049] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image signal processing method as described in any of the preceding claims.

[0050] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: Using the method of this disclosure, the reliability of the reference parameter items corresponding to the first channel information can be examined. When the reference parameter items are unreliable, the second channel information is determined by combining the corrected parameter items. This improves the smoothness of image texture edges and alleviates the problem of uneven image texture edges caused by unreliable reference parameter items.

[0051] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0053] Figure 1 This is a schematic diagram of image texture edges in related technologies.

[0054] Figure 2 This is a flowchart illustrating a method according to an exemplary embodiment.

[0055] Figure 3 This is a flowchart illustrating a method according to an exemplary embodiment.

[0056] Figure 4 This is a flowchart illustrating a method according to an exemplary embodiment.

[0057] Figure 5 This is a flowchart illustrating a method according to an exemplary embodiment.

[0058] Figure 6 This is a schematic diagram of a photosensitive device array according to an exemplary embodiment.

[0059] Figure 7 This is a schematic diagram of image texture edges according to an exemplary embodiment.

[0060] Figure 8 This is a schematic diagram illustrating the difference between image texture edges and related technologies according to an exemplary embodiment.

[0061] Figure 9 This is a block diagram of an apparatus according to an exemplary embodiment.

[0062] Figure 10 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0064] In the image processing process, the photosensitive device of the electronic device needs to receive the red (R), green (G), and blue (B) color information in the light, and then synthesize a color image based on the three color information.

[0065] Since photosensitive devices typically cannot directly identify colors, electronic devices usually have color filter arrays (CFAs) positioned above the photosensitive device. This allows each photosensitive unit in the device to receive light of a specific color, and the resulting photosensitive data is the photosensitive data corresponding to that color. For any given photosensitive unit, after sensing one color, the color data of the remaining color channels of that unit need to be interpolated to obtain three color data sets for synthesizing a color image.

[0066] Color interpolation algorithms typically employ Demosaic technology. Demosaic technology is based on a color filter array in the form of a Bayer matrix. It interpolates the data of the other two color channels in each photosensitive unit based on the single color channel data received. This yields the interpolated data for all photosensitive units, which are then combined with the received data to synthesize a color image.

[0067] In related technologies, when using Demosaic technology for color interpolation, there is a problem of inconsistency between the interpolation resolution and the smoothness of the obtained image texture edges. For example, pursuing high resolution may result in uneven edges, or ensuring smooth edges may result in a decrease in resolution.

[0068] The main reasons for the aforementioned problems in related technologies include:

[0069] When interpolating to determine the unknown color channel data for each photosensitive unit, the second derivative of the known color channel data of that photosensitive unit is often directly used to replace the second derivative of the unknown color channel data. However, when the second derivative of the known color channel data is unreliable—for example, when the second derivative of the unknown color channel data to be interpolated is inconsistent with the second derivative of the known color channel data—using the second derivative of the known color channel data as a substitute will result in interpolation errors. This can lead to defects in the image, such as uneven lines, broken lines, or bright / dark spots, and consequently, abrupt points at the edges of the image texture, causing edge smoothing problems. Figure 1 As shown. However, if a low-pass filtering smoothing algorithm is used to improve smoothness, it can easily lead to a decrease in resolution.

[0070] This disclosure proposes an image signal processing method. Based on first channel information of a first channel in a preset photosensitive unit, a reference parameter is determined. The reliability of the reference parameter is then determined. In response to an unreliable reference parameter, second channel information of a second channel in the preset photosensitive unit is determined based on a correction parameter. Using this method, the reliability of the reference parameter corresponding to the first channel information can be examined. When the reference parameter is unreliable, the second channel information is determined by combining it with the corrected parameter. This improves the smoothness of image texture edges and mitigates the problem of uneven image texture edges caused by unreliable reference parameter.

[0071] In one exemplary embodiment, the image signal processing method of this embodiment can be applied to an electronic device, such as a mobile phone, camera, laptop, tablet computer, smart wearable device, or other smart device with image signal processing (ISP) capabilities. The electronic device includes a photosensitive element array and a Bayer-type color filter array. Under the action of the color filter array, each photosensitive unit in the photosensitive element array can sense environmental data of a color.

[0072] like Figure 2 As shown, the processing method in this embodiment includes the following steps:

[0073] S110. Determine the reference parameter item based on the first channel information of the first channel in the preset photosensitive unit.

[0074] S120. Determine the reliability of the reference parameter item.

[0075] S130. In response to the unreliability of the reference parameter item, determine the second channel information of the second channel in the preset photosensitive unit based on the correction parameter item.

[0076] In step S110, the preset photosensitive unit can be any one of the photosensitive units in the photosensitive device array. The preset photosensitive unit may include a first channel, a second channel, and a third channel. Under the action of the Bayer-type color filter array, after each light reception, one channel of the preset photosensitive unit is activated and receives photosensitive data of one color.

[0077] Taking the red light that can be sensed at the preset photosensitive unit as an example, the first channel is used to detect the photosensitive data of the transmitted red light as the first channel information, while the channel information of the second and third channels needs to be determined by interpolation algorithm.

[0078] In this step, the reference parameter is the second derivative of the first channel information. It can be calculated based on the Hamilton-Adams (HA) algorithm and the Low Cost Edge Sensing for High Quality Demosaicking (LED) algorithm in Demosaic technology, as well as the first channel information detected by the preset photosensitive unit, as detailed in the following embodiments.

[0079] In step S120, after determining the reference parameter item, its credibility is verified.

[0080] For example, it could verify whether a reference parameter is within a reasonable threshold range; if so, the reference parameter is reliable; otherwise, it is unreliable. Alternatively, it could verify whether the difference between the reference parameter and the threshold is within a preset range; if so, the reference parameter is reliable; otherwise, it is unreliable.

[0081] In step S130, if the reference parameter is unreliable and the second channel information is still calculated using the reference parameter, it will cause the image texture edges to be uneven.

[0082] In this step, if the reference parameter is unreliable, the second channel information will be calculated based on the corrected calibration parameter. The calibration parameter is used to characterize the second derivative of the second channel. The calibration parameter can be calculated and determined based on the photosensitive data of photosensitive units adjacent to the preset photosensitive unit, such as by correcting the mean of the channel information of adjacent photosensitive units, or by correcting the channel information of adjacent photosensitive units in a weighted manner with different weights.

[0083] Understandably, the method for determining the third channel information is similar to the interpolation method for the second channel information. Therefore, the embodiments of this disclosure are all described using the determination of the second channel information as an example. Furthermore, since human vision is most sensitive to green, in a Bayer-type color filter array, the number of G units (transparent green) is often twice that of R units (transparent red) or B units (transparent blue). Based on this characteristic, the following embodiments of this disclosure use the second channel of the photosensitive unit as the G channel, and the interpolated second channel information is used to characterize the G color channel data in the preset photosensitive unit as an example for illustrative description. The interpolation data for other channels can be calculated according to the same principle.

[0084] In one exemplary embodiment, the reference parameter item includes a first sub-item and a second sub-item. For example... Figure 3 As shown, step S110 in this embodiment may include the following steps:

[0085] S1101. Based on the information of the first channel, determine the second derivative of the first channel in the first direction, and use it as the first sub-term.

[0086] S1102. Based on the information of the first channel, determine the second derivative of the first channel in the second direction, and use it as the second sub-item.

[0087] Wherein, the first direction and the second direction are perpendicular, and the first direction and the second direction are the arrangement directions of the photosensitive units in the photosensitive device array. For example, the arrangement of the photosensitive device array can be referenced. Figure 6 As shown, the first direction can be horizontal and the second direction can be vertical.

[0088] In step S1101, combining the HA algorithm and the LED algorithm, the second derivative corresponding to any channel of any photosensitive unit in the first direction (horizontal direction) is calculated as follows:

[0089]

[0090] Where (m,n) represents the position or coordinates of the photosensitive unit in the array, and in this embodiment, it is used to refer to the photosensitive unit at that position. bayer(m,n) represents the channel data detected by the photosensitive unit (generally, one channel receives one type of color data).

[0091] Combining the second derivative formula mentioned above, in this step, we combine... Figure 6 The 5×5 photosensitive array shown is illustrated using a preset photosensitive unit that senses red light and the preset photosensitive unit is the center of the array (i, j). The first channel detects red light data, and the information obtained from the first channel is denoted as R(i, j).

[0092] The second derivative (first sub-term) of the first channel in the first direction (horizontal direction) can be calculated as follows:

[0093]

[0094] In this formula, R(i,j-2) represents: in the horizontal direction, the second photosensitive unit to the right of the preset photosensitive unit, and the first channel information detected; considering the characteristics of the Bayer arrangement, the photosensitive unit (i,j-2) can perceive light of the same color as the preset photosensitive unit. Similarly, R(i,j+2) represents: in the horizontal direction, the second photosensitive unit to the left of the preset photosensitive unit, and the first channel information detected; the light color perceived by the photosensitive unit (i,j+2) is also the same as that of the preset photosensitive unit.

[0095] In this step, the first channel information detected by two photosensitive units on the left and right sides of the preset photosensitive unit, which perceive the same color, namely R(i,j-2) and R(i,j+2), and combined with the first channel information R(i,j) detected by the preset photosensitive unit itself, can be calculated to obtain the second derivative in the horizontal direction. That is, the first sub-item.

[0096] In step S1102, still combined Figure 6 As shown, based on the same principle as step S1101, the second derivative (second sub-term) of the first channel in the second direction (vertical direction) can be calculated as follows:

[0097]

[0098] In this formula, R(i-2,j) represents the first channel information detected by the second photosensitive unit above the preset photosensitive unit in the vertical direction. Considering the characteristics of the Bayer arrangement, photosensitive unit (i-2,j) can perceive light of the same color as the preset photosensitive unit. Similarly, R(i+2,j) represents the first channel information detected by the second photosensitive unit below the preset photosensitive unit in the vertical direction; the color of light perceived by photosensitive unit (i+2,j) is also the same as that perceived by the preset photosensitive unit.

[0099] In this step, the first channel information detected by two photosensitive units on the upper and lower sides of the preset photosensitive unit, which perceive the same color, namely R(i-2,j) and R(i+2,j), and combined with the first channel information R(i,j) detected by the preset photosensitive unit itself, can be calculated to obtain the second derivative in the vertical direction. That is, the second sub-item.

[0100] In one exemplary embodiment, such as Figure 4 As shown, step S120 may include the following steps:

[0101] S1201. Determine the credibility of the first sub-item based on the first sub-item and the first threshold range.

[0102] S1202. Determine the credibility of the second sub-item based on the second sub-item and the second threshold range.

[0103] In step S1201, the first threshold range may be pre-stored in the electronic device, with each photosensitive unit having a corresponding first threshold range for its photosensitive data, or it may be determined in real time according to an algorithm.

[0104] In this step, the response is to the first sub-item. If the first sub-item is within the first threshold range, the first sub-item is determined to be trustworthy; if the first sub-item is outside the first threshold range, the first sub-item is determined to be untrustworthy.

[0105] In step S1202, the second threshold range may be pre-stored in the electronic device, with each photosensitive unit having a corresponding second threshold range for its photosensitive data, or it may be determined in real time according to an algorithm.

[0106] In this step, the response is to the second sub-item. If the second sub-item is within the second threshold range, the second sub-item is determined to be trustworthy; if the second sub-item is within the second threshold range, the second sub-item is determined to be untrustworthy.

[0107] In this embodiment, the first and second sub-items of the reference parameter item in both directions are judged separately to determine whether both the first and second sub-items are reliable. This facilitates the independent adjustment of the first or second sub-item in subsequent calculations, further improving the accuracy of the calculation process.

[0108] In an exemplary embodiment, the method of this embodiment further includes: S200, determining a target threshold range.

[0109] The target threshold range is either a first threshold range or a second threshold range. For example... Figure 5 As shown, step S200 in this embodiment may include the following steps:

[0110] S201. In the two photosensitive units adjacent to the preset photosensitive unit in the target direction, determine the second derivative of the second channel of each photosensitive unit in the target direction.

[0111] S202. Determine the target threshold range based on the second derivative of the second channel in the target direction.

[0112] The target direction is either a first direction or a second direction. There is a one-to-one correspondence between the target direction and the target threshold range: when the first threshold range is determined, the target direction is, for example, the first direction; when the second threshold range is determined, the target direction is, for example, the second direction. In this embodiment, the example of the target threshold range being the first threshold range and the target direction being the first direction will be used for explanation.

[0113] In step S201, combined Figure 5 As shown, the preset photosensitive unit (i, j) is adjacent to the photosensitive units on its left and right in the first direction, namely the photosensitive unit (i, j-1) to its left and the photosensitive unit (i, j+1) to its right. These two photosensitive units perceive the same color of light, but the color is different from that perceived by the preset photosensitive unit.

[0114] Based on the characteristics of the Bayer arrangement, if the light color sensed by these two photosensitive units is, for example, green (G), then the second channel information detected by photosensitive unit (i,j-1) is denoted as G(i,j-1), and the second channel information detected by photosensitive unit (i,j+1) is denoted as G(i,j+1).

[0115] In this step, in these two adjacent photosensitive units, the second derivative of the second channel of each photosensitive unit in the first direction is determined, denoted as . as well as The solution method can be found in the above formula for calculating the second derivative.

[0116] In step S202, in conjunction with step S201, the second derivatives corresponding to the two photosensitive units adjacent to the preset photosensitive unit (i, j) are determined: as well as

[0117] The first threshold range can be set as follows: the interval formed by the second derivatives of the two photosensitive units adjacent to the preset photosensitive unit. Let the minimum value A of the two second derivatives be denoted as: Let B be the maximum value of the two second derivatives, i.e.: The range of the first threshold is, for example, [A, B].

[0118] Based on this embodiment, when step S1201 is executed, it is necessary to determine whether the first sub-item is within the first threshold range, or to determine the relationship between the first sub-item and the minimum or maximum value within the first threshold range.

[0119] In one example or, (Where the coefficient k1 is less than 1), it indicates that the first sub-item is not within the first threshold range, thus determining the first sub-item. Unreliable.

[0120] In another example, or, (Where the coefficient k2 is greater than 1), it indicates that the first sub-item is not within the first threshold range, thus determining the first sub-item. Unreliable.

[0121] Understandably, based on the method of this embodiment, a second threshold range can also be determined, and step S1202 can be executed to verify the second sub-item using the second threshold range. Credibility.

[0122] In this embodiment, the reliability of the reference parameter is verified by using the second derivative of the G-channel data of the two photosensitive units adjacent to the preset photosensitive unit as a condition constraint, thereby ensuring the accuracy of the subsequent interpolation data.

[0123] In one exemplary embodiment, the method of this embodiment further includes:

[0124] S300. Based on the second derivative of the second channel of each of the two photosensitive units adjacent to the preset photosensitive unit in the target direction, determine the correction parameter item corresponding to the second channel in the target direction.

[0125] This step may be performed in response to an unreliable reference parameter item, or it may be performed in advance and the calibration parameter item may be pre-stored in the electronic device. An unreliable reference parameter item may include either a first sub-item or a second sub-item being unreliable.

[0126] In this step, the target direction can be either a first direction or a second direction; this embodiment will still use the first direction as an example for explanation. Referring to the above embodiment, the second derivatives of the two photosensitive units adjacent to the preset photosensitive unit in the first direction, determined in step S201, are respectively... as well as

[0127] In this step, the correction parameter for the second channel in the first direction can be:

[0128]

[0129] Similarly, the correction parameter for the second channel in the second direction can be:

[0130]

[0131] In this embodiment, the correction parameter is characterized by the average of the second derivatives of the G channel data of the two photosensitive units adjacent to the preset photosensitive unit, thus avoiding the use of unreliable reference parameter. This improves the accuracy of interpolation without reducing the interpolation resolution, thereby improving edge smoothness.

[0132] Based on the above embodiments, in order to facilitate understanding of this disclosure, a complete example will be given below to describe the implementation steps of determining the second channel information (e.g., G color channel data) of the preset photosensitive unit by interpolation method. The third channel information of the preset photosensitive unit is determined by interpolation method based on the same principle, and will not be described again in the example here.

[0133] In this embodiment, the second channel information is determined by interpolation based on the HA algorithm and the LED algorithm, combined with... Figure 6 In the example Bayer arrangement, the second channel information of the preset photosensitive unit (i,j) is denoted as G(i,j), which can be determined in the following way:

[0134] G(i,j)=w h *G h (i,j)+(1-w h )*G v (i,j)

[0135] Among them, w h G represents the weight in the first direction. h (i,j) represents the second channel information corresponding to the preset photosensitive unit in the first direction. (1-w h G represents the weight in the second direction. v (i,j) represents the second channel information corresponding to the preset photosensitive unit in the second direction.

[0136] The weight w in the first directionh The calculation method is as follows:

[0137]

[0138] In calculating weight w h During the process, the fluctuation V of the photosensitive data of the photosensitive units arranged along the first direction is involved. h (i,j):

[0139]

[0140] And the fluctuation V of the photosensitive data of the photosensitive units arranged along the second direction. v (i,j):

[0141]

[0142] The weight w in the second direction v The calculation method is as follows:

[0143] w v (i,j)=1-w h (i,j)

[0144] G h The calculation method for (i,j) is as follows:

[0145]

[0146] G v The calculation method for (i,j) is as follows:

[0147]

[0148] The meaning of each term in the above formula can be found in the description in the foregoing embodiments, and will not be repeated here. For example... The second derivative of the second channel (G channel) of the preset photosensitive unit (i,j) in the first direction (subscript h) is represented. For example, (i,j-1) represents the photosensitive unit that is adjacent to the preset photosensitive unit (i,j) and located to the left of the preset photosensitive unit in the first direction.

[0149] As can be seen from the above formula, if interpolation is needed to determine G(i,j), then G must first be determined. h (i,j) and G v (i,j).

[0150] In G h (i,j) and G v In the calculation of (i,j), the relevant techniques are based on reference parameter terms. Direct replacement Calculation. Therefore, if the interpolated data is unreliable, it can cause uneven texture at image edges, such as... Figure 1 As shown.

[0151] In this embodiment of the disclosure, the reference parameter item will be used. The reliability of the calculation determines whether to use a reference parameter or a correction parameter. The reference parameter is the second derivative corresponding to the first channel information of the preset photosensitive unit.

[0152] In step S110, the reference parameter item is determined. (For example, including: the first sub-item) Second sub-item After that, the confidence level of the reference parameter item is determined through step S120.

[0153] Combining steps S1201 and S1202, it can be seen that the first sub-item can be determined respectively. Second sub-item The credibility of the first item will be explained here by taking the verification of the credibility of the first sub-item within the first threshold range [A, B] as an example:

[0154] First Sub-item If the two terms are within the first threshold range, they are considered to be consistent, and the first sub-term is reliable and can be used to characterize G. h (i,j). Then use the first sub-item. replace To calculate G h (i,j).

[0155] And when If the value is outside the threshold range (e.g., less than or much less than A, or greater than or much greater than B), then the first sub-item is unreliable and cannot represent G. h (i,j). According to step S300, the correction parameter item for the first direction is determined as follows: Using this correction parameter as To calculate G h (i,j).

[0156] Similarly, combining the second sub-item The credibility of the second sub-item is determined when the second sub-item is within the second threshold range. for Calculate G v (i,j). When the second sub-item is not within the range of the second threshold, determine the correction parameter term for the second direction. Using this correction parameter as

[0157] Then, according to G h (i,j) and G v (i,j) determines the second channel information G(i,j) of the interpolation.

[0158] Based on the above principles, the third channel information of the preset photosensitive unit can also be determined by interpolation, or the unknown color channel information can be determined by using the known color channel information of other photosensitive units, such as the interpolation determination of R or B channel information.

[0159] After interpolation optimization in the embodiments of this disclosure, the obtained image effect is as follows: Figure 7 As shown, the differences between the optimized effect of this embodiment and related technologies are as follows: Figure 8 As shown in the figure, a comparison with related technologies reveals that the optimized image exhibits significantly fewer abrupt edges in the texture, increased smoothness, and no decrease in resolution.

[0160] The image signal processing method of this disclosure, based on the HA algorithm and the LED algorithm, can verify the reliability of reference parameter terms and determine correction parameter terms for calculation when they are unreliable, thereby improving the accuracy of interpolation in determining unknown color channel information. The method is simple and easy to implement, and hardware-friendly; at the same time, it can greatly improve the quality of the interpolated image, improve the smoothness of texture edges, and enhance the robustness of flat areas to noise.

[0161] In one exemplary embodiment, this disclosure also provides an image signal processing apparatus, such as... Figure 9 As shown, the apparatus of this embodiment includes: a first determining module 110, a second determining module 120, and a third determining module 130. The apparatus of this embodiment is used to implement... Figure 1 The method is illustrated. The first determining module 110 is used to determine a reference parameter item based on the first channel information of the first channel in the preset photosensitive unit; the reference parameter item is the second derivative of the first channel information. The second determining module 120 is used to determine the reliability of the reference parameter item. The third determining module 130 is used to determine the second channel information of the second channel in the preset photosensitive unit based on a correction parameter item in response to the unreliability of the reference parameter item; the correction parameter item is used to characterize the second derivative of the second channel. In this embodiment, the third determining module 130 is used to: determine the second channel information based on the reference parameter item in response to the reliability of the reference parameter item.

[0162] In one exemplary embodiment, the reference parameter item includes a first sub-item and a second sub-item. (Continuing with...) Figure 9 As shown, in this embodiment, the first determining module 110 is used to: determine the second derivative of the first channel in the first direction according to the first channel information, as a first sub-item; and determine the second derivative of the first channel in the second direction according to the first channel information, as a second sub-item; wherein the first direction and the second direction are perpendicular, and the first direction and the second direction are the arrangement directions of the photosensitive units in the photosensitive device array.

[0163] In one exemplary embodiment, reference is still made to... Figure 9As shown, in this embodiment, the second determining module 120 is used to: determine the credibility of the first sub-item based on the first sub-item and the first threshold range; and determine the credibility of the second sub-item based on the second sub-item and the second threshold range.

[0164] In an exemplary embodiment, the apparatus further includes a fourth determining module, configured to determine a target threshold range. The fourth determining module is configured to: determine, in two photosensitive units adjacent to a preset photosensitive unit in the target direction, the second derivative of the second channel of each photosensitive unit in the target direction corresponding to the second channel in the target direction; determine the target threshold range based on the second derivative of the second channel in the target direction; the target threshold range is either a first threshold range or a second threshold range, and the target direction is either a first direction or a second direction.

[0165] In one exemplary embodiment, reference is still made to... Figure 9 As shown, in response to the unreliability of the reference parameter, the third determining module 130 is used to: determine the correction parameter corresponding to the second channel in the target direction based on the second derivative of the second channel of each of the two photosensitive units adjacent to the preset photosensitive unit in the target direction.

[0166] like Figure 10 The diagram shown is a block diagram of an electronic device. This disclosure also provides an electronic device, for example, device 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0167] Device 500 may include one or more of the following components: processing component 502, memory 504, power component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.

[0168] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0169] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of this data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0170] The power supply component 506 provides power to the various components of the device 500. The power supply component 506 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 500.

[0171] Multimedia component 508 includes a screen that provides an output interface between device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0172] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0173] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0174] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0175] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0176] In an exemplary embodiment, device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0177] Another exemplary embodiment of this disclosure provides a non-transitory computer-readable storage medium, such as a memory 504 including instructions that can be executed by a processor 520 of a device 500 to perform the described method. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device. When the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the described method.

[0178] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0179] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method of processing an image signal, characterized by, The method comprises: determining a reference parameter item according to first channel information of a first channel in a preset photosensitive unit; the reference parameter item is a second derivative of the first channel information, and the first channel information is photosensitive data received by the first channel; determining the credibility of the reference parameter item; in response to the reference parameter item being unreliable, determining second channel information of a second channel in the preset photosensitive unit according to a correction parameter item; the correction parameter item is used to represent the second derivative of the second channel, and the correction parameter item is determined according to photosensitive data of a photosensitive unit adjacent to the preset photosensitive unit.

2. The image signal processing method according to claim 1, characterized by, The reference parameter item comprises a first sub-item and a second sub-item; The method comprises: determining the second derivative of the first channel in the first direction as the first sub-item according to the first channel information; determining the second derivative of the first channel in the second direction as the second sub-item according to the first channel information; wherein the first direction and the second direction are perpendicular, and the first direction and the second direction are the arrangement direction of the photosensitive unit in the photosensitive device array.

3. The method of processing an image signal according to claim 2, wherein, The method comprises: determining the credibility of the first sub-item according to the first sub-item and a first threshold range; determining the credibility of the second sub-item according to the second sub-item and a second threshold range.

4. The method of processing an image signal according to claim 3, wherein The method comprises: determining a target threshold range: determining the second derivative of the second channel in the target direction for each of the two photosensitive units adjacent to the preset photosensitive unit in the target direction; determining the target threshold range according to the second derivative of the second channel in the target direction; 5. The method of processing an image signal according to claim 4, characterized by, wherein the target threshold range is the first threshold range or the second threshold range, and the target direction is the first direction or the second direction. In response to the reference parameter item being unreliable, the method comprises:

6. The method of processing an image signal according to claim 1, wherein determining the correction parameter item of the second channel in the target direction according to the second derivative of the second channel in the target direction for each of the two photosensitive units adjacent to the preset photosensitive unit in the target direction. The method comprises:

7. An image signal processing apparatus characterized by comprising: in response to the reference parameter item being reliable, determining the second channel information according to the reference parameter item. The method comprises: a first determination module for determining a reference parameter item according to first channel information of a first channel in a preset photosensitive unit; the reference parameter item is a second derivative of the first channel information, and the first channel information is photosensitive data received by the first channel; a second determination module for determining the credibility of the reference parameter item; a third determination module for determining second channel information of a second channel in the preset photosensitive unit according to a correction parameter item in response to the reference parameter item being unreliable; the correction parameter item is used to represent the second derivative of the second channel, and the correction parameter item is determined according to photosensitive data of a photosensitive unit adjacent to the preset photosensitive unit.

8. The image signal processing apparatus according to claim 7, characterized by The reference parameter item includes a first sub-item and a second sub-item; The first determining module is configured to: determine, according to the first channel information, a second derivative corresponding to the first channel in a first direction as the first sub-item; determine, according to the first channel information, a second derivative corresponding to the first channel in a second direction as the second sub-item; The first direction and the second direction are perpendicular, and the first direction and the second direction are arrangement directions of light sensing units in a light sensing device array.

9. The image signal processing apparatus according to claim 8, characterized by The second determining module is configured to: determine, according to the first sub-item and a first threshold range, a credibility of the first sub-item; determine, according to the second sub-item and a second threshold range, a credibility of the second sub-item.

10. The image signal processing apparatus according to claim 9, wherein The device includes a fourth determining module configured to determine a target threshold range, and the fourth determining module is configured to: determine, in two light sensing units adjacent to the preset light sensing unit in a target direction, a second derivative corresponding to a second channel of each of the light sensing units in the target direction; determine, according to the second derivative corresponding to the second channel in the target direction, the target threshold range; The target threshold range is the first threshold range or the second threshold range, and the target direction is the first direction or the second direction.

11. The apparatus of claim 10, wherein In response to the reference parameter item being untrustworthy, the third determining module is configured to: determine, according to the second derivative corresponding to the second channel of each of the light sensing units in the target direction, the correction parameter item corresponding to the second channel in the target direction.

12. The apparatus of claim 7, wherein The third determining module is configured to: in response to the reference parameter item being trustworthy, determine the second channel information according to the reference parameter item.

13. An electronic device, comprising: comprise: a processor; a memory for storing executable instructions of the processor; The processor is configured to execute the image signal processing method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the image signal processing method according to any one of claims 1 to 6.

Images