Image denoising method and related apparatus

Abstract

An image denoising method and a related apparatus, relating to the technical field of image processing. The method is applied to an image signal processor. The method comprises: acquiring target information, and, on the basis of the target information, performing denoising on a fused image. The target information comprises the fused image, a first brightness interval, and a pixel brightness condition. The fused image is obtained by fusing at least two images, the first brightness interval comprises an image brightness interval for determining a brightness interval to be denoised in the fused image, and the pixel brightness condition comprises a brightness value condition of pixels in each block among at least one block comprised in the fused image. By using the present solution, denoising can be accurately performed, and the signal-to-noise ratio of the image can be improved.

Description

Image noise reduction processing methods and related devices Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image noise reduction processing method and related apparatus. Background Technology

[0002] Currently, cameras with a pixel-separated structure are widely used in fields such as autonomous driving and mobile phones. This type of camera differs from the color filter array (CFA) of a conventional camera's image sensor. In a pixel-separated structure camera, two types of pixel diodes—one large (LPD) and one small (SPD)—are positioned adjacent to each other. Because of the different sizes of the pixel diodes, their physical photosensitivity differs, resulting in different responses to light intensity and brightness. This leads to different exposure effects, meaning the sensor can capture different dynamic ranges. The dynamic range of a sensor is its ability to simultaneously capture both highlights and shadows in an image, or the range of brightness the sensor can capture. Furthermore, because the LPD and SPD are positioned very close together, controlling simultaneous exposure ensures temporal and spatial consistency, minimizing inter-frame motion blur while maximizing dynamic range. Two images can be created using an LPD and an SPD, and then fused together using high-dynamic-range (HDR) techniques such as dual conversion gain (DCG) to obtain a single high-dynamic-range image. However, because the LPD and SPD capture different amounts of light at the same exposure time, their brightness responses differ, leading to differences in signal intensity. This, in turn, introduces fusion noise after fusion, reducing the image's signal-to-noise ratio. Summary of the Invention

[0003] This application provides an image noise reduction method and related apparatus, which can accurately reduce noise and improve the signal-to-noise ratio of the image.

[0004] In a first aspect, this application provides an image denoising processing method applied to an image signal processor. The method includes: acquiring target information and performing denoising processing on a fused image based on the target information. The target information includes a fused image, a first brightness range, and pixel brightness information. The fused image is obtained by fusing at least two images; the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image; and the pixel brightness information includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0005] In the above scheme, based on the first brightness range, the brightness values ​​of each block in the fused image are considered to denoise the fused image. Specifically, since the first brightness range includes the image brightness range used to determine the brightness range to be denoised in the fused image, and combined with the brightness values ​​of each block in the fused image, the areas in the fused image that need denoising can be obtained. This allows for precise image denoising, reducing the possibility of reducing texture details and affecting sharpness by denoising areas that do not require denoising. In other words, this scheme can achieve precise denoising, improving the signal-to-noise ratio of the fused image while maintaining its texture details and sharpness.

[0006] In one possible implementation, the target information mentioned above also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

[0007] In the above scheme, if the first brightness range includes the brightness range of the at least two images used for fusion, then the target information also includes the corresponding fusion gain. Only by combining the first brightness range and the corresponding fusion gain can the brightness range to be denoised in the fused image be calculated, thus enabling subsequent precise noise reduction operations.

[0008] In one possible implementation, the target information includes information after embedding the first information into the fused image, wherein the first information includes at least one of a first brightness range and pixel brightness conditions. Optionally, the first information may also include the fusion gain. Exemplarily, the information after embedding the first information into the fused image is presented in an image data layout format.

[0009] For example, the acquisition of target information described above includes: receiving target information from an image sensor in an image data layout format. The target information includes information other than the fused image, which is embedded in the fused image to form target information in an image data layout format.

[0010] In the above scheme, the image signal processor receives target information from the image sensor, and the received target information is arranged according to the image data layout format. This implementation utilizes the processing capabilities of the image sensor itself to obtain the target information, thereby improving the overall processing efficiency.

[0011] In one possible implementation, the above-mentioned noise reduction processing of the fused image based on target information includes: determining the noise reduction region in the fused image based on the target information; and performing noise reduction on the noise reduction region in the fused image.

[0012] The above approach first identifies the noise reduction areas in the fused image, and then precisely performs noise reduction on these areas, reducing the risk of reducing texture details and affecting clarity by performing noise reduction on areas that do not require it.

[0013] In one possible implementation, the first brightness range includes an image brightness range for determining the brightness range to be denoised in the fused image, comprising: the first brightness range includes an image brightness range for fusing at least two images into a fused image. That is, the first brightness range is the brightness range used in the process of fusing the at least two images into the aforementioned fused image.

[0014] The aforementioned target information also includes a fusion gain, which comprises the gain used for fusion in each of at least two images. Determining the denoising region in the fused image based on the target information includes: determining a second brightness range based on a first brightness range and the fusion gain; and then determining the denoising region in the fused image based on pixel brightness and the second brightness range. The second brightness range includes at least one brightness range in the fused image to be denoised.

[0015] The above scheme requires converting the first brightness range into the brightness range of the fused image by combining the fusion gain, and then combining the pixel brightness of each block of the fused image to accurately determine the noise reduction area.

[0016] In one possible implementation, determining the denoising region in the fused image based on pixel brightness and a second brightness interval includes: identifying one or more target blocks in the fused image as denoising candidate blocks based on pixel brightness and a second brightness interval; and determining the denoising region based on the texture intensity type and the second brightness interval of each denoising candidate block. The texture intensity type is used to indicate the complexity of the texture. The target block is contained within at least one block, and the target block includes w pixels, where the brightness value of each of the w pixels is within the second brightness interval, and w is greater than or equal to a first threshold.

[0017] The above scheme, after obtaining the candidate noise reduction region, can further determine a smaller noise reduction region within the region based on the texture intensity type of the candidate noise reduction region, so as to achieve more accurate noise reduction and reduce the impact of noise reduction on image texture.

[0018] In one possible implementation, the above-mentioned determination of the denoising region based on the texture intensity type and second brightness range of each denoising candidate block includes:

[0019] Each denoising candidate block is divided into multiple sub-blocks based on its texture intensity type. The texture complexity indicated by the texture intensity type of the denoising candidate block is positively correlated with the number of sub-blocks obtained from the corresponding denoising candidate block.

[0020] Sub-blocks that satisfy the first condition are identified as noise reduction blocks. The first condition is used to characterize that among the r pixels included in the sub-block, the brightness value of each pixel is within a second brightness range, and r is greater than or equal to a second threshold.

[0021] The noise reduction area can be determined based on the identified noise reduction blocks.

[0022] The above method can further divide the candidate denoising blocks, with blocks of higher texture complexity being divided into more sub-blocks. Then, denoising blocks are identified from these sub-blocks, thus determining the denoising region. This allows for more granular determination of the denoising region, preserving more texture details.

[0023] In one possible implementation, the identified noise reduction block further includes a sub-block in the extended region that satisfies the first condition. The extended region includes the region obtained by extending the first boundary of the target block outwards. The target block is any one of the noise reduction candidate blocks, and the first sub-block in the target block is determined as the noise reduction block, and the first sub-block is located at the first boundary of the target block.

[0024] The above scheme allows for further searching of noise reduction blocks by expanding the area outside the candidate noise reduction block boundary when the noise reduction block is within the boundary of the candidate noise reduction block. This reduces the number of noise reduction blocks that need to be removed and improves the noise reduction effect.

[0025] In one possible implementation, the noise reduction region can be determined based on the identified noise reduction blocks, including: merging multiple adjacent noise reduction blocks into a candidate region. If the area of ​​the candidate region is greater than or equal to a preset area, the candidate region is determined to belong to the noise reduction region.

[0026] The above method requires the area of ​​the region to be denoised to meet certain conditions. If the area to be denoised is too small, its impact on the overall image noise will be small, and it is unnecessary to denoise the small area separately. This saves processing resources and improves processing efficiency.

[0027] In one possible implementation, the denoising regions in the fused image identified above include one or more. The denoising of the denoising regions in the fused image includes: denoising each denoising region based on its texture intensity type. The texture complexity indicated by the texture intensity type of the denoising region is negatively correlated with the denoising intensity of the corresponding denoising region.

[0028] The above scheme can reduce noise based on the texture intensity type of each noise reduction region. The higher the texture complexity of the region, the weaker the noise reduction intensity, thus preserving more texture details and reducing the impact of noise reduction on image clarity.

[0029] In one possible implementation, the method further includes: superimposing the denoised region onto the target region in the fused image. The target region is the region in the fused image that corresponds to the denoised region.

[0030] The above method can also weighted superimpose the denoised area with the corresponding area of ​​the fused image before denoising, thereby restoring more texture details and improving the clarity of the fused image after denoising.

[0031] Secondly, this application provides an image denoising method applied to an image sensor. The method includes: acquiring target information and sending the target information to an image signal processor. The target information includes a target image and a first brightness range. The target image is associated with a fused image, and the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image. The target information is used by the image signal processor to perform denoising processing on the fused image.

[0032] For example, the target image described above is a fused image. Alternatively, the target image described above comprises at least two images, which are fused together to obtain a fused image. Alternatively, the target image described above comprises two original images generated by an image sensor, which are used to perform gain conversion to obtain at least two gain-converted images, which are then fused together to obtain a fused image.

[0033] In the above scheme, the image sensor uses its own processing capabilities to obtain the target information and sends it to the image signal processor. This implementation fully utilizes the image sensor's processing capabilities, thereby improving overall processing efficiency. Furthermore, the target information can be used by the image signal processor to perform precise noise reduction on the fused image. Specifically, since the first brightness range includes the image brightness range used to determine the brightness range to be denoised in the fused image, the areas in the fused image that need noise reduction can be obtained. This enables precise image noise reduction, reducing the likelihood of reducing texture details and affecting sharpness by denoising areas that do not require noise reduction.

[0034] In one possible implementation, the target information mentioned above also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

[0035] In the above scheme, if the first brightness range includes the brightness range of the at least two images used for fusion, then the target information also includes the corresponding fusion gain. This allows the image signal processor to combine the first brightness range and the corresponding fusion gain to calculate the brightness range to be denoised in the fused image, thus enabling subsequent precise noise reduction operations.

[0036] In one possible implementation, the target information further includes pixel brightness information, which includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0037] The above scheme also includes pixel brightness information in the target information, which allows the image signal processor to accurately reduce noise in the fused image by considering the brightness values ​​in each block of the fused image based on the first brightness range.

[0038] In one possible implementation, sending target information to the image signal processor includes: sending the target information to the image signal processor in an image data layout format. The target information includes information other than the fused image, which is embedded in the fused image to form the target information in the image data layout format.

[0039] The above scheme sends target information to the image signal processor according to the image data arrangement format, which enables the image signal processor to receive target data as if it were receiving an image, thereby improving reception efficiency.

[0040] Thirdly, embodiments of this application provide an image signal processor, which includes:

[0041] The acquisition unit is used to acquire target information, which includes a fused image, a first brightness range, and pixel brightness information. The fused image is obtained by fusing at least two images; the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image; and the pixel brightness information includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0042] The noise reduction processing unit is used to perform noise reduction processing on the fused image based on the target information.

[0043] In one possible implementation, the target information mentioned above also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

[0044] In one possible implementation, the target information includes information after embedding the first information into the fused image, wherein the first information includes at least one of a first brightness range and pixel brightness conditions.

[0045] For example, the acquisition unit described above is specifically used to: receive target information from an image sensor in an image data layout format. The target information includes information other than the fused image, which is embedded in the fused image to form the target information in the image data layout format.

[0046] In one possible implementation, the aforementioned denoising unit is specifically used to: determine the denoising region in the fused image based on the target information; and perform denoising on the denoising region in the fused image.

[0047] In one possible implementation, the first brightness range includes an image brightness range for determining the brightness range to be denoised in the fused image, comprising: the first brightness range includes an image brightness range for fusing at least two images into a fused image. The aforementioned target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion. The denoising processing unit is specifically used for:

[0048] A second brightness range is determined based on a first brightness range and a fusion gain. The second brightness range includes at least one brightness range in the fused image to be denoised.

[0049] The noise reduction region in the fused image is determined based on pixel brightness and the second brightness range.

[0050] In one possible implementation, the noise reduction processing unit described above is specifically used for:

[0051] One or more target blocks in the fused image are identified as noise reduction candidate blocks based on pixel brightness and a second brightness range. The target block is contained in at least one block and includes w pixels, where the brightness value of each of the w pixels is within the second brightness range, and w is greater than or equal to a first threshold.

[0052] The denoising region is determined based on the texture intensity type and second brightness range of each denoising candidate block. The texture intensity type is used to indicate the complexity of the texture.

[0053] In one possible implementation, the noise reduction processing unit described above is specifically used for:

[0054] Each denoising candidate block is divided into multiple sub-blocks based on its texture intensity type. The texture complexity indicated by the texture intensity type of the denoising candidate block is positively correlated with the number of sub-blocks obtained from the corresponding denoising candidate block.

[0055] Sub-blocks that satisfy the first condition are identified as noise reduction blocks. The first condition is used to characterize that among the r pixels included in the sub-block, the brightness value of each pixel is within a second brightness range, and r is greater than or equal to a second threshold.

[0056] The noise reduction area can be determined based on the identified noise reduction blocks.

[0057] In one possible implementation, the identified noise reduction block further includes a sub-block in the extended region that satisfies the first condition. The extended region includes the region obtained by extending the first boundary of the target block outwards. The target block is any one of the noise reduction candidate blocks, and the first sub-block in the target block is determined as the noise reduction block, and the first sub-block is located at the first boundary of the target block.

[0058] In one possible implementation, the noise reduction processing unit is specifically used to: merge multiple adjacent noise reduction blocks into a candidate region. If the area of ​​the candidate region is greater than or equal to a preset area, the candidate region is determined to belong to the noise reduction region.

[0059] In one possible implementation, the denoising regions in the determined fused image include one or more. Specifically, the denoising processing unit is used to: denoise each denoising region based on its texture intensity type. The texture complexity indicated by the texture intensity type of the denoising region is negatively correlated with the denoising intensity of the corresponding denoising region.

[0060] In one possible implementation, the denoising unit is further configured to: superimpose the denoised region onto the target region in the fused image. The target region is the region in the fused image corresponding to the denoised region.

[0061] Fourthly, this application provides an image sensor, which includes:

[0062] The acquisition unit is used to acquire target information.

[0063] The transmitting unit is used to send target information to the image signal processor. The target information includes a target image and a first brightness range. The target image is associated with the fused image, and the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image. The target information is used by the image signal processor to perform denoising processing on the fused image.

[0064] In one possible implementation, the target information mentioned above also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

[0065] In one possible implementation, the target information further includes pixel brightness information, which includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0066] In one possible implementation, the aforementioned sending unit is specifically used to: send target information to the image signal processor in an image data layout format. The target information includes information other than the fused image, which is embedded within the fused image to form the target information in the image data layout format.

[0067] In one possible implementation, the target image is a fused image. Alternatively, the target image comprises at least two images, which are fused to obtain a fused image. Or, the target image comprises two original images generated by an image sensor, which are used to perform gain conversion to obtain at least two gain-converted images, which are then fused to obtain a fused image.

[0068] Fifthly, this application provides an image signal processor, which includes a processor and a memory. The memory is coupled to the processor, and when the processor executes a computer program stored in the memory, it can implement the vehicle control method described in any of the first aspects above. The image signal processor may also include a communication interface for communicating with other image signal processors. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0069] In one possible implementation, the image signal processor may include:

[0070] Memory, used to store computer programs;

[0071] The processor is configured to: acquire target information and perform noise reduction processing on the fused image based on the target information. The target information includes the fused image, a first brightness range, and pixel brightness information. The fused image is obtained by fusing at least two images; the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image; and the pixel brightness information includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0072] It should be noted that the computer program in the memory of this application can be pre-stored or downloaded from the Internet when using the image signal processor. This application does not specifically limit the source of the computer program in the memory. The coupling in the embodiments of this application is an indirect coupling or connection between image signal processors, units, or modules, which can be electrical, mechanical, or other forms, for information interaction between image signal processors, units, or modules.

[0073] Sixthly, this application provides an image sensor including a processor and a memory. The memory is coupled to the processor, and when the processor executes a computer program stored in the memory, it can implement the vehicle control method described in any of the second aspects above. The image sensor may also include a communication interface for communicating with other image sensors. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0074] In one possible implementation, the image sensor may include:

[0075] Memory, used to store computer programs;

[0076] The processor is configured to: acquire target information and send the target information to the image signal processor. The target information includes a target image and a first brightness range. The target image is associated with the fused image, and the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image. The target information is used by the image signal processor to perform denoising processing on the fused image.

[0077] It should be noted that the computer program in the memory of this application can be pre-stored or downloaded from the Internet and stored after use of the image sensor. This application does not specifically limit the source of the computer program in the memory. The coupling in the embodiments of this application is an indirect coupling or connection between image sensors, units, or modules, which can be electrical, mechanical, or other forms, for information interaction between image sensors, units, or modules.

[0078] In a seventh aspect, this application provides a computer-readable storage medium storing a computer program or computer instructions that are executed by a processor to implement the method described in any of the first aspects above.

[0079] Eighthly, this application provides a computer-readable storage medium storing a computer program or computer instructions that are executed by a processor to implement the method described in any of the second aspects above.

[0080] Ninthly, this application provides a computer program product that, when executed by a processor, implements the method described in any of the first aspects above.

[0081] In a tenth aspect, this application provides a computer program product that, when executed by a processor, implements the method described in any of the second aspects above.

[0082] Eleventhly, this application provides a vehicle including an image sensor and an image signal processor. The image signal processor is the image signal processor described in any of the first aspects above. The image sensor is the image sensor described in any of the second aspects above.

[0083] The beneficial effects corresponding to the third to eleventh aspects mentioned above can be found in the corresponding descriptions in the first and second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0084] Figure 1 is a schematic diagram of the color filter array arrangement of the image sensor.

[0085] Figure 2 is a schematic diagram of image fusion.

[0086] Figures 3 and 4 are schematic diagrams of the brightness response of the image under different ambient light levels.

[0087] Figure 5 is a schematic diagram of the signal-to-noise ratio of the fused image.

[0088] Figure 6 is a schematic diagram of the system architecture provided in the embodiment of this application.

[0089] Figure 7 is a schematic diagram of the method flow provided in the embodiment of this application.

[0090] Figure 8 is a schematic diagram of image segmentation provided in an embodiment of this application.

[0091] Figure 9 is a schematic diagram of the pixel brightness situation represented by a histogram provided in an embodiment of this application.

[0092] Figures 10 to 12 are schematic diagrams of the extended regions of the noise reduction candidate blocks provided in the embodiments of this application.

[0093] Figure 13 is a schematic diagram of the noise reduction block provided in the embodiment of this application.

[0094] Figure 14 is a schematic diagram of the method flow provided in the embodiment of this application.

[0095] Figures 15 to 18 are schematic diagrams of the device structure provided in the embodiments of this application. Detailed Implementation

[0096] In this application embodiment, "multiple" refers to two or more. In this application embodiment, "and / or" is used to describe the association relationship of related objects, indicating three relationships that can exist independently. For example, A and / or B can mean: A exists alone, B exists alone, or A and B exist simultaneously. The description methods used in this application embodiment, such as "at least one of a1, a2, ... and an (or at least one of them)," include the case where any one of a1, a2, ... and an exists alone, as well as the case where any combination of any multiple of a1, a2, ... and an exists alone. Each case can exist alone. For example, the description method of "at least one of a, b, and c" includes the cases where a, b, c, a and b combined, a and c combined, b and c combined, or a, b, and c combined.

[0097] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with substantially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another.

[0098] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0099] To facilitate understanding of the embodiments of this application, the terminology involved in this application will be introduced by way of example below.

[0100] 1. High dynamic range (HDR).

[0101] HDR refers to the ability of an image or video to capture and display a wider range of light intensity during processing, thereby providing more detail and richer brightness and color, making the picture more realistic and delicate.

[0102] For example, high dynamic range images can include a wider range of color responses and retain more image details.

[0103] 2. Cameras with separate large and small pixels.

[0104] In the image sensor of a camera with a large and small pixel separation structure, two types of pixel diodes, one large and one small, are placed in adjacent positions to obtain images with different brightness response ranges. That is, a large pixel diode (LPD) and a small pixel diode (SPD) are set. For ease of understanding, please refer to Figure 1 for an example.

[0105] For example, as shown in Figure 1. Figure 1(a) illustrates an exemplary diagram of the color filter array (CFA) arrangement of an image sensor. Here, R represents red, G represents green, and B represents blue. Each color corresponds to one photodiode (PD) with a larger photosensitive area and one photodiode with a smaller photosensitive area. The photodiode with the larger photosensitive area is denoted as PD1, and the photodiode with the smaller photosensitive area is denoted as PD2, as shown in Figure 1(a). A pixel diode includes photodiodes corresponding to the three colors RGB. For example, Figure 1(b) illustrates an exemplary structural diagram of a pixel diode. In Figure 1(b), a pixel diode may include one red, two green, and one blue photodiode. Due to the pixel size settings, an LPD and an SPD are arranged adjacently. That is, an LPD includes PD1_1, PD1_2, PD1_3, and PD1_4 as shown in Figure 1(b). An SPD includes PD2_1, PD2_2, PD2_3 and PD2_4 as shown in Figure 1(b).

[0106] For example, a single pixel diode can generate data for one pixel, while adjacent LPDs and SPDs can generate pixel data with different brightness responses. Based on this, the image sensor of a camera with a pixel size separation structure can generate two images with different brightness response ranges at the same shutter speed and under the same exposure conditions.

[0107] 3. HDR technology.

[0108] HDR technology is a set of techniques used to achieve a wider dynamic range (i.e., greater contrast between light and dark areas) than ordinary digital imaging techniques. HDR techniques include many types, such as, but not limited to: dual conversion gain (DCG), triple conversion gain (TCG), double analog gain, lateral overflow integral capacitor (LOFIC), split-pixel technology, or multi-frame synthesis technology. It is understood that the HDR techniques listed herein are merely examples and do not constitute a limitation on the embodiments of this application.

[0109] To facilitate understanding of how HDR technology achieves high dynamic range in images, the following explanation uses dual conversion gain technology as an example. See Figure 2 for an example.

[0110] The LPD and SPD shown in Figure 2 can be, for example, the LPD and SPD in the image sensor shown in Figure 1. Exemplarily, in a specific implementation, the image sensor of a camera with a size-pixel separation structure can acquire two images under the same shutter speed and exposure conditions. One image is acquired through the LPD in the image sensor, hereinafter referred to as the LPD image. The other is acquired through the SPD in the image sensor, hereinafter referred to as the SPD image. Then, high conversion gain (HCG) and low conversion gain (LCG) are applied to the LPD image to obtain an LPD high conversion gain image and an LPD low conversion gain image. These LPD high conversion gain image and LPD low conversion gain image can be represented as LPD_HCG and LPD_LCG, respectively, as shown in Figure 2. Similarly, high conversion gain (HCG) and low conversion gain (LCG) are applied to the SPD image to obtain an SPD high conversion gain image and an SPD low conversion gain image. The SPD high conversion gain image and SPD low conversion gain image can be represented as SPD_HCG and SPD_LCG, respectively, as shown in Figure 2. LPD_HCG, LPD_LCG, SPD_HCG, and SPD_LCG are four images with different photosensitivity and brightness response ranges. These four images are then fused to obtain a high dynamic range fused image, i.e., an HDR image, as shown in Figure 2. For example, Figure 3 illustrates the brightness response of the four images under different ambient light conditions and the brightness response of the fused HDR image.

[0111] For example, in Figure 3, the horizontal axis represents the ambient luminance, primarily reflecting the intensity of light in the environment. Therefore, in some possible implementations, this horizontal axis can be replaced with light intensity. The vertical axis represents the image sensor's brightness response to luminance, which can be represented by the image's brightness value. The image brightness value refers to the lightness or darkness of each pixel in the image. A larger brightness value indicates greater brightness. For example, if the brightness value data length is 8 bits, the brightness value ranges from 0 to 255. Alternatively, if the brightness value data length is 12 bits, the brightness value ranges from 0 to 4095. Alternatively, if the brightness value data length is 16 bits, the brightness value ranges from 0 to 65535. It is understood that the range of brightness values ​​depends on the brightness value data length, and the description herein is merely an example and does not constitute a limitation on the embodiments of this application.

[0112] As shown in Figure 3, due to the inherent limitations of image sensors, the brightness response of the image obtained after gain conversion still has a limit. It is precisely because the brightness response of a single image to ambient light is limited that HDR technology fuses multiple images to obtain an HDR image. For example, in Figure 3, the brightness response range of each image in LPD_HCG, LPD_LCG, SPD_HCG, and SPD_LCG is not large and is limited to the limit of gain conversion. However, by fusing LPD_HCG and LPD_LCG, then LPD_LCG and SPD_HCG, and finally SPD_HCG and SPD_LCG, the brightness response range of the final HDR image is significantly increased, resulting in a high dynamic range image.

[0113] Figure 3 illustrates, exemplarily, the luminance blending regions in an HDR image: luminance blending region 1, luminance blending region 2, and luminance blending region 3. The luminance in luminance blending region 1 of the HDR image is obtained by blending the luminance of LPD_HCG and LPD_LCG. The luminance in luminance blending region 2 of the HDR image is obtained by blending the luminance of LPD_LCG and SPD_HCG. The luminance in luminance blending region 3 of the HDR image is obtained by blending the luminance of SPD_HCG and SPD_LCG. To facilitate understanding of the process of blending two images, the following explanation uses the blending of LPD_LCG and SPD_HCG as an example.

[0114] For example, in order to obtain the brightness fusion region of the HDR image, each brightness fusion region is provided with a corresponding image brightness range for fusion, and the upper and lower thresholds of the brightness range are preset. For example, brightness fusion region 2 is obtained by fusing LPD_LCG and SPD_HCG. The brightness range for fusion corresponding to brightness fusion region 2 is based on LPD_LCG, that is, the brightness range for fusion is the brightness value range in LPD_LCG, and the preset brightness range is [a, b], as shown in Figure 3. Alternatively, in another possible implementation, the brightness range for fusion corresponding to brightness fusion region 2 can be based on SPD_HCG, that is, the brightness range for fusion is the brightness value range in SPD_HCG, as shown in Figure 4, then the preset brightness range is [c, d]. The setting of the brightness range for fusion can be based on either of the two images being fused, and is specifically set according to the needs of the actual application. This application embodiment does not limit this. The following description mainly uses LPD_LCG as the reference.

[0115] For example, during the fusion of LPD_LCG and SPD_HCG, the luminance range [a, b] corresponding to luminance fusion region 2 is known to be used for fusion. Therefore, LPD_LCG and SPD_HCG can be fused according to this luminance range. For instance, since this luminance range [a, b] is preset based on LPD_LCG, if the luminance value of a pixel in LPD_LCG is less than the lower threshold a of the luminance range, the luminance value of that pixel can be multiplied by the fusion gain corresponding to LPD_LCG, for example, by multiplying by the luminance value obtained by G1 shown in Figure 3, and used as the luminance value of the pixel with the same pixel coordinates after fusion. If the luminance value of a pixel in LPD_LCG is greater than the upper threshold b of the luminance range, the luminance value of the pixel with the same pixel coordinates in SPD_HCG can be multiplied by the fusion gain corresponding to SPD_HCG, for example, by multiplying by the luminance value obtained by G2 shown in Figure 3, and used as the luminance value of the pixel with the same pixel coordinates after fusion. If the luminance value of a pixel in LPD_LCG is within the luminance range [a, b], then the luminance value obtained by weighting the luminance value of the pixel in LPD_LCG by multiplying it by the corresponding fusion gain G1 of LPD_LCG, and adding this luminance value to the luminance value obtained by weighting the luminance value of the pixel at the same pixel coordinate in SPD_HCG by multiplying it by the corresponding fusion gain G2 of SPD_HCG, can be used as the luminance value of the pixel at the same pixel coordinate after fusion. For example, the fusion gain G1 of LPD_LCG and the fusion gain G2 of SPD_HCG can be pre-configured. Similarly, the fusion gain G3 of SPD_LCG in Figure 3 is also pre-configured. LPD_HCG has no fusion gain, or its fusion gain is 1. For example, in some possible implementations, in order to align the luminance responses between images and make the fused image luminance exhibit a linear trend, such as the linear luminance response of the HDR image shown in Figure 3, gain compensation can be applied to the multiplied fusion gain when multiplying the pixel luminance value of the image by its corresponding fusion gain. The specific implementation of gain compensation is not limited in the embodiments of this application, and will not be described in detail here.

[0116] The above description mainly uses the fusion of LPD_LCG and SPD_HCG as an example to illustrate the fusion process. The implementation process of fusion of LPD_HCG and LPD_LCG, as well as the implementation process of fusion of SPD_HCG and SPD_LCG, is similar and will not be repeated here.

[0117] It is understood that the image fusion shown in Figure 3 above is only an illustration. In other possible implementations, if the ambient light intensity is not so strong, then only a portion of the images from LPD_HCG, LPD_LCG, SPD_HCG, and SPD_LCG can be fused. For example, referring to Figure 3, assuming the ambient light intensity when the image was captured is less than light intensity 1, then only LPD_HCG and LPD_LCG can be fused to obtain an HDR image. Exemplarily, light intensity 1 can be any ambient light intensity between the ambient light intensity corresponding to the lower limit of the brightness of brightness fusion region 1 and the ambient light intensity corresponding to the lower limit of the brightness of brightness fusion region 2. Or, for example, referring to Figure 3, assuming the ambient light intensity when the image was captured is less than light intensity 2, then LPD_HCG and LPD_LCG, as well as LPD_LCG and SPD_HCG, can be fused to obtain an HDR image. Exemplarily, light intensity 2 can be any ambient light intensity between the ambient light intensity corresponding to the lower limit of the brightness of brightness fusion region 2 and the ambient light intensity corresponding to the lower limit of the brightness of brightness fusion region 3.

[0118] It is understood that the above description mainly uses the dual conversion gain technique as an example to illustrate the implementation scheme for achieving high dynamic range of images. The above description is only an example and does not constitute a limitation on the embodiments of this application. Other HDR techniques can also be used to fuse images with high dynamic range, and the embodiments of this application will not elaborate on them one by one.

[0119] 4. Image signal-to-noise ratio (SNR).

[0120] SNR is an important indicator for evaluating image quality. The higher the SNR, the better the image quality, the less noise, and the clearer the image details. Conversely, the lower the SNR, the worse the image quality, the more noise, and the blurrier the details.

[0121] Based on the above description of SNR, since the aforementioned high dynamic range image, i.e., HDR image, is obtained by fusing multiple images, the differences in brightness, texture, and color between the different images will generate noise when fused together, reducing the signal-to-noise ratio (SNR) of the fused image. For example, see Figure 5. Figure 5 illustrates an example of the SNR of an HDR image. It can be seen that the SNR decreases significantly in the brightness fusion region of the HDR image. Furthermore, besides the brightness fusion region, the SNR of the brightness regions bordering the brightness fusion region, such as the boundary brightness region shown in Figure 5, also decreases significantly. Among these, the SNR decrease is most severe in the brightness region with the most severe SNR decrease shown in Figure 5. The decrease in SNR indicates severe noise contamination in the brightness fusion region of the HDR image, resulting in blurred details and poor image quality. To achieve accurate image noise reduction and improve the image's SNR, embodiments of this application provide an image noise reduction processing method and related apparatus.

[0122] For example, the solution of this application is not limited to noise reduction processing of fused images obtained from cameras with pixel separation structures of arbitrary sizes. It can also be applied to noise reduction processing of fused images with arbitrarily expanded dynamic ranges. For instance, it can be applied to noise reduction processing of fused images obtained from cameras in any application field, such as automotive cameras, security cameras, machine vision cameras, or mobile phone cameras. This application does not limit the specific application scenarios.

[0123] Before introducing the image denoising processing method provided in the embodiments of this application, let's first introduce a possible image denoising processing system to which the embodiments of this application are applicable. For example, as shown in FIG6, the image denoising processing system 600 may include an image sensor 601 and an image signal processor (ISP) 602.

[0124] For example, image sensor 601 can generate an image through photoelectric conversion using pixel diodes. In one possible implementation, image sensor 601 may be, for example, an image sensor in a camera with the aforementioned pixel-size separation structure. Alternatively, in another possible implementation, image sensor 601 may be an image sensor in any other ordinary camera. This application embodiment does not impose any limitations on this.

[0125] For example, in some possible implementations, the image sensor 601 can also perform various processing on the images it generates, such as fusion or data statistics. For details, please refer to the following description, which will not be elaborated here.

[0126] For example, the image signal processor 602 can be used to process the image output by the image sensor 601, such as performing noise reduction. Details will be provided later.

[0127] It is understood that the description of the image denoising processing system 600 in Figure 6 above is merely illustrative and does not constitute a limitation on the embodiments of this application. In some other possible implementations, the image denoising processing system 600 may include multiple image sensors, and the images generated by these multiple image sensors may be sent to the image signal processor 602. The image signal processor 602 fuses the received multiple images and performs denoising processing on the fused image. The embodiments of this application do not limit the number of image sensors included in the image denoising processing system 600.

[0128] The following is an exemplary description of an image noise reduction method provided by an embodiment of this application. Referring to Figure 7, the method may include, but is not limited to, steps S701 to S702.

[0129] S701, The image signal processor acquires target information, which includes a fused image, a first brightness range, and pixel brightness information; wherein, the fused image is obtained by fusing at least two images, the first brightness range includes the image brightness range used to determine the brightness range to be denoised in the fused image, and the pixel brightness information includes the brightness value of pixels in each block of at least one block included in the fused image.

[0130] For example, the image signal processor described above may be the image signal processor 602 in the image noise reduction processing system 600 shown in FIG6.

[0131] Exemplarily, the fused image described above is obtained by fusing at least two images. Exemplarily, these at least two images can be images obtained by performing gain conversion on the original images generated by the image sensor. For example, these at least two images can be at least two of the LPD_HCG, LPD_LCG, SPD_HCG, and SPD_LCG images described in the aforementioned HDR technology. Alternatively, these at least two images can be images used to achieve HDR fusion in other HDR technologies besides DCG technology, which will not be elaborated upon in this application embodiment. Alternatively, these at least two images can be images generated by at least two adjacent image sensors, etc. It is understood that the description of the at least two images herein is merely illustrative and does not constitute a limitation on the embodiments of this application.

[0132] In one possible implementation, the fused image can be sent to an image signal processor, for example, from an image sensor. This image sensor can be, for example, image sensor 601 in the image noise reduction processing system 600 shown in FIG. 6. Furthermore, the image sensor can be the image sensor in a camera with a pixel-size separation structure. Based on the preceding description of a camera with a pixel-size separation structure, this image sensor can generate two images with different brightness response ranges under the same shutter speed and exposure conditions. Then, in one possible implementation, the image sensor can perform fusion processing on these two images with different brightness response ranges using HDR technology to obtain a fused image with high dynamic range. This fused image is then sent to the image signal processor. Exemplarily, the implementation of fusing the two images with different brightness response ranges using HDR technology can be exemplarily described in the relevant description of HDR technology in the preceding terminology, and will not be repeated here.

[0133] Alternatively, as an example, in some other possible implementations, after the image sensor generates the two images with different brightness response ranges, it can first perform gain conversion on the two images to obtain at least two gain-converted images. Then, the at least two images are sent to an image signal processor. The image signal processor performs fusion processing on the at least two images to obtain a fused image with high dynamic range. For specific implementations, please refer to the relevant descriptions of HDR technology in the foregoing terminology, which will not be repeated here.

[0134] Alternatively, as an example, in another possible implementation, after the aforementioned image sensor generates images with the two different brightness response ranges, or after at least two adjacent image sensors generate images respectively, the generated images can be sent to an image signal processor. The image signal processor performs gain conversion on the received images to obtain at least two gain-converted images. Then, HDR technology is used to fuse the at least two gain-converted images to obtain a fused image with high dynamic range. For specific implementations, please refer to the relevant descriptions of HDR technology in the foregoing terminology; they will not be repeated here.

[0135] Alternatively, as an example, in some other possible implementations, the image signal processor described above can receive the fused image from any other electronic device, such as a mobile phone, tablet, or smart wearable device. That is, the image signal processor can also be used to assist in noise reduction processing of the fused image generated by other electronic devices.

[0136] It is understood that the above-described implementation method of the image signal processor acquiring the above-fused image is merely an example and does not constitute a limitation on the embodiments of this application.

[0137] For example, the first brightness interval mentioned above includes two cases of image brightness intervals used to determine the brightness interval to be denoised in the fused image. In one possible implementation, the first brightness interval includes the image brightness intervals used for fusion of the at least two images, for example, it may include one or more image brightness intervals used to guide the fusion of the at least two images. That is, the first brightness interval is the brightness interval used in the process of fusing the at least two images into the fused image. For ease of understanding, the dual conversion gain technique and Figure 3 are used as examples in the above terminology introduction. For example, in the above dual conversion gain technique and the fusion implementation of Figure 3, if the fused image is obtained by fusing four images, LPD_HCG, LPD_LCG, SPD_HCG and SPD_LCG, then the first brightness interval may include three image brightness intervals. Among them, the first image brightness interval is the brightness interval used for fusion corresponding to brightness fusion area 1 in Figure 3. The brightness interval used for fusion corresponding to brightness fusion area 1 may be a brightness interval determined based on LPD_HCG, or it may be a brightness interval determined based on LPD_LCG. The second image brightness range is the brightness range [a, b] used for fusion corresponding to brightness fusion area 2 in Figure 3. This brightness range of brightness fusion area 2 can be a brightness range determined based on LPD_LCG or SPD_HCG. The third image brightness range is the brightness range used for fusion corresponding to brightness fusion area 3 in Figure 3. This brightness range of brightness fusion area 3 can be a brightness range determined based on SPD_HCG or SPD_LCG. For specific examples, please refer to the description of the brightness range used for fusion corresponding to brightness fusion area 2 in Figure 3 above; it will not be repeated here.

[0138] Alternatively, for example, in the above-described dual conversion gain technique and the fusion implementation of Figure 3, if the fused image is obtained by fusing two images, LPD_HCG and LPD_LCG, then the first brightness range can be the first image brightness range described above.

[0139] Alternatively, for example, in the above-described dual conversion gain technique and the fusion implementation of Figure 3, if the fused image is obtained by fusing three images, LPD_HCG, LPD_LCG and SPD_HCG, then the first brightness range may include the first image brightness range and the second image brightness range described above.

[0140] For example, in the first brightness range scenario described above, where the first brightness range includes the brightness range of the at least two images used for fusion, the target information may further include a fusion gain. Based on this fusion gain and the aforementioned first brightness range, the brightness range to be denoised in the fused image can be determined. The implementation process of this fusion gain and determining the brightness range to be denoised in the fused image can be exemplarily described in the second brightness range scenario below, and will not be detailed here.

[0141] In another possible implementation, the second brightness range scenario, the first brightness range mentioned above can include the brightness range to be denoised in the fused image. This brightness range to be denoised can be used to filter out candidate denoising blocks in the fused image, as detailed in the subsequent description, which will not be elaborated here. For example, the brightness range to be denoised can be determined based on the image brightness ranges and fusion gain of the at least two images used for fusion. The fusion gain includes the gain of the at least two images used to achieve fusion. For ease of understanding, the dual conversion gain technique described above and Figure 3 are used as examples. For instance, if the fused image is obtained by fusing four images: LPD_HCG, LPD_LCG, SPD_HCG, and SPD_LCG, then the fusion gain can include the three fusion gains G1, G2, and G3 shown in Figure 3. Alternatively, if the fused image is obtained by fusing two images: LPD_HCG and LPD_LCG, then the fusion gain can include the single fusion gain G1 shown in Figure 3. Alternatively, if the fused image is obtained by fusing three images: LPD_HCG, LPD_LCG, and SPD_HCG, then the fusion gain can include the two fusion gains G1 and G2 shown in Figure 3. To facilitate understanding of the process of determining the brightness range to be denoised based on the brightness range of the images used for fusion and the fusion gain of at least two of the images, the following explanation uses the brightness range used for fusion corresponding to brightness fusion region 2 and the corresponding fusion gain as an example.

[0142] For example, suppose the brightness fusion region 2 corresponds to a brightness range of [a, b] for fusion, and the corresponding fusion gain is the fusion gain G1 of LPD_LCG and / or the fusion gain G2 of SPD_HCG. In one possible implementation, as shown in Figure 5 above, the signal-to-noise ratio (SNR) of the image is severely reduced in the brightness fusion region. In particular, the SNR drop is most severe in the brightness region with a severe SNR decrease shown in Figure 5, and this severe SNR drop is unacceptable. Based on this, in order to achieve the most accurate noise reduction in subsequent noise reduction processing, the brightness range of [a, b] corresponding to the brightness fusion region 2 can be transformed into a brightness range [(a+b) / 2, b+(ba)*k / 2]. Here, k is a preset adjustable parameter. Combined with the corresponding fusion gain G1 and / or G2, the brightness range to be denoised in the fused image can be calculated. For example, the brightness range [(a+b) / 2, b+(ba)*k / 2] can be multiplied by G1 or G2, or by the weighted sum of G1 and G2, to obtain the brightness range to be denoised. This obtained brightness range includes more brightness areas with decreased signal-to-noise ratio, such as the brightness area with severely reduced SNR shown in Figure 5. This allows for more precise noise reduction in subsequent denoising processes. It is understood that the aforementioned transformed brightness range [(a+b) / 2, b+(ba)*k / 2] is only an example; in specific implementations, other transformations can be used to obtain the transformed brightness range. The goal is to include at least some or all of the brightness areas with decreased signal-to-noise ratio. The embodiments in this application will not be described in detail here.

[0143] For example, in another possible implementation, the brightness range [a, b] can be multiplied by G1 or G2, or by the weighted sum of G1 and G2, to obtain the brightness range to be denoised. This is also one implementation method, and the specific choice depends on the actual application requirements. This application embodiment does not limit this approach.

[0144] For example, in some possible implementations, the brightness range to be denoised is obtained by multiplying the brightness range [(a+b) / 2, b+(ba)*k / 2] or the brightness range [a, b] by G1 or G2, or by a weighted sum of G1 and G2. To align the brightness response and ensure that the obtained brightness range to be denoised and the brightness of the fused image exhibit a consistent trend, gain compensation can be applied to the multiplied fusion gain. The specific implementation of gain compensation is not limited in this application embodiment and will not be elaborated further.

[0145] For example, the above description mainly uses luminance fusion region 2 as an example to illustrate the process of determining a luminance range to be denoised in the fused image. Similarly, a luminance range to be denoised in the fused image can also be determined based on luminance fusion region 1 or luminance fusion region 3. The specific implementation is described above and will not be repeated here. For example, combining the dual conversion gain technique mentioned above with Figure 3 as an example: If the fused image is obtained by fusing four images: LPD_HCG, LPD_LCG, SPD_HCG, and SPD_LCG, then three luminance ranges to be denoised in the fused image can be determined based on luminance fusion region 1, luminance fusion region 2, and luminance fusion region 3. That is, the first luminance range includes these three luminance ranges to be denoised. If the fused image is obtained by fusing two images: LPD_HCG and LPD_LCG, then a luminance range to be denoised in the fused image can be determined based on luminance fusion region 1. That is, the first luminance range includes this one luminance range to be denoised. If the aforementioned fused image is obtained by fusing three images: LPD_HCG, LPD_LCG, and SPD_HCG, then two brightness regions to be denoised in the fused image can be determined based on brightness fusion region 1 and brightness fusion region 2. That is, the aforementioned first brightness region includes the two brightness regions to be denoised.

[0146] It is understood that the above-described process for determining the brightness range to be denoised is merely an example and does not constitute a limitation on the embodiments of this application.

[0147] The above examples illustrate two scenarios where the first brightness range includes the image brightness range used to determine the brightness range to be denoised in the fused image. It is understood that the above descriptions are merely examples and do not constitute a limitation on the embodiments of this application.

[0148] For example, based on the above description, the image signal processor can obtain the first brightness range in the following ways.

[0149] Firstly, if the first brightness range is the same as the first brightness range described above, and the first brightness range and the fusion gain are pre-configured in the register of the image sensor, then the image sensor can send the first brightness range and the corresponding fusion gain to the image signal processor. This allows the image signal processor to calculate the brightness range to be denoised in the fused image based on the first brightness range and the corresponding fusion gain. The specific calculation method can be found in the foregoing description and will not be repeated here.

[0150] Secondly, if the first brightness range is the same as the first brightness range described above, and the first brightness range and the fusion gain are pre-configured in the registers of the image signal processor, then the image signal processor can obtain the first brightness range and the corresponding fusion gain from its own registers. It can then calculate the brightness range to be denoised in the fused image based on the first brightness range and the corresponding fusion gain. The specific calculation method can be found in the foregoing description and will not be repeated here.

[0151] Thirdly, if the first brightness interval is the same as the second brightness interval described above, and the brightness intervals of the at least two images used for fusion and the fusion gain are pre-configured in the register of the image sensor, then the image sensor can first calculate the brightness interval to be denoised in the fused image based on the brightness intervals of the at least two images used for fusion and the corresponding fusion gain, i.e., calculate the first brightness interval. The specific calculation method can be found in the foregoing description and will not be repeated here. The image sensor then sends the calculated first brightness interval to the image signal processor.

[0152] Fourthly, if the first brightness range is the same as the second brightness range described above, and the first brightness range is pre-configured in the register of the image sensor, then the image sensor can send the first brightness range to the image signal processor.

[0153] Fifth, if the first brightness range is the same as the second brightness range described above, and the first brightness range is pre-configured in the register of the image signal processor, then the image signal processor can read the first brightness range from its own register.

[0154] It is understood that the method by which the image signal processor obtains the first brightness range described above is merely an example and does not constitute a limitation on the embodiments of this application.

[0155] For example, the pixel brightness situation mentioned above includes the brightness value of pixels in each block of at least one block included in the fused image. For example, in one possible implementation, the fused image itself is a block. Alternatively, for example, in another possible implementation, the fused image can be divided into n*m blocks, as shown in Figure 8. As can be seen in Figure 8, the fused image can be divided into blocks of n rows and m columns, and the block in the i-th row and j-th column can be represented as block ij. Here, i is an integer between 1 and n, and j is an integer between 1 and m. The values ​​of n and m are set according to the actual application, and this application embodiment does not limit them. Since the fused image is represented by a pixel matrix, each block in the n*m ​​blocks is also a small pixel matrix. Each pixel has a pixel coordinate and corresponding attribute data such as brightness value. For example, the pixel brightness situation can include two possible implementation methods. These are described below.

[0156] The first method for implementing pixel brightness includes the number of pixels corresponding to each brightness value in each of the n*m ​​blocks. For example, the brightness value ranges from 0 to 255. Therefore, the pixel brightness includes the number of pixels corresponding to each brightness value from 0 to 255 in each block. For ease of understanding, let's take one block as an example. The pixel brightness of this block includes the number of pixels with a brightness value of 0, 1, 2, 3, ..., 253, 254, and 255. The same applies to other blocks. For example, the pixel brightness of each block can be represented by a histogram, as shown in Figure 9. Figure 9 illustrates an example with a brightness value range of 0-255. Figure 9 is merely an example and does not constitute a limitation on the embodiments of this application. Alternatively, for example, the pixel brightness of each block can also be represented by a table, a line graph, or any other arbitrary method, and this application embodiment does not limit this.

[0157] The second method for implementing pixel brightness includes the brightness value of each pixel in each of the n*m ​​blocks. This means that it's not necessary to count the number of pixels corresponding to each brightness value in a block; only the brightness value of each pixel in each block is needed.

[0158] It is understood that the possible implementations of pixel brightness described above are merely examples and do not constitute a limitation on the embodiments of this application.

[0159] For example, based on the above description, the image signal processor obtains the pixel brightness information in the following ways.

[0160] Firstly, if the pixel brightness situation described above belongs to the first pixel brightness situation implementation method, and the fused image is obtained by the image sensor through fusion processing, then the image sensor can divide the fused image into the aforementioned n*m blocks. Furthermore, the image sensor counts the number of pixels corresponding to each brightness value in each block, thus obtaining the aforementioned pixel brightness situation. Then, this pixel brightness situation is sent to the image signal processor.

[0161] The second approach involves implementing the pixel brightness situation as described in the first approach, and the fused image is obtained through fusion processing by an image sensor. In this case, the image sensor sends the fused image to the image signal processor (SSP). Since the sent fused image includes the brightness values ​​of each pixel, the SSP divides the fused image into the aforementioned n*m blocks and counts the number of pixels corresponding to each brightness value in each block. This yields the pixel brightness situation. This implementation is also applicable to scenarios where the SSP receives fused images from other electronic devices for noise reduction, and will not be elaborated further.

[0162] Thirdly, if the pixel brightness situation described above belongs to the second pixel brightness situation implementation method, then after the image signal processor obtains the above fused image, it can obtain the brightness value of each pixel in each block. That is, it obtains the pixel brightness situation.

[0163] It is understood that the method by which the image signal processor obtains the pixel brightness described above is merely an example and does not constitute a limitation on the embodiments of this application.

[0164] In one possible implementation, if the fused image is sent by the image sensor to the image signal processor, and one or more of the first brightness range, fusion gain, and pixel brightness condition are also sent by the image sensor to the image signal processor, then this one or more pieces of information can be embedded in the fused image to form target information for the image data layout format. Then, the image sensor sends this target information for the image data layout format to the image signal processor. For ease of understanding, the following describes the embedding of the first brightness range, fusion gain, and pixel brightness condition into the fused image to form target information, using an image data layout format as an example.

[0165] For example, the aforementioned first brightness range, fusion gain, and pixel brightness can be embedded in the frame beginning and / or frame end of the fused image to form target information for image data arrangement. For instance, suppose the size of the fused image is w*h. Here, w represents the number of pixels in the width direction of the fused image. Alternatively, w can be said to be the width of the fused image. Alternatively, w can be said to be the number of columns in the pixel matrix of the fused image. h represents the number of pixels in the height direction of the fused image. Alternatively, h can be said to be the height of the fused image. Alternatively, h can be said to be the number of rows in the pixel matrix of the fused image. That is, the size of the pixel matrix of the fused image of size w*h is h rows and w columns.

[0166] For example, based on the above description, the first brightness range, fusion gain, and pixel brightness can be embedded in the frame header of the fused image according to the row arrangement format of the pixel matrix of the fused image. Optionally, data verification information can also be embedded in the fused image as the last row of the fused image in the same row arrangement format to ensure the reliability of data acquisition. This data verification information can be, for example, a cyclic redundancy check (CRC) or other verification information, and this application embodiment does not limit this.

[0167] Alternatively, for example, the first brightness range, fusion gain, and pixel brightness information can be embedded at the end of the frame of the fused image. Optionally, data verification information can also be embedded in the first line of the fused image. Alternatively, the first brightness range, fusion gain, pixel brightness information, and data verification information can all be embedded at the beginning or end of the frame of the fused image.

[0168] For example, taking the frame beginning of the embedded fused image as an example. Since the amount of embedded data is large, the fused image can be embedded in t rows. For example, the minimum value of t can be: 2 + the length of the data included in the pixel brightness condition / w, rounded up. Here, 2 represents the first and last rows of the embedded fused image. Furthermore, the length of the data included in the pixel brightness condition includes the data length of the pixel brightness condition obtained under the first pixel brightness condition implementation method described above. For example, the length of the data included in the pixel brightness condition = 2 * the data length of the pixel brightness condition in each block * the number of blocks. The number of blocks is, for example, n * m. w is the width of the aforementioned fused image, and the length of the data included in the pixel brightness condition / w can obtain the number of rows of embedded pixel brightness conditions. In addition, since there is an upper limit threshold for the data length of the embedded fused image, the length of the data in the embedded fused image, i.e., the length of the t rows, cannot exceed this upper limit threshold. For example, in these t rows, the first row can be used to embed the first brightness interval and the corresponding fusion gain. The last row can be used to embed data verification information. The remaining t-2 rows can be used to embed pixel brightness conditions.

[0169] Furthermore, as an example, in one possible implementation, the data arrangement format of the embedded rows should be consistent with the data arrangement of the rows in the fused image. For example, if the data length of a pixel in the fused image is 12 bits, and the length of each data point in the pixel brightness information is 16 bits, then to maintain the consistency of the row format, the 16-bit data of the pixel brightness information can be split into two data points of 4 bits + 12 bits, and these two data points can be placed in two pixels. For example, the 16-bit data can also be split into two data points of other lengths, such as 8 bits + 8 bits or 7 bits + 9 bits, as long as the length of the split data is not greater than 12 bits. It is understood that the data length described herein is merely an example and does not constitute a limitation on the embodiments of this application. In specific implementations, any data length can be used.

[0170] For example, in other examples, one or more of the first brightness range, fusion gain, and pixel brightness conditions may be embedded into the fused image to form target information, so that the target information is arranged in an image data layout format. The specific information embedded in the fused image is selected according to the needs of the actual application, and the embodiments of this application are not limited thereto.

[0171] It is understood that the above-described implementation of embedding data into a fused image to form the target information of the image data arrangement format is merely an example and does not constitute a limitation on the embodiments of this application.

[0172] For example, after the image sensor obtains the target information of the image data layout format, it sends out the data line by line according to the image data layout format.

[0173] For example, after receiving the target information of the image data arrangement format, the image signal processor can split the target information according to preset rules to obtain the fused image, the first brightness range, the fusion gain, and the pixel brightness.

[0174] In other possible implementations, the fused image, the first brightness range, the fusion gain, and the pixel brightness can be sent to the image signal processor separately. Alternatively, they can be sent to the image signal processor in other ways, which is not limited in this embodiment.

[0175] S702, the image signal processor performs noise reduction processing on the fused image based on the target information.

[0176] For example, after obtaining the target information, the image signal processor can determine the denoising region of the fused image based on the target information, and then perform denoising on the denoising region. Some possible implementation processes are described below.

[0177] In one possible implementation, as described above, if the first brightness range included in the target information is the same as the first brightness range described in the first brightness range case, then the target information also includes the corresponding fusion gain. Based on this, the image signal processor can determine the second brightness range according to the first brightness range and the corresponding fusion gain. The specific implementation process for determining the second brightness range can be exemplified by the implementation process for determining the brightness range to be denoised in the second brightness range case described above. The second brightness range includes at least one brightness range to be denoised in the fused image. The second brightness range can, for example, be the first brightness range described in the second brightness range case described above. Alternatively, in another possible implementation, the first brightness range included in the target information is the same as the first brightness range described in the second brightness range case described above. That is, the first brightness range included in the target information is the aforementioned second brightness range.

[0178] For example, after obtaining the second brightness range, the image signal processor can determine the noise reduction region of the fused image based on the second brightness range and the pixel brightness in the target information. An example is described below.

[0179] For example, the image signal processor can first determine one or more target blocks in the fused image as noise reduction candidate blocks based on the aforementioned pixel brightness conditions and the second brightness interval. The target block is a block in the at least one block included in the fused image where the number of pixels with brightness values ​​within the second brightness interval is greater than or equal to a first threshold. For example, the target block includes w pixels, where the brightness value of each of the w pixels is within the second brightness interval, and w is greater than or equal to the first threshold. The at least one block included in the fused image can be seen in the relevant description in Figure 8 above, and will not be repeated here. Furthermore, the pixel brightness conditions include the brightness values ​​of pixels in each block of the at least one block. For example, if the pixel brightness conditions belong to the first pixel brightness condition implementation method described above, that is, the pixel brightness conditions include the number of pixels corresponding to each brightness value in each block of the at least one block, then the number of pixels corresponding to the brightness values ​​belonging to the second brightness interval in a single block can be added together to obtain a pixel count sum. If the pixel count sum obtained in a certain block is greater than or equal to the first threshold... Alternatively, if the sum of the pixel counts in a block, as a percentage of the total pixel count in that block, is greater than or equal to a preset threshold, then that block can be identified as the aforementioned target block, i.e., a noise reduction candidate block. Alternatively, if the pixel brightness situation falls under the second pixel brightness situation implementation method described above, i.e., the pixel brightness situation includes the brightness values ​​of each pixel in each of the aforementioned at least one block, then the brightness values ​​of each pixel in a single block can be compared with a second brightness interval to count the number of pixels whose brightness values ​​belong to the second brightness interval. If the counted number of pixels is greater than or equal to a first threshold, then that block can be identified as the aforementioned target block, i.e., a noise reduction candidate block.

[0180] For example, it is understood that the above implementation of determining the denoising candidate blocks of the fused image is merely an example and does not constitute a limitation on the embodiments of this application.

[0181] For example, in one possible implementation, the identified denoising candidate blocks can be used as the denoising region of the fused image. For instance, one denoising candidate block can be used as a denoising fusion region. Alternatively, multiple adjacent denoising candidate blocks can be merged into a single denoising region.

[0182] In another possible implementation, the image signal processor described above can further determine the denoising region in each denoising candidate block based on the texture intensity type of each denoising candidate block determined above and the second brightness range described above. The texture intensity type is used to indicate the complexity of the texture. For ease of understanding, an exemplary description is given below.

[0183] For example, the texture intensity types described above can include multiple types. For instance, they can include flat texture types, weak texture types, and strong texture types. Alternatively, they can include flat texture types, weak texture types, medium texture types, and strong texture types. The texture intensity, or texture complexity, increases sequentially from flat texture types to weak texture types, medium texture types, and strong texture types. It is understood that the classification of texture intensity types here is merely illustrative and does not constitute a limitation on the embodiments of this application. In specific implementations, there may be more or fewer types, and the embodiments of this application do not impose any limitations on this.

[0184] For example, to more accurately determine the areas requiring denoising, the aforementioned candidate denoising blocks can be further divided into smaller sub-blocks. Since the texture intensities of the candidate denoising blocks differ, to preserve more texture details, candidate denoising blocks with higher texture intensities can be divided into more sub-blocks, thus achieving more accurate denoising. Based on this, after the image signal processor determines the candidate denoising blocks, it can determine the texture intensity type of each candidate denoising block using a texture intensity judgment method. For example, this texture intensity judgment method may include, but is not limited to, methods such as gray-level variance method, gray-level co-occurrence matrix method, gray-level difference matrix method, gray-level histogram method, or wavelet transform method. This application embodiment does not limit the specific texture intensity judgment method used. To facilitate understanding of the implementation process of determining the texture intensity type of each candidate denoising block using a texture intensity judgment method, the following description uses the gray-level variance method as an example.

[0185] For example, if the texture intensity type of a candidate noise reduction region is determined by the gray-level variance method, the gray-level variance of the candidate noise reduction region can be calculated. If the gray-level variance is less than or equal to a preset variance threshold 1, it indicates that the difference in brightness variation of the candidate noise reduction region is small, that is, there are few texture details and low texture complexity, and the texture intensity type of the candidate noise reduction region can be determined as a flat texture type. If the calculated gray-level variance is greater than or equal to threshold 2, it indicates that the difference in brightness variation of the candidate noise reduction region is large, that is, there are many texture details and high texture complexity, and the texture intensity type of the candidate noise reduction region can be determined as a strong texture type. If the gray-level variance is greater than threshold 1 and less than threshold 2, it indicates that the candidate noise reduction region has some texture details but not many, and the texture complexity is average, and the texture intensity type of the candidate noise reduction region can be determined as a weak texture type. It is understood that the description herein is merely an example and does not constitute a limitation on the embodiments of this application. In other possible implementations, the types of texture intensity types are not limited to those described herein, and the embodiments of this application will not elaborate on them one by one.

[0186] For example, after determining the texture intensity type of each denoising candidate block, the corresponding denoising candidate block is divided into sub-blocks according to the denoising region division precision corresponding to each texture intensity type. For example, assume that the texture intensity types include flat texture type, weak texture type, and strong texture type. The denoising region division precision corresponding to the flat texture type can be, for example, r1*c1. This means that the candidate denoising block belonging to the flat texture type is divided into sub-blocks of r1 rows and c1 columns. Similarly, the denoising region division precision corresponding to the weak texture type can be, for example, r2*c2. This means that the candidate denoising block belonging to the weak texture type is divided into sub-blocks of r2 rows and c2 columns. The denoising region division precision corresponding to the strong texture type can be, for example, r3*c3. This means that the candidate denoising block belonging to the strong texture type is divided into sub-blocks of r3 rows and c3 columns. Where r1*c1 < r2*c2 < r3*c3. That is, the texture complexity indicated by the texture intensity type of the denoising candidate block is positively correlated with the number of sub-blocks obtained by dividing the corresponding denoising candidate block. It is understood that the examples given herein do not constitute a limitation on the embodiments of this application.

[0187] In another possible implementation, each noise reduction candidate block can be divided into the same number of sub-blocks. Whether the sub-blocks are divided according to texture intensity type or uniformly divided into the same number of sub-blocks can be determined based on actual application requirements; this application does not impose any restrictions on this.

[0188] After dividing each candidate noise reduction area into sub-blocks, the number of pixels in each sub-block whose brightness values ​​fall within the second brightness interval can be counted. If the number of pixels in a sub-block whose brightness values ​​fall within the second brightness interval is greater than or equal to a second threshold, or if the ratio of the number of pixels in a sub-block whose brightness values ​​fall within the second brightness interval to the total number of pixels in that sub-block is greater than or equal to a preset threshold, then that sub-block can be identified as a noise reduction block. For example, a sub-block that satisfies a first condition can be identified as a noise reduction block. This first condition indicates that the brightness value of each of the r pixels included in the sub-block falls within the second brightness interval, and that r is greater than or equal to the second threshold. Alternatively, this first condition indicates that the ratio of r to the total number of pixels included in the sub-block is greater than or equal to a preset threshold. That is, if a sub-block satisfies the first condition, then that sub-block can be identified as a noise reduction block.

[0189] In one possible implementation, if a sub-block identified as a noise reduction block is located at the boundary of the noise reduction candidate block to which it belongs, then the boundary of the noise reduction candidate block can be expanded outward to obtain an extended region including more sub-blocks. Then, the search continues within the extended region. If a sub-block satisfying the first condition described above exists within the extended region, then that sub-block can also be identified as a noise reduction block. For ease of understanding, the following description, in conjunction with Figures 10, 11, or 12, provides an example.

[0190] For example, referring to Figure 10, suppose the fused image is divided into n rows and m columns of blocks, where block 2.2 is a candidate block for denoising. This candidate block is further divided into multiple sub-blocks, as shown by the solid lines on the left side of Figure 10. Several of these sub-blocks are identified as denoising blocks. Suppose one of the sub-blocks identified as a denoising block is located on the left boundary of block 2.2, as shown in Figure 10. Then, the left boundary of block 2.2 can be extended outwards to obtain an extended region. It can be seen that the block connected to the left boundary of block 2.2 is block 2.1. Block 2.1 can then be divided into multiple sub-blocks according to the same division precision as block 2.2. Then, several columns of sub-blocks near the left boundary of block 2.2 are selected from block 2.1 as the extended region of block 2.2. For example, Figure 10 shows 3 columns as an example. Then, the presence of a denoising block is further determined within this extended region. Figure 10 illustrates an example where a noise reduction block exists within the extended region. For instance, if the sub-block identified as the noise reduction block is located on the right boundary of block 2.2, the same procedure applies, and will not be elaborated further.

[0191] For example, see Figure 11. Compared to Figure 10, Figure 11 assumes that a sub-block identified as a noise reduction block is located at the lower boundary of block 2.2. Therefore, the lower boundary of block 2.2 can be extended outwards to obtain an extended region. It can be seen that the block connected to the lower boundary of block 2.2 is block 3.2. Block 3.2 can then be divided into multiple sub-blocks according to the same division precision as block 2.2. Then, several rows of sub-blocks near the lower boundary of block 2.2 are selected in block 3.2 as the extended region of block 2.2. For example, three rows are shown in Figure 11. Then, it is further determined whether a noise reduction block exists within this extended region. Figure 11 shows an example where two noise reduction blocks exist in the extended region. For example, if the sub-block identified as a noise reduction block is located at the upper boundary of block 2.2, the same process is followed, and will not be elaborated further.

[0192] For example, see Figure 12. Figure 12 differs from Figures 10 or 11 in that it assumes a sub-block identified as a noise reduction block is located at the top corner of block 2.2. For example, as shown in Figure 12, it is located at the bottom left corner of block 2.2. Then, the left, bottom, and bottom left boundaries of block 2.2 can be extended outwards to obtain an extended region. It can be seen that the block connected to the left boundary of block 2.2 is block 2.1, the block connected to the bottom boundary of block 2.2 is block 3.2, and the block connected to the bottom left corner of block 2.2 is block 3.1. Therefore, block 2.1, block 3.2, and block 3.1 can be divided according to the division precision of block 2.2 to obtain their respective corresponding sub-blocks. Then, several columns of sub-blocks near the left boundary of block 2.2 are selected in block 2.1 as sub-blocks of the extended region of block 2.2. For example, Figure 12 shows an example with 3 columns. In block 3.2, several rows of sub-blocks near the lower boundary of block 2.2 are selected as sub-blocks of the extended region of block 2.2. For example, Figure 12 shows an example with 3 rows. Furthermore, several rows and columns (e.g., three rows and three columns in Figure 12) of sub-blocks are selected from the upper right corner of block 3.1 as sub-blocks of the extended region. The sub-blocks selected from blocks 2.1, 3.2, and 3.1 constitute the extended region of block 2.2. The area of ​​the sub-blocks is indicated by the dashed lines in Figure 12. Then, it is further determined whether a noise reduction block exists within this extended region. Figure 12 shows an example where three noise reduction blocks exist in the extended region. For example, if the sub-block determined to be a noise reduction block is located at the top left, top right, or bottom right corner of block 2.2, the same process is followed, and will not be elaborated further.

[0193] It is understood that Figures 10 to 12 above are merely examples and do not constitute a limitation on the embodiments of this application.

[0194] For example, based on the above description, the noise reduction block determined above includes sub-blocks in the extended region that satisfy the first condition described above.

[0195] For example, after the noise reduction blocks are determined as described above, in one possible implementation, these determined noise reduction blocks can be used as the noise reduction region of the fused image. For example, one noise reduction block can be used as one noise reduction region. Alternatively, multiple adjacent noise reduction blocks can be merged into one noise reduction region.

[0196] For example, in another possible implementation, multiple adjacent denoising blocks can be merged into a candidate region. For instance, one or more candidate regions can be obtained. If the area of ​​a candidate region is greater than or equal to a preset area, the candidate region is determined to belong to the denoising region of the fused image. For ease of understanding, the following explanation uses Figure 12 as an example.

[0197] For example, refer to Figure 12. Based on the preceding introduction, in Figure 12, assume that sub-blocks A, B, C, D, E, F, G, H, and I are noise reduction blocks. Sub-blocks B, C, D, E, F, G, H, and I are multiple adjacent noise reduction blocks, and therefore can be merged into a single candidate region, hereinafter referred to as Candidate Region 1. Sub-block A has no adjacent noise reduction blocks, so sub-block A itself is a candidate region, hereinafter referred to as Candidate Region 2. Then, the area of ​​each candidate region is calculated. For example, the area of ​​Candidate Region 1 is the sum of the areas of sub-blocks B, C, D, E, F, G, H, and I. The area of ​​each sub-block can be obtained by multiplying its width and height. The width of the sub-block is equal to the number of columns in its pixel matrix, and the height is equal to the number of rows in its pixel matrix. Based on this, the area of ​​each candidate region can be calculated. Then, the area of ​​each candidate region is compared with a preset area. Candidate regions with an area greater than or equal to the preset area are selected as noise reduction regions. It is understood that Figure 12 is only an example and does not constitute a limitation on the embodiments of this application. Furthermore, sub-blocks A, B, C, D, E, F, G, H, and I shown in Figure 12 can be partial noise reduction blocks in the fused image; the processing of other noise reduction blocks is similar and will not be described in detail here.

[0198] Based on the above description, one or more noise reduction regions can be obtained, and the image signal processor can perform noise reduction processing on these one or more noise reduction regions. Exemplarily, any image noise reduction processing method can be used to perform noise reduction on each noise reduction region. This application does not limit the specific image noise reduction processing method used.

[0199] For example, in one possible implementation, after obtaining one or more denoising regions, denoising can be performed on the corresponding denoising region based on the texture intensity type of each denoising region. For ease of understanding, an example is described below.

[0200] For example, in one possible implementation, the texture intensity type of the denoising region can be determined using the aforementioned texture intensity determination method. Alternatively, in another possible implementation, as described above, the texture intensity type of each denoising candidate block has already been determined. The denoising region is then determined from these candidate blocks. Therefore, to save computational resources, the texture intensity type of the denoising region can reuse the texture intensity type of the denoising candidate block to which it belongs. For instance, if a denoising region is determined based on a denoising candidate block, then the texture intensity type of that candidate block is the texture intensity type of the denoising region. For ease of understanding, Figure 12 can be used as an example. In Figure 12, the region where sub-blocks B, C, D, E, F, G, H, and I are merged is the denoising region. This denoising region is determined based on block 2.2. Therefore, the texture intensity type of block 2.2 is the texture intensity type of the denoising region. Alternatively, suppose that the denoising block 2.2 in Figure 12 only includes sub-blocks B, C, D, and E. The area where sub-blocks B, C, D, and E are merged together is the denoising region. Similarly, the texture intensity type of block 2.2 is the same as the texture intensity type of the denoising region.

[0201] Alternatively, in another possible implementation, a denoising region may include sub-blocks from two denoising candidate blocks. For example, see Figure 13. In Figure 13, sub-blocks a, b, c, d, e, and f are the aforementioned denoising blocks and are adjacent. Therefore, these adjacent denoising blocks can be merged into a single denoising region, hereinafter referred to as denoising region 1. However, sub-blocks a, b, c, and d are sub-blocks within denoising candidate block 1. Sub-blocks e and f are sub-blocks within denoising candidate block 2. If denoising candidate block 1 and denoising candidate block 2 have the same texture intensity type, then the texture intensity type of denoising candidate block 1 and denoising candidate block 2 is the texture intensity type of denoising region 1. If the texture intensity types of denoising candidate block 1 and denoising candidate block 2 are different, then the texture intensity type of the denoising candidate block that includes more sub-blocks in denoising region 1 is selected as the texture intensity type of denoising region 1. For example, denoising candidate block 1 includes 4 sub-blocks in denoising region 1, and denoising candidate block 2 includes 2 sub-blocks in denoising region 1. Noise reduction candidate block 1 includes more sub-blocks in denoising region 1. Therefore, the texture intensity type of denoising candidate block 1 can be selected as the texture intensity type of denoising region 1. It should be understood that Figure 13 is only an example and does not constitute a limitation on the embodiments of this application.

[0202] For example, the above method for determining the texture intensity type of the noise reduction region is merely an example and does not constitute a limitation on the embodiments of this application.

[0203] For example, after determining the texture intensity type of each denoising region, denoising can be performed on the corresponding denoising region based on its texture intensity type. For example, different texture intensity types can correspond to different denoising intensities. For instance, the texture complexity indicated by the texture intensity type is negatively correlated with the corresponding denoising intensity. That is, the more complex the texture complexity, the smaller the denoising intensity. For example, assuming the texture intensity types include the aforementioned flat texture type, weak texture type, and strong texture type, then the flat texture type corresponds to the highest denoising intensity, followed by the weak texture type, and the strong texture type corresponds to the lowest. Based on this, denoising can be performed on the denoising region using the denoising intensity corresponding to its texture intensity type. The specific denoising method is not limited in this embodiment. This implementation method, where more textures result in lower denoising intensity, can reduce the texture data removed due to denoising, thereby retaining more texture information.

[0204] For example, in one possible implementation, after the above-mentioned denoising processing is completed for each denoised region, the denoised fused image can be obtained.

[0205] Alternatively, in another possible implementation, after denoising each denoised region as described above, the denoised region can be superimposed on the target region in the fused image. For example, weighted superposition can be used to obtain the superimposed denoised region. The target region is the region in the fused image corresponding to the denoised region. For instance, the weighted superposition involves weightedly superimposing the brightness values ​​of each pixel in the denoised region with the brightness values ​​of pixels at the same pixel coordinates in the fused image to obtain the superimposed denoised region. After superposition, the denoised fused image can be output. This implementation, by weighted superposition with the original fused image, allows the output denoised fused image to recover more texture details, improving the clarity and overall quality of the fused image.

[0206] In summary, this application's solution, based on the aforementioned first brightness range, considers the brightness values ​​in each block of the fused image, such as the number of pixels corresponding to each brightness value. This data can be used to accurately determine the areas in the fused image that require noise reduction, thereby achieving precise image noise reduction and reducing the risk of diminished texture details and reduced sharpness caused by denoising areas that do not require it. In other words, this solution achieves precise noise reduction, improving the signal-to-noise ratio of the fused image while maintaining its texture details and sharpness.

[0207] Referring to Figure 14, another possible image noise reduction method provided in an embodiment of this application is illustrated. This method is performed by the image sensor described above. Exemplarily, the method includes, but is not limited to, the following steps.

[0208] S1401, The image sensor acquires target information. The target information includes a target image and a first brightness range. The target image is associated with the fused image. The first brightness range includes the image brightness range used to determine the brightness range to be denoised in the fused image.

[0209] For example, the description of the first brightness range can be referred to the relevant introduction in the aforementioned S701, and will not be repeated here.

[0210] For example, in one possible implementation, the target image mentioned above is the fused image mentioned above. A description of the fused image can be found in the relevant introduction in S701 above, and will not be repeated here.

[0211] For example, in one possible implementation, the target image includes at least two images. These at least two images are used to fuse to obtain the fused image. These at least two images may, for example, be images obtained by performing gain conversion on the original images generated by the image sensor. For details, please refer to the relevant description in the aforementioned S701, which will not be repeated here.

[0212] For example, in one possible implementation, the target image includes two original images generated by an image sensor. These two original images are used to perform gain conversion to obtain at least two gain-converted images. These at least two images are then fused to obtain the fused image. The two original images are, for example, images with different brightness response ranges formed by a camera with the aforementioned pixel-size separation structure at the same shutter speed and under the same exposure conditions. For specific examples, please refer to the relevant description in S701 above; it will not be repeated here.

[0213] In one possible implementation, if the first brightness range is the first brightness range described in S701 above, the target information further includes a fusion gain. The fusion gain includes the gain used by each of the at least two images to achieve fusion. For specific examples, please refer to the relevant description in the aforementioned S701; it will not be repeated here.

[0214] In one possible implementation, the target information may further include pixel brightness information. Brightness information includes the brightness values ​​of pixels in each block of at least one block included in the fused image. For specific examples, please refer to the relevant description in S701 above, which will not be repeated here.

[0215] S1402, The image sensor sends target information to the image signal processor. The target information is used by the image signal processor to perform noise reduction processing on the fused image.

[0216] For example, the implementation of the image sensor sending target information to the image signal processor can be referred to the relevant description in the aforementioned S701, which will not be repeated here.

[0217] In summary, the image sensor described above utilizes its own processing capabilities to obtain the target information and sends it to the image signal processor. This implementation fully leverages the image sensor's processing capabilities, thereby improving overall processing efficiency. Furthermore, the target information can be used by the image signal processor to perform precise noise reduction on the fused image. Specifically, since the first brightness range includes the image brightness range used to determine the brightness range to be denoised in the fused image, the areas in the fused image that require noise reduction can be obtained. This enables precise image noise reduction, reducing the likelihood of reducing texture details and affecting sharpness by denoising areas that do not require noise reduction.

[0218] The foregoing mainly describes the methods provided in the embodiments of this application. It is understood that each device, in order to achieve the corresponding functions, includes hardware structures and / or software modules for executing each function. Based on the units and steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0219] This application embodiment can divide the device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0220] In the case of dividing each functional module according to its corresponding function, embodiments of this application also provide an apparatus for implementing any of the above methods. For example, an apparatus is provided that includes a unit (or means) for implementing each step in any of the above methods.

[0221] For example, please refer to Figure 15, which is a schematic diagram of the structure of an image signal processor 1500 provided in an embodiment of this application. The image signal processor 1500 shown in Figure 15 can be an image signal processor used to implement any of the methods described in Figure 7 and its possible embodiments. The image signal processor 1500 may include an acquisition unit 1501 and a noise reduction processing unit 1502. Wherein:

[0222] The acquisition unit 1501 is used to acquire target information, which includes a fused image, a first brightness range, and pixel brightness information. The fused image is obtained by fusing at least two images, the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image, and the pixel brightness information includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0223] The noise reduction processing unit 1502 is used to perform noise reduction processing on the fused image based on the target information.

[0224] In one possible implementation, the target information mentioned above also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

[0225] In one possible implementation, the target information includes information after embedding the first information into the fused image, wherein the first information includes at least one of a first brightness range and pixel brightness conditions.

[0226] For example, the acquisition unit 1501 is specifically used to: receive target information from an image sensor in an image data layout format. The target information includes information other than the fused image, which is embedded in the fused image to form the target information in the image data layout format.

[0227] In one possible implementation, the denoising processing unit 1502 is specifically used to: determine the denoising region in the fused image based on the target information; and perform denoising on the denoising region in the fused image.

[0228] In one possible implementation, the first brightness range includes an image brightness range for determining the brightness range to be denoised in the fused image, comprising: the first brightness range includes an image brightness range for fusing at least two images into a fused image. The aforementioned target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion. The denoising processing unit 1502 is specifically used for:

[0229] A second brightness range is determined based on a first brightness range and a fusion gain. The second brightness range includes at least one brightness range in the fused image to be denoised.

[0230] The noise reduction region in the fused image is determined based on pixel brightness and the second brightness range.

[0231] In one possible implementation, the noise reduction processing unit 1502 is specifically used for:

[0232] One or more target blocks in the fused image are identified as noise reduction candidate blocks based on pixel brightness and a second brightness range. The target block is contained in at least one block and includes w pixels, where the brightness value of each of the w pixels is within the second brightness range, and w is greater than or equal to a first threshold.

[0233] The denoising region is determined based on the texture intensity type and second brightness range of each denoising candidate block. The texture intensity type is used to indicate the complexity of the texture.

[0234] In one possible implementation, the noise reduction processing unit 1502 is specifically used for:

[0235] Each denoising candidate block is divided into multiple sub-blocks based on its texture intensity type. The texture complexity indicated by the texture intensity type of the denoising candidate block is positively correlated with the number of sub-blocks obtained from the corresponding denoising candidate block.

[0236] Sub-blocks that satisfy the first condition are identified as noise reduction blocks. The first condition is used to characterize that among the r pixels included in the sub-block, the brightness value of each pixel is within a second brightness range, and r is greater than or equal to a second threshold.

[0237] The noise reduction area can be determined based on the identified noise reduction blocks.

[0238] In one possible implementation, the identified noise reduction block further includes a sub-block in the extended region that satisfies the first condition. The extended region includes the region obtained by extending the first boundary of the target block outwards. The target block is any one of the noise reduction candidate blocks, and the first sub-block in the target block is determined as the noise reduction block, and the first sub-block is located at the first boundary of the target block.

[0239] In one possible implementation, the noise reduction processing unit 1502 is specifically used to: merge multiple adjacent noise reduction blocks into a candidate region. If the area of ​​the candidate region is greater than or equal to a preset area, the candidate region is determined to belong to the noise reduction region.

[0240] In one possible implementation, the denoising regions in the determined fused image include one or more. The denoising processing unit 1502 is specifically configured to: denoise each denoising region based on its texture intensity type. The texture complexity indicated by the texture intensity type of the denoising region is negatively correlated with the denoising intensity of the corresponding denoising region.

[0241] In one possible implementation, the denoising processing unit 1502 is further configured to: superimpose the denoised region onto the target region in the fused image. The target region is the region in the fused image corresponding to the denoised region.

[0242] The specific operation and beneficial effects of each unit in the image signal processor 1500 shown in Figure 15 can be found in the descriptions in Figure 7 and its possible embodiments above, and will not be repeated here.

[0243] For example, please refer to Figure 16, which is a schematic diagram of the structure of an image sensor 1600 provided in an embodiment of this application. The image sensor 1600 shown in Figure 16 can be an image sensor used to implement any of the methods described in Figure 7 or Figure 14 and their possible embodiments. The image sensor 1600 may include an acquisition unit 1601 and a transmission unit 1602. Wherein:

[0244] Acquisition unit 1601 is used to acquire target information.

[0245] The transmitting unit 1602 is used to transmit target information to the image signal processor. The target information includes a target image and a first brightness range. The target image is associated with the fused image, and the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image. The target information is used by the image signal processor to perform denoising processing on the fused image.

[0246] In one possible implementation, the target information mentioned above also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

[0247] In one possible implementation, the target information further includes pixel brightness information, which includes the brightness values ​​of pixels in each block of at least one block included in the fused image.

[0248] In one possible implementation, the sending unit 1602 is specifically used to: send target information to the image signal processor in an image data layout format. The target information includes information other than the fused image, which is embedded in the fused image to form the target information in the image data layout format.

[0249] In one possible implementation, the target image is a fused image. Alternatively, the target image comprises at least two images, which are fused to obtain a fused image. Or, the target image comprises two original images generated by an image sensor, which are used to perform gain conversion to obtain at least two gain-converted images, which are then fused to obtain a fused image.

[0250] The specific operation and beneficial effects of each unit in the image sensor 1600 shown in Figure 16 can be found in the descriptions in Figure 7 or Figure 14 and their possible embodiments, and will not be repeated here.

[0251] For ease of description, the image signal processor and image sensor mentioned above are collectively referred to as a device. It should be understood that the division of units within this device is merely a logical functional division; in actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units within the device can be implemented by a processor calling software; for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to realize the functions of each unit within the device. The processor can be, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functionality of some or all units can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC). The functionality of some or all of the above units is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it can include a large number of logic gates. The connection relationships between the logic gates are configured through a configuration file, thereby achieving the functionality of some or all of the above units. All units of the above device can be implemented entirely through processor-invoked software, entirely through hardware circuits, or partially through processor-invoked software with the remaining parts implemented through hardware circuits.

[0252] In this application embodiment, a processor is a circuit with data processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), Tensor Processing Unit (TPU), or Deep Learning Processing Unit (DPU).

[0253] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0254] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0255] For example, referring to Figure 17, which is a schematic diagram of the structure of a possible physical entity of the image signal processor provided in this application. The image signal processor 1700 shown in Figure 17 can be the image signal processor in the method described in the above embodiments. The image signal processor 1700 includes: a processor 1701, a memory 1702, and a communication interface 1703. The processor 1701, the communication interface 1703, and the memory 1702 can be interconnected or interconnected through a bus 1704.

[0256] For example, memory 1702 is used to store computer programs and data of image signal processor 1700. Memory 1702 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0257] The software or program code required for all or part of the functions of the image signal processor in the above method embodiments is stored in memory 1702.

[0258] In one possible implementation, if the software or program code required for some functions is stored in the memory 1702, the processor 1701, in addition to calling the program code in the memory 1702 to implement some functions, can also cooperate with other components to complete other functions described in the method embodiment. For example, it can cooperate with the communication interface 1703 to implement the function of receiving or sending data. Alternatively, the device may also include a display module, which can cooperate to implement the display function of the user interface, etc.

[0259] The number of communication interfaces 1703 can be multiple, used to support the image signal processor 1700 in communication, such as receiving or sending data or signals.

[0260] For example, processor 1701 may be a CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types, as described above. Processor 1701 may be used to read the program stored in memory 1702 and execute the operations performed by the image signal processor in FIG. 7 and its possible embodiments.

[0261] The specific operation and beneficial effects of each unit in the image signal processor 1700 shown in Figure 17 can be found in the descriptions in Figure 7 and its possible method embodiments above, and will not be repeated here.

[0262] For example, referring to Figure 18, which is a schematic diagram of the structure of a possible physical entity of the image sensor provided in this application. The image sensor 1800 shown in Figure 18 can be the image sensor in the method described in the above embodiments. The image sensor 1800 includes: a processor 1801, a memory 1802, and a communication interface 1803. The processor 1801, the communication interface 1803, and the memory 1802 can be interconnected or interconnected via a bus 1804.

[0263] For example, memory 1802 is used to store computer programs and data of image sensor 1800. Memory 1802 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0264] The software or program code required for all or part of the functionality of the image sensor in the above method embodiments is stored in memory 1802.

[0265] In one possible implementation, if the software or program code required for some functions is stored in the memory 1802, the processor 1801, in addition to calling the program code in the memory 1802 to implement some functions, can also cooperate with other components to complete other functions described in the method embodiments. For example, it can cooperate with the communication interface 1803 to implement the function of receiving or sending data. Alternatively, the device may also include a display module, which can cooperate to implement the display function of the user interface, etc.

[0266] There can be multiple communication interfaces 1803, which are used to support the image sensor 1800 in communicating, such as receiving or sending data or signals.

[0267] For example, processor 1801 may be a CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types, as described above. Processor 1801 may be used to read the program stored in memory 1802 and execute the operations performed by the image sensor in FIG. 7 or FIG. 14 and their possible embodiments.

[0268] The specific operation and beneficial effects of each unit in the image sensor 1800 shown in Figure 18 can be found in the descriptions in Figure 7 or Figure 14 and their possible method embodiments, and will not be repeated here.

[0269] This application also provides a chip including logic circuitry and a communication interface. The communication interface is used to receive and / or send information, or to input and / or output information. The logic circuitry is used to process the information. This chip is used to implement the aforementioned image denoising processing method, such as the image denoising processing method and its possible implementations shown in FIG. 7 or FIG. 14.

[0270] This application provides a vehicle that includes the image noise reduction processing system shown in FIG1 above.

[0271] This application also provides a computer-readable storage medium storing a computer program or computer instructions that are executed by a processor to implement the method implemented by the image signal processor in FIG7 and its possible embodiments.

[0272] This application also provides a computer-readable storage medium storing a computer program or computer instructions that are executed by a processor to implement the method implemented by the image sensor in FIG7 or FIG14 and their possible embodiments.

[0273] For example, the aforementioned computer-readable storage media may include, but are not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0274] This application also provides a computer program product, which, when read and executed by a computer, executes the method implemented by the image signal processor in FIG7 and its possible embodiments.

[0275] This application also provides a computer program product, which, when read and executed by a computer, executes the method implemented by the image sensor in FIG7 or FIG14 and their possible embodiments.

[0276] For example, the aforementioned computer program product includes, but is not limited to, a computer program, code, or electronic (digital) signal used to transmit computer program instruction code that enables the computer to implement the method when it is running.

[0277] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0278] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0279] It should also be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0280] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image noise reduction processing method, characterized in that, The method is applied to an image signal processor, and the method includes: Obtain target information, which includes a fused image, a first brightness range, and pixel brightness information; wherein, the fused image is obtained by fusing at least two images, the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image, and the pixel brightness information includes the brightness value of pixels in each block of at least one block included in the fused image; The fused image is denoised based on the target information.

2. The method according to claim 1, characterized in that, The target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

3. The method according to claim 1 or 2, characterized in that, The target information includes information after embedding the first information into the fused image, wherein the first information includes at least one of the first brightness range and the pixel brightness condition.

4. The method according to any one of claims 1-3, characterized in that, The noise reduction process on the fused image based on the target information includes: Based on the target information, the noise reduction region in the fused image is determined; Noise reduction is performed on the noise reduction region in the fused image.

5. The method according to claim 4, characterized in that, The first brightness range includes an image brightness range for determining the brightness range to be denoised in the fused image, including: the first brightness range includes an image brightness range for fusing the at least two images into the fused image; The target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion; Determining the noise reduction region in the fused image based on the target information includes: A second brightness range is determined based on the first brightness range and the fusion gain; wherein the second brightness range includes at least one of the brightness ranges to be denoised in the fused image; The noise reduction region in the fused image is determined based on the pixel brightness and the second brightness range.

6. The method according to claim 5, characterized in that, Determining the noise reduction region in the fused image based on the pixel brightness and the second brightness range includes: Based on the pixel brightness and the second brightness range, one or more target blocks in the fused image are determined as noise reduction candidate blocks; the target block is contained in the at least one block, and the target block includes w pixels, the brightness value of each of the w pixels is within the second brightness range, and w is greater than or equal to a first threshold; The noise reduction region is determined based on the texture intensity type of each noise reduction candidate block and the second brightness range; the texture intensity type is used to indicate the complexity of the texture.

7. The method according to claim 6, characterized in that, The step of determining the noise reduction region based on the texture intensity type of each noise reduction candidate block and the second brightness range includes: Each denoising candidate block is divided into multiple sub-blocks based on its texture intensity type; the texture complexity indicated by the texture intensity type of the denoising candidate block is positively correlated with the number of sub-blocks obtained by dividing the corresponding denoising candidate block. The sub-block that satisfies the first condition is determined as the noise reduction block; the first condition is used to characterize that among the r pixels included in the sub-block, the brightness value of each pixel is within the second brightness range, and the r is greater than or equal to the second threshold. The noise reduction region is determined based on the identified noise reduction blocks.

8. The method according to claim 7, characterized in that, The determined noise reduction block also includes sub-blocks in the extended region that satisfy the first condition; The extended region includes the region obtained by extending the first boundary of the target block outward, the target block being any one of the noise reduction candidate blocks, the first sub-block in the target block being determined as the noise reduction block, and the first sub-block being located at the first boundary of the target block.

9. The method according to claim 7 or 8, characterized in that, The step of determining the noise reduction region based on the identified noise reduction blocks includes: Multiple adjacent noise reduction blocks are merged into candidate regions; If the area of ​​the candidate region is greater than or equal to a preset area, the candidate region is determined to belong to the noise reduction region.

10. The method according to any one of claims 4-9, characterized in that, The noise reduction region includes one or more; the noise reduction of the noise reduction region in the fused image includes: Denoising is performed on each of the denoising regions based on the texture intensity type of each denoising region; the texture complexity indicated by the texture intensity type of the denoising region is negatively correlated with the denoising intensity of the corresponding denoising region.

11. The method according to any one of claims 4-10, characterized in that, The method further includes: The denoised region is superimposed on the target region in the fused image; the target region is the region in the fused image that corresponds to the denoised region.

12. An image noise reduction processing method, characterized in that, The method is applied to an image sensor, and the method includes: Obtain target information; The target information is sent to the image signal processor; the target information includes a target image and a first brightness range; the target image is associated with the fused image, and the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image; the target information is used by the image signal processor to perform noise reduction processing on the fused image.

13. The method according to claim 12, characterized in that, The target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

14. The method according to claim 12 or 13, characterized in that, The target information also includes pixel brightness information, which includes the brightness values ​​of pixels in each block of at least one block in the fused image.

15. The method according to any one of claims 12-14, characterized in that, Sending target information to the image signal processor includes: The target information is sent to the image signal processor in an image data layout format; wherein, the target information, other than the fused image, is embedded in the fused image to form the target information in the image data layout format.

16. The method according to any one of claims 12-15, characterized in that, The target image is the fused image; Alternatively, the target image may comprise at least two images, which are used to fuse the fused image. Alternatively, the target image may include two original images generated by the image sensor, which are used to perform gain conversion to obtain at least two images after gain conversion, and the at least two images are used to fuse to obtain the fused image.

17. An image signal processor, characterized in that, The image signal processor includes: An acquisition unit is used to acquire target information, the target information including a fused image, a first brightness range, and pixel brightness information; wherein, the fused image is obtained by fusing at least two images, the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image, and the pixel brightness information includes the brightness value of pixels in each block of at least one block included in the fused image; A noise reduction processing unit is used to perform noise reduction processing on the fused image based on the target information.

18. The image signal processor according to claim 17, characterized in that, The target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

19. The image signal processor according to claim 17 or 18, characterized in that, The target information includes information after embedding the first information into the fused image, wherein the first information includes at least one of the first brightness range and the pixel brightness condition.

20. The image signal processor according to any one of claims 17-19, characterized in that, The noise reduction processing unit is specifically used for: Based on the target information, the noise reduction region in the fused image is determined; Noise reduction is performed on the noise reduction region in the fused image.

21. The image signal processor according to claim 20, characterized in that, The first brightness range includes an image brightness range for determining the brightness range to be denoised in the fused image, including: the first brightness range includes an image brightness range for fusing the at least two images into the fused image; The target information also includes a fusion gain, which includes the gain used by each of the at least two images to achieve fusion; the noise reduction processing unit is specifically used for: A second brightness range is determined based on the first brightness range and the fusion gain; wherein the second brightness range includes at least one of the brightness ranges to be denoised in the fused image; The noise reduction region in the fused image is determined based on the pixel brightness and the second brightness range.

22. The image signal processor according to claim 21, characterized in that, The noise reduction processing unit is specifically used for: Based on the pixel brightness and the second brightness range, one or more target blocks in the fused image are determined as noise reduction candidate blocks; the target block is contained in the at least one block, and the target block includes w pixels, the brightness value of each of the w pixels is within the second brightness range, and w is greater than or equal to a first threshold; The noise reduction region is determined based on the texture intensity type of each noise reduction candidate block and the second brightness range; The texture intensity type is used to indicate the complexity of the texture.

23. The image signal processor according to claim 22, characterized in that, The noise reduction processing unit is specifically used for: Each denoising candidate block is divided into multiple sub-blocks based on its texture intensity type; the texture complexity indicated by the texture intensity type of the denoising candidate block is positively correlated with the number of sub-blocks obtained by dividing the corresponding denoising candidate block. The sub-block that satisfies the first condition is determined as the noise reduction block; the first condition is used to characterize that among the r pixels included in the sub-block, the brightness value of each pixel is within the second brightness range, and the r is greater than or equal to the second threshold. The noise reduction region is determined based on the identified noise reduction blocks.

24. The image signal processor according to claim 23, characterized in that, The determined noise reduction block also includes sub-blocks in the extended region that satisfy the first condition; The extended region includes the region obtained by extending the first boundary of the target block outward, the target block being any one of the noise reduction candidate blocks, the first sub-block in the target block being determined as the noise reduction block, and the first sub-block being located at the first boundary of the target block.

25. The image signal processor according to claim 23 or 24, characterized in that, The noise reduction processing unit is specifically used for: Multiple adjacent noise reduction blocks are merged into candidate regions; If the area of ​​the candidate region is greater than or equal to a preset area, the candidate region is determined to belong to the noise reduction region.

26. The image signal processor according to any one of claims 20-25, characterized in that, The noise reduction region includes one or more; the noise reduction processing unit is specifically used for: Denoising is performed on each of the denoising regions based on the texture intensity type of each denoising region; the texture complexity indicated by the texture intensity type of the denoising region is negatively correlated with the denoising intensity of the corresponding denoising region.

27. The image signal processor according to any one of claims 20-26, characterized in that, The noise reduction processing unit is also used for: The denoised region is superimposed on the target region in the fused image; the target region is the region in the fused image that corresponds to the denoised region.

28. An image sensor, characterized in that, The image sensor includes: The acquisition unit is used to acquire target information; A transmitting unit is configured to transmit target information to an image signal processor; the target information includes a target image and a first brightness range; the target image is associated with a fused image, and the first brightness range includes an image brightness range used to determine the brightness range to be denoised in the fused image; the target information is used by the image signal processor to perform denoising processing on the fused image.

29. The image sensor according to claim 28, characterized in that, The target information also includes a fusion gain, which includes the gain of each of the at least two images used to achieve fusion.

30. The image sensor according to claim 28 or 29, characterized in that, The target information also includes pixel brightness information, which includes the brightness values ​​of pixels in each block of at least one block in the fused image.

31. The image sensor according to any one of claims 28-30, characterized in that, The sending unit is specifically used for: The target information is sent to the image signal processor in an image data layout format; wherein, the target information, other than the fused image, is embedded in the fused image to form the target information in the image data layout format.

32. The image sensor according to any one of claims 28-31, characterized in that, The target image is the fused image; Alternatively, the target image may comprise at least two images, which are used to fuse the fused image. Alternatively, the target image may include two original images generated by the image sensor, which are used to perform gain conversion to obtain at least two images after gain conversion, and the at least two images are used to fuse to obtain the fused image.

33. An image signal processor, characterized in that, The image signal processor includes a processor and a memory, wherein the memory is used to store computer programs or computer instructions, and the processor is used to execute the computer programs or computer instructions stored in the memory, causing the image signal processor to perform the method as described in any one of claims 1-11.

34. An image sensor, characterized in that, The image sensor includes a processor and a memory, wherein the memory is used to store computer programs or computer instructions, and the processor is used to execute the computer programs or computer instructions stored in the memory, causing the image sensor to perform the method as described in any one of claims 12-16.

35. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or computer instructions that are executed by a processor to implement the method of any one of claims 1-11; or, the computer program or computer instructions are executed by a processor to implement the method of any one of claims 12-16.

36. A computer program product, characterized in that, When the computer program product is executed by a processor, the method described in any one of claims 1-11 will be implemented; or, the method described in any one of claims 12-16 will be implemented.

37. A vehicle, characterized in that, The vehicle includes an image sensor and an image signal processor, wherein the image signal processor is the image signal processor described in any one of claims 1-11, and the image sensor is the image sensor described in any one of claims 12-16.

Images