IMAGE PROCESSING METHOD, DEVICE, ELECTRONIC DEVICE, AND MEDIUM
The image processing system addresses uneven brightness in combined images by determining brightness values and weights for each pixel, enhancing image quality and user experience.
Patent Information
- Application Number
- JP2023546001
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-01-29
- Filing Date
- 2021-11-25
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-11-25
AI Technical Summary
Conventional image processing methods fail to address the issue of uneven image brightness distribution when combining multiple images taken at different exposure levels, leading to poor quality composite images.
An image processing system that determines the image brightness value and pixel-level brightness weights for each image, using these values to perform synthesis processing and produce a target image that mitigates the effects of uneven brightness distribution.
The system improves the quality of composite images by accounting for pixel-level brightness variations, resulting in enhanced user experience.
Smart Images

Figure 0007594126000001 
Figure 0007594126000002 
Figure 0007594126000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing method, device, electronic device, and medium.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to a Chinese patent application filed with the State Intellectual Property Office of China on January 29, 2021, bearing application number 202110126858.2 and titled "Image processing method, apparatus, electronic device, and medium," the entire contents of which are incorporated herein by reference. [Background technology]
[0003] As people's living standards continue to improve, various electronic devices are gradually and widely applied to all aspects of life, and more and more electronic devices are equipped with cameras for taking pictures. During user's taking pictures, due to environmental factors, it is inevitable that the image will be locally too bright or the image will be locally too dark.
[0004] In the prior art, for an image that is locally too bright or locally too dark due to environmental factors, multiple images with different luminances can be captured using different exposure values, and then the multiple images can be synthesized into one image. When synthesized in this way, if the image luminance of the multiple images is bright, the synthesized image becomes bright, and if the image luminance of the multiple images is dark, the synthesized image becomes dark, that is, the image luminance of the synthesized image is related to the image luminance distribution of the multiple images waiting to be synthesized. If the image luminance distribution of the multiple images waiting to be synthesized is uneven, that is, if there is a large difference in the number of images waiting to be synthesized that have bright image luminance and images waiting to be synthesized that have dark image luminance, the image luminance of the synthesized image becomes brighter or darker, which affects the quality of the synthesized image. Summary of the Invention
[0005] The present disclosure provides an image processing method, device, electronic device, and medium that can solve the problem of degradation in image quality when multiple images waiting to be combined are combined into one image when the image brightness of the multiple images waiting to be combined is unevenly distributed.
[0006] According to the first aspect, acquiring at least two images to be combined, the images being images of the same subject captured with different exposures; determining an image luminance value of each image to be synthesized; determining, for each image to be synthesized, a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized and a luminance value of each pixel point in the image to be synthesized; performing image synthesis processing based on the luminance values of each pixel point in at least two images to be synthesized and the luminance weights of each pixel point to obtain a target image; The present invention provides an image processing method, comprising:
[0007] According to the second aspect, an acquisition module for acquiring at least two images to be synthesized, the images being images of the same subject captured with different exposures; a first determination module for determining an image luminance value of each to-be-composited image; a second determination module for determining a luminance weight of each pixel point in each image to be synthesized based on an image luminance value of the image to be synthesized and a luminance value of each pixel point in the image to be synthesized; a synthesis module for performing image synthesis processing based on the luminance values of each pixel point in at least two images to be synthesized and the luminance weights of each pixel point to obtain a target image; The present invention provides an image processing device, including:
[0008] According to a third aspect, one or more processors; Memory, one or more applications stored in memory; which, when executed by the one or more processors, performs operations corresponding to the image processing method according to the first aspect. One or more applications configured to The present invention provides an electronic device including:
[0009] According to a fourth aspect, There is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, causes the image processing method according to the first aspect to be carried out.
[0010] The beneficial effects of the technical solution provided by the present disclosure are as follows:
[0011] The present disclosure provides an image processing method, device, electronic device, and medium. Compared to the conventional technology, the present disclosure determines the luminance weight of each pixel point in each image waiting to be synthesized based on the image luminance value of the image waiting to be synthesized and the luminance value of each pixel point in the image waiting to be synthesized. Then, image synthesis processing is performed based on the luminance value of each pixel point in at least two images waiting to be synthesized and the luminance weight of each pixel point to obtain a target image, so that the luminance weight of each pixel point in each image waiting to be synthesized correlates with the image luminance value of the image waiting to be synthesized and the luminance value of each pixel point in the image waiting to be synthesized. That is, the luminance weight of each pixel point in each image waiting to be synthesized takes into account the image luminance value of the image waiting to be synthesized itself, thereby reducing the influence of uneven distribution of the image luminance of at least two images waiting to be synthesized, improving the quality of the synthesized image, and improving the user experience. [Brief description of the drawings]
[0012] The above and other features, advantages, and aspects of each embodiment of the present disclosure will become more apparent with reference to the following specific embodiments in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. [Figure 1]FIG. 2 is a schematic diagram showing a flow of an image processing method according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram illustrating a mapping relationship between luminance and luminance weight according to an embodiment of the present disclosure; [Diagram 3] FIG. 13 is a schematic diagram illustrating a mapping relationship between other luminance and luminance weight according to an embodiment of the present disclosure. [Figure 4] FIG. 13 is a schematic diagram illustrating a mapping relationship between still another luminance and luminance weight according to an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a schematic diagram illustrating a configuration of an image processing device according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a schematic diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. The accompanying drawings show several embodiments of the present disclosure, but the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, it should be understood that these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are used for illustrative purposes and are not used to limit the scope of protection of the present disclosure.
[0014] It should be understood that the steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Also, method embodiments may include additional steps and / or steps that omit performance indications. The scope of the present disclosure is not limited in this respect.
[0015] As used herein, the term "including" and variations thereof are intended to be open-ended, meaning "including, but not limited to." The term "one embodiment" denotes "at least one embodiment," the term "another embodiment" denotes "at least one other embodiment," and the term "some embodiments" denotes "at least some embodiments." Definitions of other terms are provided below in the description.
[0016] It should be noted that the concepts of "first", "second", etc. referred to in this disclosure are merely intended to distinguish between devices, modules, or units, and do not limit these devices, modules, or units to being necessarily different devices, modules, or units, or limit the order or interdependence of functions performed by these devices, modules, or units.
[0017] It should be understood that the modifications "a" and "a plurality" referred to in this disclosure are general rather than limiting, and that a person skilled in the art should understand "one or more" unless the context clearly indicates otherwise.
[0018] The names of messages or information exchanged among multiple devices in the embodiments of the present disclosure are used for descriptive purposes only and are not intended to limit the scope of these messages or information.
[0019] The technical solution of the present disclosure and how the technical solution of the present disclosure solves the above technical problems are described in detail below using specific examples. Some specific examples below can be combined with each other, and the same or similar concepts or processes may not be mentioned any more in any example. The following describes the embodiments of the present disclosure in conjunction with the accompanying drawings.
[0020] An embodiment of the present disclosure provides an image processing method that may be executed in any one of terminal devices such as a mobile phone, a notebook computer, a desktop computer, a tablet PC, etc., or may be executed in a server or a server cluster. As shown in Fig. 1, the method includes steps S101 to S104.
[0021] In step S101, at least two images to be combined, in which the same subject is captured with different exposures, are acquired.
[0022] In one embodiment of the present disclosure, the at least two images to be combined may be images of the same scene captured by a camera using different exposure parameters, so that the image brightness of each image to be combined is different. Among them, the camera may be hardware in a terminal device, or may be an independent electronic device connected to a terminal. The terminal device may acquire the at least two images to be combined captured by the camera, and the server may acquire the at least two images to be combined captured by the camera via a client in the terminal device.
[0023] In step S102, the image luminance value of each image to be synthesized is determined.
[0024] The image brightness value can reflect the brightness degree of the image. In the embodiment of the present disclosure, the method of determining the image brightness value of any one image to be synthesized is not limited.
[0025] For example, the image brightness value of the image waiting to be synthesized may be calculated using the brightness value of each pixel point in any one of the images waiting to be synthesized, or it may be calculated by first screening pixel points whose brightness values satisfy a predetermined brightness value range from any one of the images waiting to be synthesized, and then using the brightness values of the screened pixel points.
[0026] In step S103, for each image waiting to be combined, a luminance weight for each pixel point in the image waiting to be combined is determined based on the image luminance value of the image waiting to be combined and the luminance value of each pixel point in the image waiting to be combined.
[0027] Each of the images to be combined corresponds to an image luminance value, which indicates the brightness level of the image to be combined. Each of the pixels in the images to be combined corresponds to a luminance value, which indicates the brightness level of the pixel point.
[0028] For any one pixel point of the images to be synthesized, the luminance weight of the pixel point is related to the luminance value of the pixel point and the image luminance value of the images to be synthesized.
[0029] In one possible embodiment, the closer the luminance value of a pixel point is to the image luminance value of the image to be synthesized, and the closer the brightness degree of the pixel point is to the brightness degree of the whole image, the larger the luminance weight of the pixel point is. In this case, the corresponding closeness degree of a pixel point, that is, the closeness degree of the luminance value of the pixel point to the image luminance value of the image to be synthesized, can be represented by the absolute difference value between the luminance value of the pixel point and the image luminance value of the image to be synthesized.
[0030] For example, if the image brightness value of the image waiting to be synthesized is 50, the brightness value of pixel point A in the image waiting to be synthesized is 48, and the brightness value of pixel point B is 55, then since 48 is closer to 50 than 55, the brightness weight of pixel point A will be greater than the brightness weight of pixel point B.
[0031] In one possible embodiment, in step S103, a luminance weight of each pixel point in the composition waiting image is determined based on the image luminance value of the composition waiting image and the luminance value of each pixel point in the composition waiting image. Specifically, the method includes determining a first associated image which is a composition waiting image whose image luminance value is adjacent to the image luminance value of the composition waiting image, and determining a luminance weight of each pixel point in the composition waiting image based on the image luminance value of the composition waiting image, the image luminance value of the first associated image, and the luminance value of each pixel point in the composition waiting image.
[0032] In one embodiment of the present disclosure, each image to be synthesized can be sorted according to the magnitude of the image luminance value. For any one image to be synthesized, the first related image of the image to be synthesized is an image to be synthesized whose image luminance value is adjacent to the image luminance value of the image to be synthesized. Among them, the number of the first related images of the image to be synthesized is one or two. When the image to be synthesized is the image to be synthesized with the smallest or the largest image luminance value, the number of the first related images of the image to be synthesized is one. When the image to be synthesized is not the one with the smallest image luminance value and not the one with the largest image luminance value, the number of the first related images of the image to be synthesized is two.
[0033] For example, the image luminance values of the images to be synthesized from image to be synthesized 1 to image to be synthesized 5 are P1 to P5 respectively, where P1 < P2 < P3 < P4 < P5. The number of the first related images of image to be synthesized 1 and image to be synthesized 5 is one respectively. For example, since the image luminance value P5 of image to be synthesized 5 is adjacent to the image luminance value P4 of image to be synthesized 4, the first related image of image to be synthesized 5 is image to be synthesized 4. The number of the first related images of image to be synthesized 2, image to be synthesized 3, and image to be synthesized 4 is two respectively. For example, since the image luminance value P3 of image to be synthesized 3 is adjacent to the image luminance value P2 of image to be synthesized 2 and the image luminance value P3 of image to be synthesized 3 is adjacent to the image luminance value P4 of image to be synthesized 4, the first related images of image to be synthesized 3 are image to be synthesized 2 and image to be synthesized 4.
[0034] Optionally, the first related image may be one image to be synthesized whose image luminance value is adjacent to the image luminance value of the image to be synthesized, or may be all images to be synthesized whose image luminance values are adjacent to the image luminance value of the image to be synthesized.
[0035] In an embodiment of the present disclosure, for each image to be synthesized, based on the image luminance value of the image to be synthesized and the image luminance value of the first related image of the image to be synthesized, the mapping relationship between the luminance and the luminance weight corresponding to the image to be synthesized is determined, and based on the mapping relationship, the luminance weight of each pixel point in the image to be synthesized can be determined based on the luminance value of each pixel point in the image to be synthesized.
[0036] As shown in FIG. 2, FIG. 2 is a schematic diagram showing the mapping relationship between luminance and luminance weight according to an embodiment of the present disclosure. Among them, P1 to P5 respectively represent the image luminance values from the image to be synthesized 1 to the image to be synthesized 5, and P1 < P2 < P3 < P4 < P5. For the image to be synthesized 3, based on the image luminance value P2 of the image to be synthesized 2, the image luminance value P3 of the image to be synthesized 3, and the image luminance value P4 of the image to be synthesized 4, the mapping relationship between the luminance and the luminance weight corresponding to the image to be synthesized 3 is determined as shown in FIG. 2, and for each pixel point in the image to be synthesized 3, the luminance weight of the pixel point can be determined based on the mapping relationship and the luminance of the pixel point.
[0037] It should be noted that the mapping relationship between the luminance and the luminance weight corresponding to the image to be synthesized 3 in FIG. 2 is approximate. For example, the mapping relationship may be a triangle shown in FIG. 2, or a trapezoid, or a curve, etc., and is not limited here. Optionally, the mapping relationship may be a linear relationship like the relationship curve shown in FIG. 2.
[0038] In step S104, image synthesis processing is performed based on the luminance values of each pixel point and the luminance weights of each pixel point in at least two images to be synthesized, and a target image is obtained.
[0039] In the embodiment of the present disclosure, for each image to be synthesized, if any one pixel point of the image to be synthesized has its luminance value and luminance weight, first, the luminance value and luminance weight of each pixel point in the image to be synthesized are used to adjust the luminance of each pixel point in the image to be synthesized, and an image to be synthesized after the luminance adjustment can be obtained. Then, at least two of the adjusted images to be synthesized are synthesized into one target image.
[0040] The embodiment of the present disclosure provides an image processing method. Compared with the conventional technology, the embodiment of the present disclosure determines the luminance weight of each pixel point in each image waiting to be synthesized based on the image luminance value of the image waiting to be synthesized and the luminance value of each pixel point in the image waiting to be synthesized. Then, based on the luminance value of each pixel point in at least two images waiting to be synthesized and the luminance weight of each pixel point, image synthesis processing is performed to obtain a target image, so that the luminance weight of each pixel point in each image waiting to be synthesized correlates with the image luminance value of the image waiting to be synthesized and the luminance value of each pixel point in the image waiting to be synthesized. That is, the luminance weight of each pixel point in each image waiting to be synthesized takes into account the image luminance value of the image waiting to be synthesized itself, thereby reducing the influence of uneven distribution of the image luminance of at least two images waiting to be synthesized, improving the quality of the synthesized image, and improving the user experience.
[0041] In another possible embodiment of the present disclosure, the brightness weight of each pixel point in the image waiting to be synthesized is determined based on the image brightness value of the image waiting to be synthesized, the image brightness value of a first related image of the image waiting to be synthesized, and the brightness value of each pixel point in the image waiting to be synthesized, and specifically, may include steps S1031 and S1032.
[0042] In step S1031, a luminance value interval of each target pixel point of the image waiting to be combined is determined based on the image luminance value of the image waiting to be combined and the image luminance value of a first related image of the image waiting to be combined.
[0043] Among them, the luminance value interval may be an open interval, a closed interval, or a semi-open and semi-closed interval, and the first associated image of the image to be synthesized is one or two.
[0044] In an embodiment of the present disclosure, for the luminance value interval of each target pixel point of the image to be synthesized, the maximum luminance value of the luminance value interval is less than or equal to the maximum value in the image luminance values of each image to be synthesized, and the minimum luminance value of the luminance value interval is greater than or equal to the minimum value in the image luminance values of each image to be synthesized.
[0045] For example, if P1 to P5 respectively represent the image luminance values of the images to be synthesized from image to be synthesized 1 to image to be synthesized 5, and P1 < P2 < P3 < P4 < P5, the maximum luminance value of the luminance value interval of any one image to be synthesized is less than or equal to P5, and the minimum luminance value of the luminance value interval of any one image to be synthesized is greater than or equal to P1.
[0046] When the first associated image of the image to be synthesized is one, the luminance value interval is determined by method A1.
[0047] In method A1, based on the image luminance value of the image to be synthesized and the image luminance value of the first associated image of the image to be synthesized, the luminance value interval of each target pixel point of the image to be synthesized is determined, but the determination method is not limited.
[0048] For example, as an optional embodiment, in step S1031, based on the image luminance value of the image to be synthesized and the image luminance value of the first associated image of the image to be synthesized, the luminance value interval of each target pixel point of the image to be synthesized is determined. Specifically, it includes determining the interval composed of the image luminance value of the image to be synthesized and the image luminance value of the first associated image of the image to be synthesized as the luminance value interval of each target pixel point of the image to be synthesized.
[0049] A specific determination method is to first determine the magnitude between the image luminance value of the image to be synthesized and the image luminance value of the first related image of the image to be synthesized, and set the smaller image luminance value as the minimum luminance value of the luminance value range, and the larger image luminance value as the maximum luminance value of the luminance value range. In other words, the luminance value range may be set to [small image luminance value, large image luminance value].
[0050] For example, if the image luminance value P1 of image 1 waiting to be synthesized is smaller than the image luminance value P2 of image 2 waiting to be synthesized, P1 is set as the minimum luminance value of the luminance value interval, and P2 is set as the maximum luminance value of the luminance value interval. In other words, the luminance value interval is set as [P1, P2].
[0051] As another embodiment of the option, for example, the determination method first determines the magnitude of the image luminance value of the image to be synthesized, the magnitude of the image luminance value of the first related image of the image to be synthesized, and the difference value between the two. The small image luminance value-predetermined value*difference value is set as the minimum luminance value of the luminance value range, and the large image luminance value+predetermined value*difference value is set as the maximum luminance value of the luminance value range, and the predetermined value is not limited, and may be, for example, 0.1.
[0052] For example, if the image brightness value P1 of image 1 waiting to be synthesized is smaller than the image brightness value P2 of image 2 waiting to be synthesized, and the difference between the two is P2-P1, the minimum brightness value can be calculated according to P1-0.1(P2-P1), and the maximum brightness value can be calculated according to P2+0.1(P2-P1). That is, the brightness value interval is [P1-0.1(P2-P1), P2+0.1(P2-P1)].
[0053] When the number of first related images of the image to be synthesized is two, the luminance value interval is determined by the method A2.
[0054] In method A2, for each first related image, first, a brightness value interval corresponding to the first related image is determined according to method A1, and then, based on the brightness value intervals corresponding to the two first related images, the brightness value intervals of each target pixel point of the image to be synthesized can be obtained.
[0055] For example, if the first related images of image 2 waiting to be synthesized are image 1 waiting to be synthesized and image 3 waiting to be synthesized, and the image brightness value P1 of image 1 waiting to be synthesized is smaller than the image brightness value P2 of image 2 waiting to be synthesized and P2 is smaller than the image brightness value P3 of image 3 waiting to be synthesized, then according to method A1, the brightness value interval [P1, P2] corresponding to image 1 waiting to be synthesized and the brightness value interval [P2, P3] corresponding to image 3 waiting to be synthesized are determined, and then the brightness value interval [P1, P3] of each target pixel point of image 2 waiting to be synthesized is obtained based on [P1, P2] and [P2, P3].
[0056] In addition, for example, when the first related images of the image waiting for composition 2 are the image waiting for composition 1 and the image waiting for composition 3, and the image luminance value P1 of the image waiting for composition 1 is smaller than the image luminance value P2 of the image waiting for composition 2, and P2 is smaller than the image luminance value P3 of the image waiting for composition 3, according to the method A1, the luminance value interval [P1-0.1(P2-P1), P2+0.1(P2-P1)] corresponding to the image waiting for composition 1, the sum The brightness value interval [P2-0.1(P3-P2), P3+0.1(P3-P2)] corresponding to image 3 to be synthesized is determined, and then the brightness value interval [P1-0.1(P2-P1), P3+0.1(P3-P2)] of each target pixel point of image 2 to be synthesized is obtained based on [P1-0.1(P2-P1), P2+0.1(P2-P1)] and [P2-0.1(P3-P2), P3+0.1(P3-P2)].
[0057] In step S1032, for any one pixel point of the image to be synthesized, if the luminance value of the pixel point is within the luminance value range, the luminance weight of the pixel point is determined based on the image luminance value of the image to be synthesized, the image luminance value of the first related image of the image to be synthesized, and the luminance value of the pixel point. If the luminance value of the pixel point is outside the luminance value range, the luminance weight of the pixel point is determined as a first setting value.
[0058] Therein, the magnitude of the first set value is not limited, but the first set value may be 0, for example.
[0059] In an embodiment of the present disclosure, for any one pixel point of an image to be synthesized, if the luminance value of the pixel point is within a luminance value interval, first, a mapping relationship between the luminance and luminance weight corresponding to the image to be synthesized is determined based on the image luminance value of the image to be synthesized and the image luminance value of a first related image of the image to be synthesized, and then, based on the luminance value of the pixel point and the mapping relationship, the luminance weight of the pixel point is determined. If the luminance value of the pixel point is outside the luminance value interval, the luminance weight of the pixel point is determined as a first setting value, for example, the luminance weight of the pixel point can be determined as 0.
[0060] Another possible embodiment of the present disclosure is to determine the brightness weight of each pixel point in the image waiting to be synthesized based on the image brightness value of the image waiting to be synthesized, the image brightness value of a first related image of the image waiting to be synthesized, and the brightness value of each pixel point in the image waiting to be synthesized, and specifically, can include steps S1033 to S1034.
[0061] In step S1033, a luminance value range for each target pixel point of the image waiting to be synthesized is determined based on the image luminance value of the image waiting to be synthesized, the image luminance value of a first related image of the image waiting to be synthesized, and the image luminance value of a second related image of the image waiting to be synthesized.
[0062] Among them, the second related image is an image waiting to be combined other than the image waiting to be combined, the image brightness value of which is adjacent to the image brightness value of the first related image.
[0063] In an embodiment of the present disclosure, when at least two images to be combined are sorted according to the magnitude of image brightness values, there are four possibilities as follows, which are denoted as Possibility 1 to Possibility 4, respectively.
[0064] In possibility 1, if there are two images waiting to be combined, then for any one image waiting to be combined, there is one first related image for that image waiting to be combined, and there is no second related image for that image waiting to be combined.
[0065] In possibility 2, if there are three images waiting to be composited, for the first or last image waiting to be composited, there will be one first related image for that image and one second related image for that image, and for the second image waiting to be composited, there will be two first related images for that image and there is no second related image for that image.
[0066] In possibility 3, if there are four images waiting to be composited, for the first or last image waiting to be composited, there will be one first related image for that image and one second related image for that image, and for the second or third image waiting to be composited, there will be two first related images for that image and one second related image for that image.
[0067] In possibility 4, if there are at least five images waiting to be combined, for the first or last image waiting to be combined, the first related image for that image is one and the second related image for that image is also one; for the second or last image waiting to be combined, the first related image for that image is two and the second related image for that image is one; and for any one of the images waiting to be combined other than the first, second, second and last image, the first related image for that image is two and the second related image for that image is also two.
[0068] For example, the image luminance values of the images to be synthesized from image 1 to image 5 to be synthesized are P1 to P5 respectively. Among them, P1 < P2 < P3 < P4 < P5. The first related image of the image to be synthesized 1 is the image to be synthesized 2, the second related image of the image to be synthesized 1 is the image to be synthesized 3, the first related image of the image to be synthesized 5 is the image to be synthesized 4, the second related image of the image to be synthesized 5 is the image to be synthesized 3, the first related images of the image to be synthesized 2 are the image to be synthesized 1 and the image to be synthesized 3, the second related image of the image to be synthesized 2 is the image to be synthesized 4, the first related images of the image to be synthesized 4 are the image to be synthesized 3 and the image to be synthesized 5, the second related image of the image to be synthesized 4 is the image to be synthesized 2, the first related image 1 of the image to be synthesized 3 is the image to be synthesized 2 and the image to be synthesized 4, and the second related image 1 of the image to be synthesized 3 is the image to be synthesized 1 and the image to be synthesized 5.
[0069] Among them, as an optional embodiment, in step S1033, based on the image luminance value of the image to be synthesized, the image luminance value of the first related image of the image to be synthesized, and the image luminance value of the second related image of the image to be synthesized, the luminance value interval of each target pixel point of the image to be synthesized is determined. Specifically, it may include steps S10331 to S10333.
[0070] In step S10331, based on the image luminance value of the image to be synthesized and the image luminance value of the first related image of the image to be synthesized, the initial luminance interval of each target pixel point of the image to be synthesized is determined.
[0071] In the embodiments of the present disclosure, the initial luminance interval can be an open interval, or a closed interval, or a semi-open and semi-closed interval. The method for determining the initial luminance interval is the same as the method for determining the luminance value interval of each target pixel point of the image to be synthesized in step S1031. That is, the luminance value interval determined in step S1031 is equivalent to the initial luminance interval. For the specific method for determining the initial luminance interval, reference can be made to the related description in step S1031, which is omitted here.
[0072] In addition, if the image to be synthesized does not have a second related image, the initial brightness range is determined as the brightness value range of each target pixel point of the image to be synthesized, and if the image to be synthesized has one or two second related images, the brightness value range of each target pixel point of the image to be synthesized may be further determined according to the methods of steps S10332 and S10333.
[0073] In step S10332, an extended luminance section of the initial luminance section is determined based on the image luminance value of the first related image of the image waiting to be combined and the image luminance value of the second related image of the image waiting to be combined.
[0074] Wherein, the extended luminance section can be an open section, a closed section, or a half-open and half-closed section. The second related image of the image to be synthesized is one or two.
[0075] When the number of second related images of the image to be synthesized is one, the number of first related images of the image to be synthesized is one or two. In this case, the first related image whose image brightness value is adjacent to the image brightness value of the second related image can be used to determine the extended brightness range of the image to be synthesized in method B1.
[0076] For example, if the first related images of image waiting to be combined are image waiting to be combined and image waiting to be combined 3, and the second related image of image waiting to be combined is image waiting to be combined 4, and the image luminance value of image waiting to be combined and image waiting to be combined 3 are adjacent, then the extended luminance section of image waiting to be combined 2 can be determined using image waiting to be combined and image waiting to be combined 4.
[0077] Method B1 determines the extended luminance range of the image to be synthesized by using the image luminance value of the first related image, the image luminance value of which is adjacent to the image luminance value of the second related image, and the image luminance value of the second related image of the image to be synthesized. In the method B1, the extended luminance range of the image to be synthesized is determined by using the image luminance value of the first related image, the image luminance value of which is adjacent to ...
[0078] For example, first, the magnitude relationship between the image luminance value of the first related image of the image waiting to be synthesized and the image luminance value of the second related image of the image waiting to be synthesized is determined, and the smaller image luminance value is set to the minimum luminance value of the extended luminance interval and the larger image luminance value is set to the maximum luminance value of the extended luminance interval, i.e., the extended luminance interval is set to [small image luminance value, large image luminance value].
[0079] For example, if the first related image of image awaiting compositing 1 is image awaiting compositing 2, the second related image of image awaiting compositing 1 is image awaiting compositing 3, and the image luminance value P2 of image awaiting compositing 2 is smaller than the image luminance value P3 of image awaiting compositing 3, the extended luminance range can be [P2, P3].
[0080] Also, for example, as an optional embodiment, in step S10332, an extended luminance section of the initial luminance section is determined based on an image luminance value of a first related image of the image waiting to be combined and an image luminance value of a second related image of the image waiting to be combined. Specifically, this includes determining a difference value of image luminance values between the image luminance value of the first related image of the image waiting to be combined and the image luminance value of the second related image of the image waiting to be combined, and determining the extended luminance section of the initial luminance section based on the difference value of the image luminance values and a third setting value.
[0081] Among them, when it is determined that the image luminance value of the first related image is greater than that of the second related image, the image luminance value of the first related image - the difference value of the image luminance values * the third set value is set as the minimum luminance value of the extended luminance section, and when it is determined that the image luminance value of the first related image is less than that of the second related image, the image luminance value of the first related image is set as the minimum luminance value of the extended luminance section, and the image luminance value of the first related image + the difference value of the image luminance values * the third set value is set as the maximum luminance value of the extended luminance section.
[0082] The magnitude of the third set value is not limited, but the third set value may be, for example, 0.3.
[0083] For example, the first associated image of the image to be synthesized 1 is set as the image to be synthesized 2, the second associated image of the image to be synthesized 1 is set as the image to be synthesized 3, and when the image luminance value P2 of the image to be synthesized 2 is smaller than the image luminance value P3 of the image to be synthesized 3, P2 is set as the minimum luminance value of the extended luminance interval, and based on P3 - P2, the third set value 0.3, and P2, the maximum luminance value P2 + 0.3(P3 - P2) of the extended luminance interval is determined. That is, the extended luminance interval is [P2, P2 + 0.3(P3 - P2)].
[0084] When there are two second associated images of the image to be synthesized, the extended luminance interval is determined by method B2.
[0085] For method B2, for each first associated image, the extended luminance interval corresponding to the first associated image can be determined by method B1. Then, the extended luminance intervals corresponding to the two first associated images are used as the extended luminance intervals of the initial luminance interval.
[0086] For example, the image luminance values of the images to be synthesized 1 - 5 are from P1 to P5 respectively, and among them, when P1 < P2 < P3 < P4 < P5, for the image to be synthesized 3, its first associated images are the image to be synthesized 2 and the image to be synthesized 4, and its second associated images are the image to be synthesized 1 and the image to be synthesized 5. The extended luminance interval [P1, P2] corresponding to the image to be synthesized 2 and the luminance extended interval [P4, P5] corresponding to the image to be synthesized 4 are determined by method B1. Then, the extended luminance intervals of the initial luminance interval include [P1, P2] and [P4, P5].
[0087] Also, for example, if the image luminance values of the images to be synthesized 1 to 5 to be synthesized are P1 to P5 respectively, and among them, when P1 < P2 < P3 < P4 < P5, for the image to be synthesized 3, its first related images are the image to be synthesized 2 and the image to be synthesized 4, and its second related images are the image to be synthesized 1 and the image to be synthesized 5. The extended luminance interval [P2 - 0.3(P2 - P1), P2] corresponding to the image to be synthesized 2 and the luminance extended interval [P4, P4 + 0.3(P5 - P4)] corresponding to the image to be synthesized 4 are determined by method B1. Then, the extended luminance intervals of the initial luminance interval include [P2 - 0.3(P2 - P1), P2] and [P4, P4 + 0.3(P5 - P4)].
[0088] In step S10333, the initial luminance interval and the extended luminance interval are stitched to obtain the luminance value interval of each target pixel point of the image to be synthesized.
[0089] In the embodiments of the present disclosure, the initial luminance interval and the extended luminance interval are stitched to obtain the luminance value interval of each target pixel point of the image to be synthesized.
[0090] For example, when the initial luminance interval of the image to be synthesized 1 is [P1, P2] and the extended luminance interval is [P2, P2 + 0.3(P3 - P2)], [P1, P2] and [P2, P2 + 0.3(P3 - P2)] can be stitched as [P1, P2 + 0.3(P3 - P2)] to be the luminance value interval of each target pixel point of the image to be synthesized 1.
[0091] Also, for example, when the initial luminance interval of the image to be synthesized 3 is [P2, P4] and the extended luminance interval includes [P2 - 0.3(P2 - P1), P2] and [P4, P4 + 0.3(P5 - P4)], [P2, P4], [P2 - 0.3(P2 - P1), P2], and [P4, P4 + 0.3(P5 - P4)] can be stitched as [P2 - 0.3(P2 - P1), P4 + 0.3(P5 - P4)] to be the luminance value interval of each target pixel point of the image to be synthesized 3.
[0092] In an embodiment of the present disclosure, for the luminance value range of each target pixel point of the image to be synthesized, the maximum luminance value of the luminance value range is less than or equal to the maximum value among the image luminance values of each image to be synthesized, and the minimum luminance value of the luminance value range is greater than or equal to the minimum value among the image luminance values of each image to be synthesized.
[0093] For example, if P1 to P5 respectively represent the image luminance values of the images to be synthesized from image to be synthesized 1 to image to be synthesized 5, and P1 < P2 < P3 < P4 < P5, the maximum luminance value of the luminance value range of any one of the images to be synthesized is less than or equal to P5, and the minimum luminance value of the luminance value range of any one of the images to be synthesized is greater than or equal to P1.
[0094] In step S1034, for any one pixel point of the image to be synthesized, when the luminance value of the pixel point is within the luminance value range, based on the image luminance value of the image to be synthesized, the image luminance value of the first related image of the image to be synthesized, the image luminance value of the second related image of the image to be synthesized, and the luminance value of the pixel point, the luminance weight of the pixel point is determined. When the luminance value of the pixel point is outside the luminance value range, the luminance weight of the pixel point is determined as the second set value.
[0095] Among them, the magnitude of the second set value is not limited. For example, the second set value may be 0.
[0096] In an embodiment of the present disclosure, for any one pixel point of the image to be synthesized, when the luminance value of the pixel point is within the luminance value range, first, based on the image luminance value of the image to be synthesized, the image luminance value of the first related image of the image to be synthesized, and the image luminance value of the second related image of the image to be synthesized, the mapping relationship between the luminance corresponding to the image to be synthesized and the luminance weight is determined. Then, based on the luminance value of the pixel point and the mapping relationship, the luminance weight of the pixel point is determined. When the luminance value of the pixel point is outside the luminance value range, the luminance weight of the pixel point is determined as the second set value. For example, the luminance weight of the pixel point is determined to be 0.
[0097] As shown in FIG. 3, FIG. 3 is a schematic diagram showing the mapping relationship between other luminance and luminance weight according to an embodiment of the present disclosure. Among them, from P1 to P5 represent the image luminance values of the images to be synthesized from image 1 to be synthesized to image 5 to be synthesized, respectively, and P1 < P2 < P3 < P4 < P5, P2 - N1 = 0.3 * (P2 - P1), N2 - P4 = 0.3 * (P5 - P4). For the image 3 to be synthesized, based on the image luminance values P1 to P5 corresponding to each of the images 1 to 5 to be synthesized, as shown in FIG. 3, the mapping relationship between the luminance and the luminance weight corresponding to the image 3 to be synthesized is determined. For each pixel point in the image 3 to be synthesized, based on the mapping relationship and the luminance of the pixel point, the luminance weight of the pixel point can be determined.
[0098] Note that the mapping relationship between the luminance and the luminance weight corresponding to the image 3 to be synthesized in FIG. 3 is approximate. For example, the mapping relationship may be a triangle shown in FIG. 3, or a trapezoid, or a curve, etc., and is not limited here.
[0099] To better understand and explain the technical solution provided by the present disclosure, hereinafter, in combination with an example of an option, the technical solution of the present disclosure will be described in detail. In the embodiment of the present disclosure, in combination with FIG. 4, three images to be synthesized will be described. Among them, FIG. 4 is a schematic diagram showing the mapping relationship between still other luminance and luminance weighted values according to an embodiment of the present disclosure. Specifically, it is as follows.
[0100] First, three images to be synthesized are obtained, and these three images to be synthesized are recorded as the image 1 to be synthesized, the image 2 to be synthesized, and the image 3 to be synthesized, respectively. Then, the image luminance values of each image to be synthesized are determined respectively. Among them, the image luminance value of the image 1 to be synthesized is P1, the image luminance value of the image 2 to be synthesized is P2, and the image luminance value of the image 3 to be synthesized is P3.
[0101] For each image waiting to be synthesized, a brightness weight of each pixel point in the image waiting to be synthesized is determined based on the image brightness value of the image waiting to be synthesized and the image brightness value of a first related image of the image waiting to be synthesized, among which the first related image is an image waiting to be synthesized whose image brightness value is adjacent to the image brightness value of the image waiting to be synthesized.
[0102] Specifically, based on P1 and P2, the mapping relationship between the luminance of the pixel point corresponding to the composition-ready image 1 shown in Fig. 4 and the luminance weight of the pixel point can be determined, and for each pixel point in the composition-ready image 1, the luminance weight of the pixel point can be obtained based on the luminance of the pixel point and the mapping relationship. Correspondingly, based on P1, P2, and P3, the mapping relationship between the luminance of the pixel point corresponding to the composition-ready image 2 shown in Fig. 4 and the luminance weight of the pixel point can be determined, and for each pixel point in the composition-ready image 2, the luminance weight of the pixel point can be obtained based on the luminance of the pixel point and the mapping relationship. Based on P2 and P3, the mapping relationship between the luminance of the pixel point corresponding to the composition-ready image 3 shown in Fig. 4 and the luminance weight of the pixel point can be determined, and for each pixel point in the composition-ready image 3, the luminance weight of the pixel point can be obtained based on the luminance of the pixel point and the mapping relationship.
[0103] Furthermore, image synthesis processing is performed based on the luminance value of each pixel point in the three images to be synthesized and the luminance weight of each pixel point, to obtain a target image.
[0104] For specific embodiments of the embodiments of the present disclosure, reference may be made to the relevant descriptions of the above-mentioned embodiments, which will be omitted here.
[0105] The above describes the image processing method in terms of method steps, and the following introduces the image processing device in terms of virtual modules.
[0106] The embodiment of the present disclosure provides an image processing device 50, which includes an acquisition module 501, a first determination module 502, a second determination module 503, and a synthesis module 504, as shown in FIG.
[0107] The acquisition module 501 acquires at least two images to be combined, in which the same subject is captured with different exposures.
[0108] The first determination module 502 determines the image brightness value of the image to be synthesized.
[0109] The second determination module 503 determines, for each image to be synthesized, a luminance weight of each pixel point in the image to be synthesized based on the image luminance value of the image to be synthesized and the luminance value of each pixel point in the image to be synthesized.
[0110] The synthesis module 504 performs image synthesis processing based on the luminance values of each pixel point in at least two images to be synthesized and the luminance weights of each pixel point, to obtain a target image.
[0111] In another possible embodiment of the present disclosure, when the second determination module 503 determines the luminance weight of each pixel point in the image to be synthesized based on the image luminance value of the image to be synthesized and the luminance value of each pixel point in the image to be synthesized, specifically, determining a first related image of the image to be combined, the first related image being an image to be combined whose image luminance value is adjacent to that of the image to be combined; A luminance weight for each pixel point in the image to be combined is determined based on an image luminance value of the image to be combined, an image luminance value of a first related image, and a luminance value of each pixel point in the image to be combined.
[0112] In another possible embodiment of the present disclosure, when the second determination module 503 determines the brightness weight of each pixel point in the image to be synthesized based on the image brightness value of the image to be synthesized, the image brightness value of the first related image of the image to be synthesized, and the brightness value of each pixel point in the image to be synthesized, specifically, determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of a first related image of the image to be synthesized; For any one pixel point of the image to be synthesized, if the luminance value of the pixel point is located within a luminance value interval, a luminance weight of the pixel point is determined based on the image luminance value of the image to be synthesized, the image luminance value of a first related image of the image to be synthesized, and the luminance value of the pixel point. If the luminance value of the pixel point is outside the luminance value interval, the luminance weight of the pixel point is determined as a first set value.
[0113] In another possible embodiment of the present disclosure, when the second determination module 503 determines the luminance value interval of each target pixel point of the image to be synthesized based on the image luminance value of the image to be synthesized and the image luminance value of the first related image of the image to be synthesized, specifically, A section consisting of the image luminance value of the image to be combined and the image luminance value of the first related image of the image to be combined is determined as a luminance value section of each target pixel point of the image to be combined.
[0114] In another possible embodiment of the present disclosure, the second determination module 503 determines the brightness weight of each pixel point in the image to be synthesized based on the image brightness value of the image to be synthesized, the image brightness value of the first related image of the image to be synthesized, and the brightness value of each pixel point in the image to be synthesized. Specifically, A luminance value interval for each target pixel point of the image waiting to be combined is determined based on an image luminance value of the image waiting to be combined, an image luminance value of a first related image of the image waiting to be combined, and an image luminance value of a second related image of the image waiting to be combined. For any one pixel point of the image to be combined, if the luminance value of the pixel point is located within a luminance value interval, a luminance weight of the pixel point is determined based on the image luminance value of the image to be combined, the image luminance value of a first related image of the image to be combined, the image luminance value of a second related image of the image to be combined, and the luminance value of the pixel point. If the luminance value of the pixel point is located outside the luminance value interval, the luminance weight of the pixel point is determined as a second set value. Among them, the second related image is an image waiting to be combined other than the first related image, the image brightness value of which is adjacent to the image brightness value of the first related image.
[0115] In another possible embodiment of the present disclosure, when the second determination module 503 determines the luminance value interval of each target pixel point of the image to be synthesized based on the image luminance value of the image to be synthesized, the image luminance value of the first related image of the image to be synthesized, and the image luminance value of the second related image of the image to be synthesized, specifically, determining an initial luminance range for each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of a first related image of the image to be synthesized; determining an extended luminance section of the initial luminance section based on an image luminance value of a first related image of the image waiting to be synthesized and an image luminance value of a second related image of the image waiting to be synthesized; The initial luminance interval and the extended luminance interval are stitched together to obtain a luminance value interval for each target pixel point of the image to be synthesized.
[0116] In another possible embodiment of the present disclosure, when the second determination module 503 determines the extended luminance section of the initial luminance section based on the image luminance value of the first related image of the image to be synthesized and the image luminance value of the second related image of the image to be synthesized, specifically, determining a difference between an image luminance value of a first related image of the image to be combined and an image luminance value of a second related image of the image to be combined; An extended luminance section of the initial luminance section is determined based on the difference between the image luminance values and the third setting value.
[0117] In another possible embodiment of the present disclosure, for a luminance value interval of each target pixel point of the image to be synthesized, the maximum luminance value of the luminance value interval is less than or equal to the maximum image luminance value of each image to be synthesized, and the minimum luminance value of the luminance value interval is greater than or equal to the minimum image luminance value of each image to be synthesized.
[0118] In the embodiment of the present disclosure, the first determination module 502 and the second determination module 503 may be the same determination module, or may be two different determination modules, and are not limited here.
[0119] The image processing device according to the embodiment of the present disclosure has the same realization principle as the image processing method shown in the embodiment of the method described above, and therefore the description is omitted here.
[0120] The embodiment of the present disclosure provides an image processing method, and compared to the prior art, the embodiment of the present disclosure determines the luminance weight of each pixel point in each image waiting to be combined based on the image luminance value of the image waiting to be combined and the luminance value of each pixel point in the image waiting to be combined. Then, image combination processing is performed based on the luminance value of each pixel point in at least two images waiting to be combined and the luminance weight of each pixel point to obtain a target image, so that the luminance weight of each pixel point in each image waiting to be combined correlates with the image luminance value of the image waiting to be combined and the luminance value of each pixel point in the image waiting to be combined. That is, the luminance weight of each pixel point in each image waiting to be combined takes into account the image luminance value of the image waiting to be combined itself, thereby reducing the influence of uneven distribution of the image luminance of at least two images waiting to be combined, improving the quality of the combined image, and improving the user experience.
[0121] The image processing device of the present disclosure has been introduced above from the perspective of a virtual module, and below, the electronic device of the present disclosure will be introduced from the perspective of a real device.
[0122] Referring to FIG. 6, FIG. 6 is a schematic diagram showing a configuration of an electronic device (e.g., a terminal device or a server in FIG. 1) 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet PCs), PMPs (portable multimedia players), and in-car terminals (e.g., car navigation terminals), as well as fixed terminals such as digital televisions and desktop computers. The electronic device shown in FIG. 6 is an example, and there is no limitation on the functions and scope of use of the embodiment of the present disclosure.
[0123] The electronic device includes a memory and a processor, where the processor may be referred to as a processing unit 601 as shown below, and the memory may include at least one of a read only memory (ROM) 602, a random access memory (RAM) 603, and a storage device 608 as shown below. Specifically,
[0124] 6, the electronic device 600 may include a processing unit (e.g., a CPU, a graphics processor, etc.) 601 capable of performing various appropriate operations and processes according to programs stored in a read only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data necessary for the operation of the electronic device 600. The processing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0125] Typically, the I / O interface 605 is connected to input devices 606 including a touch panel, keyboard, mouse, camera, microphone, accelerometer, gyro, etc., output devices 607 including a liquid crystal display (LCD), speaker, vibrator, etc., storage devices 608 including magnetic tape, hard disk, etc., and communication devices 609. The communication devices 609 may enable the electronic device 600 to communicate wirelessly or via wires with other devices to exchange data. Although FIG. 6 illustrates the electronic device 600 having various devices, it is understood that the implementation or inclusion of all the devices shown is not required. More or fewer devices may alternatively be implemented or included.
[0126] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program loaded on a non-transitory computer readable medium, the computer program including program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication device 609, can be installed from the storage device 608, or can be installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions limited to the method of the embodiment of the present disclosure are performed.
[0127] It should be noted that the computer readable medium of the present disclosure described above may be a computer readable signal medium, or a computer readable storage medium, or any combination of the two. The computer readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer readable storage medium include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer readable storage medium may be any tangible medium that contains or stores a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer readable signal medium carrying computer readable program code may be a data signal that may be included in baseband or propagated as part of a carrier wave. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium, other than a computer-readable storage medium, that can transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium may be transmitted over any suitable medium, including, but not limited to, electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0128] In some embodiments, clients and servers may communicate using any now known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), international networks (e.g., the Internet), end-to-end networks (e.g., ad hoc end-to-end networks), and any now known or later developed networks.
[0129] The computer readable medium may be included in the electronic device, or may exist separately and not integrated into the electronic device.
[0130] The computer-readable medium has one or more programs stored therein, and when the one or more programs are executed by the electronic device, the electronic device acquires at least two images to be combined, which are images of the same subject captured with different exposures, determines an image luminance value of each of the images to be combined, and for each image to be combined, determines a luminance weight for each pixel point in the image to be combined based on the image luminance value of the image to be combined and the luminance value of each pixel point in the image to be combined, and performs image synthesis processing based on the luminance value of each pixel point and the luminance weight of each pixel point in the at least two images to be combined, to obtain a target image.
[0131] Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages or combinations thereof, including, but not limited to, object-oriented programming languages such as Java®, Smalltalk®, C++®, and the like, as well as conventional process-based programming languages such as “C” or similar programming languages.
[0132] The program code may run entirely on the user computer, partially on the user computer, as a separate packaged software, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).
[0133] The flowcharts and block diagrams in the accompanying drawings illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. Here, each block in the flowcharts or block diagrams may represent a module, program segment, or part of code that includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions labeled in the blocks may occur in a different order than those labeled in the accompanying drawings. For example, two blocks shown in succession may in fact be executed substantially in parallel, or may be executed in a reverse order depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts may be implemented in a dedicated hardware-based system that executes the specified functions or operations, or may be implemented in a combination of dedicated hardware and computer instructions.
[0134] The modules or units related to the embodiments of the present disclosure may be realized in software or hardware, in which the name of the module or unit does not limit the unit itself in some cases, for example, the acquisition module can be described as "a module for acquiring at least two images to be synthesized".
[0135] The functions described herein may be performed, at least in part, by one or more hardware logic components. For example, but not limited to, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), etc.
[0136] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program used by or in combination with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of machine-readable storage media may include one or more line-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0137] An embodiment of the present disclosure provides an electronic device including one or more processors, a memory, and one or more applications stored in the memory and executed by the one or more processors and configured to perform the image processing method illustrated in accordance with an embodiment of the method.
[0138] The above describes the electronic device of the present disclosure from the perspective of an entity device, and the following describes the computer-readable medium of the present disclosure from the perspective of a medium.
[0139] An embodiment of the present disclosure provides a computer readable medium having stored thereon a computer program which, when run on a computer, enables the computer to execute corresponding contents of the above-mentioned method embodiments.
[0140] According to one or more embodiments of the present disclosure, A step of acquiring at least two images to be combined, the images being images of the same subject captured with different exposures; determining an image luminance value of each of the images to be synthesized; determining a luminance weight for each pixel point in each image to be combined based on an image luminance value of the image to be combined and a luminance value of each pixel point in the image to be combined; and performing image synthesis processing based on the luminance values of each pixel point in the at least two images to be synthesized and the luminance weights of each pixel point to obtain a target image.
[0141] According to one or more embodiments of the present disclosure, the step of determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized and a luminance value of each pixel point in the image to be synthesized includes: determining a first related image of the image to be combined, the first related image being an image to be combined that has an image luminance value adjacent to an image luminance value of the image to be combined; and determining a luminance weight of each pixel point in the image to be combined based on an image luminance value of the image to be combined, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be combined.
[0142] According to one or more embodiments of the present disclosure, the step of determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be synthesized includes: determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image; and determining, for any one pixel point of the image to be synthesized, if the luminance value of the pixel point is located within the luminance value interval, a luminance weight of the pixel point based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, and the luminance value of the pixel point, and determining, if the luminance value of the pixel point is outside the luminance value interval, the luminance weight of the pixel point as a first set value.
[0143] According to one or more embodiments of the present disclosure, the step of determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image includes: The method includes a step of determining a section consisting of the image luminance value of the image to be combined and the image luminance value of the first related image as a luminance value section of each target pixel point of the image to be combined.
[0144] According to one or more embodiments of the present disclosure, the step of determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be synthesized includes: determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and an image luminance value of a second related image of the image to be synthesized; a step of determining a luminance weight of any one pixel point of the image to be synthesized based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, the image luminance value of the second related image, and the luminance value of the pixel point if the luminance value of the pixel point is located within the luminance value interval, and determining the luminance weight of the pixel point as a second set value if the luminance value of the pixel point is outside the luminance value interval, Among them, the second related image is an image waiting to be combined other than the image waiting to be combined, the image brightness value of which is adjacent to the image brightness value of the first related image.
[0145] According to one or more embodiments of the present disclosure, the step of determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and an image luminance value of a second related image of the image to be synthesized includes: determining an initial luminance range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image; determining an extended luminance section of the initial luminance section based on an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized; The initial luminance interval and the extended luminance interval are stitched together to obtain a luminance value interval of each target pixel point of the image to be synthesized.
[0146] According to one or more embodiments of the present disclosure, the step of determining an extended luminance section of the initial luminance section based on an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized includes: determining a difference value between an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized; determining an extended luminance section of the initial luminance section based on the difference value of the image luminance values and a third setting value.
[0147] According to one or more embodiments of the present disclosure, for a luminance value interval of each target pixel point of the image to be synthesized, the maximum luminance value of the luminance value interval is less than or equal to the maximum image luminance value of each image to be synthesized, and the minimum luminance value of the luminance value interval is greater than or equal to the minimum image luminance value of each image to be synthesized.
[0148] According to one or more embodiments of the present disclosure, an acquisition module for acquiring at least two images to be synthesized, the images being images of the same subject captured with different exposures; A first determination module for determining an image luminance value of each of the images to be synthesized; a second determination module for determining a luminance weight of each pixel point in each of the images to be synthesized based on an image luminance value of the image to be synthesized and a luminance value of each pixel point in the image to be synthesized; The present invention provides an image processing device including a synthesis module that performs image synthesis processing based on the luminance value of each pixel point in the at least two images to be synthesized and the luminance weight of each pixel point to obtain a target image.
[0149] According to one or more embodiments of the present disclosure, when the second determination module determines the luminance weight of each pixel point in the image to be synthesized based on the image luminance value of the image to be synthesized and the luminance value of each pixel point in the image to be synthesized, specifically, determining a first related image of the image to be combined, the first related image being an image to be combined that has an image luminance value adjacent to the image luminance value of the image to be combined; This is to determine the brightness weight of each pixel point in the image to be combined based on the image brightness value of the image to be combined, the image brightness value of the first related image, and the brightness value of each pixel point in the image to be combined.
[0150] According to one or more embodiments of the present disclosure, when determining the luminance weight of each pixel point in the image to be synthesized based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, and the luminance value of each pixel point in the image to be synthesized, specifically, determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image; For any one pixel point of the images to be synthesized, if the luminance value of the pixel point is located within the above-mentioned luminance value interval, a luminance weight of the pixel point is determined based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, and the luminance value of the pixel point, and if the luminance value of the pixel point is located outside the above-mentioned luminance value interval, the luminance weight of the pixel point is determined as a first set value.
[0151] According to one or more embodiments of the present disclosure, when the second determination module determines the luminance value interval of each target pixel point of the image to be synthesized based on the image luminance value of the image to be synthesized and the image luminance value of the first related image, specifically, This is for determining a section consisting of the image luminance value of the image to be combined and the image luminance value of the first related image as the luminance value section of each target pixel point of the image to be combined.
[0152] According to one or more embodiments of the present disclosure, when the second determination module determines the luminance weight of each pixel point in the image to be synthesized based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, and the luminance value of each pixel point in the image to be synthesized, specifically, determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and an image luminance value of a second related image of the image to be synthesized; for determining a luminance weight of any one pixel point of the images to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, an image luminance value of the second related image, and a luminance value of the pixel point, if the luminance value of the pixel point is located within the luminance value interval, and determining the luminance weight of the pixel point as a second set value, if the luminance value of the pixel point is located outside the luminance value interval; Among them, the second related image is an image waiting to be combined other than the image waiting to be combined, the image brightness value of which is adjacent to the image brightness value of the first related image.
[0153] According to one or more embodiments of the present disclosure, when the second determination module determines the luminance value interval of each target pixel point of the image to be synthesized based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, and the image luminance value of the second related image of the image to be synthesized, specifically, determining an initial luminance section of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image; determining an extended luminance section of the initial luminance section based on an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized; The initial luminance interval and the extended luminance interval are stitched together to obtain a luminance value interval of each target pixel point of the image to be synthesized.
[0154] According to one or more embodiments of the present disclosure, when the second determination module determines the extended luminance section of the initial luminance section based on the image luminance value of the first related image and the image luminance value of the second related image of the image to be synthesized, specifically, determining a difference between an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized; The extended luminance section of the initial luminance section is determined based on the difference value of the image luminance values and a third setting value.
[0155] According to one or more embodiments of the present disclosure, for a luminance value interval of each target pixel point of the image to be synthesized, the maximum luminance value of the luminance value interval is less than or equal to the maximum image luminance value of each image to be synthesized, and the minimum luminance value of the luminance value interval is greater than or equal to the minimum image luminance value of each image to be synthesized.
[0156] According to one or more embodiments of the present disclosure, one or more processors; Memory, and one or more applications stored in the memory and executed by the one or more processors and configured to perform the image processing method illustrated in accordance with the method embodiment.
[0157] According to one or more embodiments of the present disclosure, a computer readable medium is provided having stored thereon a computer program which, when executed by a processor, implements the image processing method illustrated in the method embodiments.
[0158] The above description is merely a description of the preferred embodiment of the present disclosure and the technical principles applied thereto. Those skilled in the art should understand that the scope of the disclosure of the present disclosure is not limited to the technical solution consisting of a specific combination of the above-mentioned technical features, but should also include other technical solutions formed by arbitrarily combining the above-mentioned technical features or their equivalent features without departing from the spirit of the above disclosure. For example, a technical solution formed by replacing the above-mentioned features with technical features having similar functions disclosed in this specification (but not limited thereto).
[0159] Also, although operations are depicted in a particular order, this should not be understood as requiring that these operations be performed in the particular order depicted. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, as described above, some specific implementation details are included, but these should not be construed as limiting the scope of the disclosure. Some features that are described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may be implemented in multiple embodiments alone or in any suitable subcombination.
[0160] Although the subject matter has been described in language specific to structural features and / or methodological logical operations, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. On the contrary, the specific features and operations described above are merely example forms of implementing the claims.
Claims
1. 1. An image processing method, comprising: acquiring at least two images to be combined, the images being images of the same subject captured with different exposures; determining an image luminance value of each of the images to be synthesized; determining a luminance weight of each pixel point in each of the images to be combined based on an image luminance value of the image to be combined and a luminance value of each pixel point in the image to be combined; performing image synthesis processing based on the luminance values of each pixel point in the at least two images to be synthesized and the luminance weights of each pixel point to obtain a target image; Including, The step of determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized and a luminance value of each pixel point in the image to be synthesized includes: determining a first related image of the image to be combined, the first related image being an image to be combined that has an image luminance value adjacent to an image luminance value of the image to be combined; determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be synthesized; Including, A method comprising:
2. determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be synthesized, determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image; For any one pixel point of the image to be synthesized, If the luminance value of the pixel point is located within the luminance value interval, a luminance weight of the pixel point is determined based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, and the luminance value of the pixel point; if the luminance value of the pixel point is outside the luminance value range, determine the luminance weight of the pixel point as a first setting value; and 2. The method according to claim 1 .
3. The step of determining a luminance value range of each target pixel point of the image to be synthesized based on the image luminance value of the image to be synthesized and the image luminance value of the first related image includes: determining an interval consisting of image luminance values of the image to be synthesized and image luminance values of the first related image as a luminance value interval of each target pixel point of the image to be synthesized; 3. The method of claim 2 .
4. determining a luminance weight of each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be synthesized, determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and an image luminance value of a second related image of the image to be synthesized; For any one pixel point of the image to be synthesized, if the luminance value of the pixel point is located within the luminance value interval, determining a luminance weight of the pixel point based on the image luminance value of the image to be synthesized, the image luminance value of the first related image, the image luminance value of the second related image, and the luminance value of the pixel point; if the luminance value of the pixel point is outside the luminance value range, determine the luminance weight of the pixel point as a second set value; and Among them, the second related image is a combination-ready image other than the combination-ready image, the image brightness value of which is adjacent to the image brightness value of the first related image.
2. The method according to claim 1 .
5. The step of determining a luminance value range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and an image luminance value of a second related image of the image to be synthesized, determining an initial luminance range of each target pixel point of the image to be synthesized based on an image luminance value of the image to be synthesized and an image luminance value of the first related image; determining an extended luminance section of the initial luminance section based on an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized; stitching the initial luminance interval and the extended luminance interval to obtain a luminance value interval of each target pixel point of the image to be synthesized; Including, 5. The method according to claim 4 .
6. The step of determining an extended luminance section of the initial luminance section based on an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized includes: determining a difference value between an image luminance value of the first related image and an image luminance value of a second related image of the image to be synthesized; determining an extended luminance section of the initial luminance section based on the difference value of the image luminance values and a third setting value; Including, 6. The method according to claim 5 .
7. For a luminance value interval of each target pixel point of the images to be synthesized, a luminance maximum value of the luminance value interval is equal to or smaller than a maximum value of the image luminance values of each image to be synthesized, and a luminance minimum value of the luminance value interval is equal to or larger than a minimum value of the image luminance values of each image to be synthesized.
7. The method according to claim 2, wherein the first and second electrodes are connected to a first electrode.
8. An image processing device, an acquisition module for acquiring at least two images to be combined, the images being images of the same subject captured with different exposures; a first determination module for determining an image luminance value of each of the to-be-composited images; a second determination module for determining a luminance weight of each pixel point in each of the images to be synthesized based on an image luminance value of the image to be synthesized and a luminance value of each pixel point in the image to be synthesized; a synthesis module for performing image synthesis processing based on the luminance values of each pixel point in the at least two images to be synthesized and the luminance weights of each pixel point to obtain a target image; Including, When determining the luminance weight of each pixel point in the image to be synthesized based on the image luminance value of the image to be synthesized and the luminance value of each pixel point in the image to be synthesized, the second determination module: determining a first related image of the image to be combined, the first related image being an image to be combined that has an image luminance value adjacent to the image luminance value of the image to be combined; and determining a luminance weight for each pixel point in the image to be synthesized based on an image luminance value of the image to be synthesized, an image luminance value of the first related image, and a luminance value of each pixel point in the image to be synthesized; 1. An image processing device comprising:
9. An electronic device, one or more processors; Memory, one or more applications stored in the memory; When executed by the one or more processors, 8. Carrying out the method according to any one of claims 1 to 7 one or more applications, Including, An electronic device comprising:
10. A computer-readable medium having a computer program stored thereon, The computer program, when executed by a processor, causes the computer to carry out the method according to any one of claims 1 to 7. A computer readable medium comprising:
Citation Information
Patent Citations
Image processing apparatus, image processing method, image processing program
JP2009049547A
Image synthesizing apparatus, image synthesizing method, image synthesizing program, and recording medium
JP2013243780A
Image processing device, image processing method and program
JP2015073185A
Cited By
Image processing method and apparatus, electronic device and medium
US12602744B2
