Imaging module, imaging system, image processing method and terminal

Abstract

An imaging module (100), an imaging system (1000), an image processing method and a terminal (5000) are disclosed. The imaging module (100) comprises a module body (10), a lens group (20) received in the module body (10), an image sensor (30) received in the module body (10) and located on an image side of the lens group (20), and a filter (40). The filter (40) is configured to filter at least part of light rays having a wavelength range of 530nm-580nm from entering the image sensor (30).

Description

Technical Field

[0001] This application relates to the field of imaging technology, and more specifically, to an imaging module, an imaging system, an image processing method, and a terminal. Background Technology

[0002] With the growth of digital cameras and camera-equipped mobile phones, people are paying increasing attention to the representation of skin tones in images. Currently, when adjusting skin tones in portrait photography, cameras must first identify the skin-toned areas of the image to distinguish them from other color areas, allowing for targeted adjustments to those areas individually. However, the identification and segmentation of skin-toned areas increases the computational burden, and the identification process is easily affected by lighting conditions, resulting in unsatisfactory efficiency and results in portrait skin adjustment. Summary of the Invention

[0003] This application provides an imaging module, an imaging system, an image processing method, and a terminal.

[0004] The imaging module of this application includes a module body, a lens group, an image sensor, and a filter. The lens group is housed within the module body. The image sensor is housed within the module body and located on the image side of the lens group. The filter is used to filter at least a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm.

[0005] The imaging system of this application includes an imaging component, an imaging module, and one or more processors. The imaging component receives first light to acquire a first image. The imaging module includes a module body, a lens group, an image sensor, and a filter. The lens group is housed within the module body. The image sensor is housed within the module body and located on the image side of the lens group. The filter filters at least a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm. The imaging module receives second light outside the wavelength range of 530nm to 580nm from the first light to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image. One or more processors acquire a target image based on the first image and the second image.

[0006] The imaging system of this application includes an imaging module and a movable filter. The imaging module includes an image sensor. When the filter moves outside the incident light path of the image sensor, the image sensor receives a first light ray to acquire a first image; when the filter moves into the incident light path of the image sensor, the image sensor receives a second light ray outside a specific wavelength range of the first light ray to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image.

[0007] The imaging system of this application includes an imaging component, an imaging module, and one or more processors. The imaging component is used to receive a first light beam to acquire a first image. The imaging module is used to receive a second light beam outside a specific wavelength range of the first light beam to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image. The one or more processors are used to: generate a gain data map based on the pixel values ​​of the pixels in the first image and the pixel values ​​of the corresponding pixels in the second image; and acquire a target image based on the first image and the gain data map.

[0008] The image processing method of this application includes: receiving a first light ray to obtain a first image; receiving a second light ray outside a specific wavelength range of the first light ray to obtain a second image, wherein the pixels of the second image have pixels corresponding to those of the first image; generating a gain data map based on the pixel values ​​of the pixels in the first image and the pixel values ​​of the corresponding pixels in the second image; and obtaining a target image based on the first image and the gain data map.

[0009] The terminal according to this application includes an imaging module, which includes a module body, a lens group, an image sensor, and a filter. The lens group is housed within the module body. The image sensor is housed within the module body and located on the image side of the lens group. The filter is used to filter at least a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm.

[0010] The terminal of this application includes an imaging system, which includes an imaging component, an imaging module, and one or more processors. The imaging component receives first light to acquire a first image. The imaging module includes a module body, a lens group, an image sensor, and a filter. The lens group is housed within the module body. The image sensor is housed within the module body and located on the image side of the lens group. The filter filters at least a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm. The imaging module receives second light outside the wavelength range of 530nm to 580nm from the first light to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image. One or more processors acquire a target image based on the first image and the second image.

[0011] The terminal in this application includes an imaging system, which includes an imaging module and a movable filter. The imaging module includes an image sensor. When the filter moves outside the incident light path of the image sensor, the image sensor receives a first light to acquire a first image; when the filter moves into the incident light path of the image sensor, the image sensor receives a second light outside a specific wavelength range of the first light to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image.

[0012] The terminal in this application includes an imaging system, which includes an imaging component, an imaging module, and one or more processors. The imaging component is used to receive a first light beam to acquire a first image. The imaging module is used to receive a second light beam outside a specific wavelength range of the first light beam to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image. One or more processors are used to: generate a gain data map based on the pixel values ​​of pixels in the first image and the pixel values ​​of pixels at corresponding positions in the second image; and acquire a target image based on the first image and the gain data map.

[0013] The terminal of this application includes one or more processors, which are used to implement the image processing method described in this application. The image processing method includes: receiving a first light ray to obtain a first image; receiving a second light ray outside a specific wavelength range of the first light ray to obtain a second image, wherein the pixels of the second image have pixels corresponding to those of the first image; generating a gain data map based on the pixel values ​​of the pixels in the first image and the pixel values ​​of the corresponding pixels in the second image; and obtaining a target image based on the first image and the gain data map.

[0014] The imaging module, imaging system, image processing method, and terminal of this application filter at least a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm. The main influencing factors on skin tone in human portraits are the content of melanin and hemoglobin. In this wavelength range, the reflectivity of melanin is higher than that of hemoglobin. When a portion of the light entering the image sensor in this wavelength range is filtered by the filter, the difference between the reflectivity of melanin and hemoglobin is reduced. Furthermore, the reduction in the relative sensitivity of the green channel in the image sensor pixels is greater than the reduction in the relative sensitivity of the red and blue channels, resulting in less noticeable melanin and a pinker skin tone. Therefore, after filtering a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm, there is no need to differentiate skin tone regions in the image formed by the imaging module. Skin tone regions can be adjusted individually, reducing the computational burden and obtaining an image with satisfactory skin tone for the user, thus ensuring the efficiency and effectiveness of portrait skin adjustment.

[0015] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0016] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0017] Figure 1 This is a schematic diagram of the imaging module according to certain embodiments of this application;

[0018] Figure 2 This is a schematic diagram showing the relationship between the reflectance and transmittance of light and the corresponding melanin and hemoglobin in different wavelength bands of the imaging module of certain embodiments of this application.

[0019] Figure 3 This is a schematic diagram showing the relationship between light of different wavelengths and the reflectance of corresponding melanin and hemoglobin after the imaging module of certain embodiments of this application is filtered by a filter.

[0020] Figure 4 This is a schematic diagram showing the relationship between light of different wavelengths and the relative sensitivity of the corresponding red, green and blue channels when the imaging module of certain embodiments of this application is not filtered by a filter.

[0021] Figure 5 This is a schematic diagram showing the relationship between the relative sensitivity of light of different wavelengths and the corresponding red, green and blue channels after the imaging module of certain embodiments of this application is filtered by a light filter.

[0022] Figure 6 This is a schematic diagram of the imaging system according to certain embodiments of this application;

[0023] Figure 7 This is a schematic diagram of the lens structure of the imaging module in some embodiments of this application;

[0024] Figure 8 This is a schematic diagram of the structure of the image sensor of the imaging module according to some embodiments of this application;

[0025] Figures 9 to 11 This is a schematic diagram of the installation of the filter of the imaging module in some embodiments of this application;

[0026] Figure 12 This is a schematic diagram of a first and second image of an imaging system according to certain embodiments of this application;

[0027] Figure 13 This is a schematic diagram showing the relationship between light of different wavelengths and the reflectance of corresponding melanin and hemoglobin after the imaging system of certain embodiments of this application has been filtered by two filters.

[0028] Figure 14 This is a schematic diagram showing the relationship between light of different wavelengths and the reflectance of corresponding melanin and hemoglobin after the imaging system of certain embodiments of this application is filtered by three filters.

[0029] Figure 15 This is a schematic diagram of the imaging system according to certain embodiments of this application;

[0030] Figure 16 This is a schematic diagram of the imaging system of some embodiments of this application from another angle;

[0031] Figure 17 and Figure 18 This is a schematic diagram of a scene in which the driving element of an imaging system drives a filter according to certain embodiments of this application;

[0032] Figures 19 to 23 This is a schematic flowchart of an image processing method according to certain embodiments of this application;

[0033] Figure 24 This is a schematic diagram of the structure of a terminal according to certain embodiments of this application;

[0034] Figures 25 to 27 This is a schematic diagram of a terminal scenario according to some embodiments of this application. Detailed Implementation

[0035] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0036] Please see Figure 1 This application provides an imaging module 100. The imaging module 100 includes a module body 10, a lens group 20, an image sensor 30, and a filter 40. The lens group 20 is housed within the module body 10. The image sensor 30 is housed within the module body 10 and located on the image side 202 of the lens group 20. The filter 40 is used to filter at least a portion of the light entering the image sensor 30 in the wavelength range of 530nm to 580nm.

[0037] Specifically, the module body 10 has a receiving space 11, which is used to receive the lens group 20, the image sensor 30 and the filter 40.

[0038] It should be noted that human skin color is composed of pigments such as melanin, hemoglobin, bilirubin, and carotene. Melanin and hemoglobin are the most abundant and variable (due to significant individual differences), and these two largely determine the skin color displayed in the image generated by the imaging module 100. When the reflectivity of melanin is significantly higher than that of hemoglobin, moles and blemishes on the skin will be more prominent.

[0039] Specifically, please refer to Figure 2 and Figure 3 , Figure 2 (a) represents the reflectance of hemoglobin and melanin under different wavelengths of light when the imaging module 100 is not equipped with filter 40. The horizontal axis represents the wavelength of light, the vertical axis represents the reflectance of light, curve H represents the reflectance of hemoglobin under different wavelengths, and curve M represents the reflectance of melanin under different wavelengths. Figure 2 (b) is a graph showing the light after being filtered by filter 40 and the light transmittance. The horizontal axis represents the wavelength of the light and the vertical axis represents the light transmittance. Figure 3 The reflectance of hemoglobin and melanin under different wavelengths of light after the imaging module 100 filters at least a portion of the light in the wavelength range of 530nm to 580nm through the filter 40 is shown. The horizontal axis represents the wavelength of the light and the vertical axis represents the reflectance. Similarly, curve H represents the reflectance of hemoglobin under different wavelengths and curve M represents the reflectance of melanin under different wavelengths.

[0040] pass Figure 2(a) It can be seen that under light irradiation in the wavelength range of 530nm to 580nm, the reflectivity of melanin in this wavelength range is higher than that of hemoglobin. This results in more prominent moles and blemishes in the skin area in the image formed by the imaging module 100. Figure 2 (b) It can be seen that after adding filter 40, the transmittance of light in the 530nm-580nm wavelength range is lower than that of light in other wavelength ranges, meaning that relatively less light in the 530nm-580nm wavelength range enters the image sensor 30. Figure 3 It can be seen that after adding filter 40, under light irradiation in the 530nm–580nm wavelength range, the reflectivity of melanin in this wavelength range begins to approach that of hemoglobin, meaning the difference in reflectivity between the two gradually decreases. This results in moles, blemishes, and other skin features becoming less prominent in the images captured by imaging module 100. Therefore, imaging module 100 should reduce the transmittance in the 530nm–580nm wavelength range to improve the skin appearance in the images captured by imaging module 100.

[0041] Further, please refer to Figure 4 and Figure 5 , Figure 4 The curve represents the relative sensitivity of the R (red) channel, G (green) channel, and B (blue) channel of the imaging module 100 when it is not filtered by the filter 40. Curve R is the relative sensitivity curve of the R channel, curve G is the relative sensitivity curve of the G channel, and curve B is the relative sensitivity curve of the B channel. Figure 5 This indicates the degree of change in the relative sensitivity of the R (red), G (green), and B (blue) channels of the imaging module 100 after filtering by the filter 40. Curves R, G, and B represent the relative sensitivity curves of the R, G, and B channels of the imaging module 100 without filtering by the filter 40, respectively; curves R1, G1, and B1 represent the relative sensitivity curves of the R, G, and B channels of the imaging module 100 after filtering by the filter 40, respectively.

[0042] pass Figure 4 It can be seen that when the imaging module 100 is not filtered by the light filter 40, light with a wavelength range of 530nm to 580nm enters the pixels of the image formed by the image sensor 30, and the relative sensitivity of the G channel is higher than that of the R channel, and also higher than that of the B channel. However, through... Figure 5It can be seen that when the imaging module 100 is filtered by the light filter 40, the light in this wavelength range enters the pixels of the image sensor 30 to form an image. The decrease in the relative sensitivity of the green channel (the difference between the G curve and G1) is greater than the decrease in the relative sensitivity of the red channel (the difference between the R curve and R1), and also greater than the decrease in the relative sensitivity of the blue channel (the difference between the B curve and B1). At this time, compared with the image formed by the imaging module 100 without the light filter 40, the skin color of the image formed by the imaging module 100 will shift to pink, and the skin appearance effect will be better.

[0043] In summary, by combining Figure 2 and Figure 3 It can be seen that when a filter 40 is added to filter at least a portion of the light in the 530nm–580nm wavelength range entering the image sensor 30, the transmittance of light in this wavelength range decreases, and the reflectance of melanin in this wavelength range begins to approach that of hemoglobin. And through Figure 4 and Figure 5 It can be seen that after adding filter 40, although the relative sensitivity of the G channel, R channel and B channel corresponding to light in the wavelength range of 400nm-700nm will decrease, the decrease in the relative sensitivity of the G channel corresponding to light in the wavelength range of 530nm-580nm is greater than the decrease in the relative sensitivity of the R channel and B channel.

[0044] Therefore, by selecting at least a portion of the light in the 530nm–580nm wavelength range filtered by filter 40, it is possible to simultaneously ensure that: the transmittance of light in this wavelength range is reduced; the melanin reflectance corresponding to this wavelength range approaches the reflectance of hemoglobin; and the ratio between the relative sensitivity of the R1 (red) channel and the relative sensitivity of the G1 (green) channel increases, as does the ratio between the relative sensitivity of the B1 (blue) channel and the relative sensitivity of the G1 (green) channel. This results in the skin tone shifting towards pink in the final generated image, leading to better skin appearance.

[0045] The imaging module 100 of this embodiment uses a filter 40 to filter at least a portion of the light entering the image sensor 30 in the wavelength range of 530nm to 580nm. The main influencing factors on skin tone in a human image are the content of melanin and hemoglobin. In this wavelength range, the reflectivity of melanin is higher than that of hemoglobin. When the light entering the image sensor 30 is partially filtered by the filter 40 within this wavelength range, the difference between the reflectivity of melanin and hemoglobin can be reduced. Furthermore, the reduction in the relative sensitivity of the green channel in the pixels of the image sensor 30 is greater than the reduction in the relative sensitivity of the red channel and also greater than the reduction in the relative sensitivity of the blue channel. This results in less noticeable melanin in the skin and a more pinkish skin tone. Therefore, after the light entering the image sensor 30 is filtered by the filter 40 to filter out part of the light in the wavelength range of 530nm to 580nm, there is no need to distinguish the skin color region in the image formed by the imaging module 100. The skin color region can be adjusted separately, thereby reducing the computational burden and obtaining an image with good skin performance that satisfies the user.

[0046] The following explanation, in conjunction with the accompanying drawings, provides further details.

[0047] Please combine Figure 6 This application also provides an imaging system 1000. The imaging system 1000 includes an imaging module 100, an imaging component 200, and one or more processors 300.

[0048] Imaging component 200 is used to receive first light to obtain a first image.

[0049] The imaging module 100 is used to receive a second light outside a specific wavelength range of the first light to obtain a second image, wherein the pixels of the second image have pixels corresponding to those of the first image.

[0050] The specific wavelength range refers to the range where the reflectance of melanin is higher than that of hemoglobin, and this specific wavelength range is located within the red wavelength range. In the embodiments of this application, the specific wavelength range is 530nm to 580nm.

[0051] Please combine Figures 2 to 5 It can be concluded that when the imaging module 100 filters at least a portion of the light in the 530nm to 580nm wavelength range into the image sensor 30 through the filter 40, the transmittance of the light will be reduced, so that the reflectivity of melanin in this wavelength range approaches that of hemoglobin. After filtering out part of the light in this wavelength range, the ratio between the relative sensitivity of the red channel and the relative sensitivity of the green channel increases, and the ratio between the relative sensitivity of the blue channel and the relative sensitivity of the green channel increases, thereby causing the skin color in the final generated target image to shift to pink, resulting in better skin appearance.

[0052] Specifically, the lens group 20 includes at least one lens, and the lens group 20 has an object side 201 and an image side 202. For example... Figure 7 As shown, taking the lens group 20 as an example with only one lens, it can be seen that the positions of the object side 201 and the image side 202 are related to the incident direction of the light. The object side 201 is the side where the light-incident surface 21 is located before the light enters the lens, and the image side 202 is the side where the light-exit surface 22 of the lens is located after the light enters the lens and is converged by the lens and then exits the lens.

[0053] Please combine Figure 1 and Figure 8 The image sensor 30 is located on the image side 202 of the lens group 20. When the image sensor 30 receives the second light after the first light has been filtered by the filter 40, it can acquire a second image. The pixels in the second image have pixels corresponding to those in the first image.

[0054] In one example, the image sensor 30 may include a microlens array 31, a filter array 32, and a pixel array 33. The microlens array 31, the filter array 32, and the pixel array 33 are stacked sequentially.

[0055] Specifically, the microlens array 31 includes multiple microlenses 311, which can converge the light emitted through the lenses to guide more of the incident light to the filter array 32. The filter array 32 includes multiple filters 321, which are used to filter part of the light to allow red, green, and blue (or red, yellow, and blue, or other channels) light to enter the pixel array 31. The pixel array 33 includes multiple pixels 331, which are used to convert the received optical signals into electrical signals.

[0056] The filter array 32 is disposed between the microlens array 31 and the pixel array 33. Each microlens 311 corresponds to a filter 321 and a pixel 331. Along the light-receiving direction of the image sensor 30, the light passes through the microlens 311 to the filter 321, and then passes through the filter 321 to reach the corresponding pixel 331.

[0057] Please refer to the following: Figure 1 The filter 40 is used to filter at least a portion of the light entering the image sensor 30 in the wavelength range of 530nm to 580nm. The filter 40 can be disposed on the outside of the module body 10 or on the inside of the module body 10.

[0058] Specifically, when the filter 40 is located outside the module body 10, the filter 40 needs to be located on the object side 201 of the lens group 20.

[0059] In one implementation, such as Figure 9 As shown in (a), the filter 40 is disposed on the top wall of the module body 10. Before the light enters the module body 10, it will pass through the filter 40 to filter at least a portion of the light in the wavelength range of 530nm to 580nm. When the light enters the module body 10 and enters the image sensor 30, the light entering the image sensor 30 is the second light, and the image sensor 30 can acquire the second image.

[0060] In another embodiment, generally, to ensure that the imaging module 100 is not easily damaged, a protective cover 50 is provided on the outside of the module body 10, and the protective cover 50 does not affect the light received by the imaging module 100. Therefore, when the filter 40 is provided on the outside of the module body 10, such as Figure 9 As shown, the filter 40 can also be integrated into the protective cover 50. Specifically, as... Figure 9 As shown in (b), the filter 40 can be embedded within the protective cover 50; as Figure 9 As shown in (c), the filter 40 can be disposed on the lower surface of the protective cover 50 (the surface near the module body 10); as Figure 9 As shown in (d), the filter 40 can also be disposed on the upper surface of the protective cover 50 (the surface away from the module body 10). Thus, before light enters the module body 10, it will also pass through the filter 40 to filter at least a portion of the light in the wavelength range of 530nm to 580nm, ensuring that the light entering the imaging module 100 is already the second type of light, thereby enabling the imaging module 100 to acquire a second image. It should be noted that the protective cover 50 can be a light-transmitting cover integrating a cover glass and a display screen.

[0061] When the filter 40 is disposed inside the module body 10, the filter 40 can be disposed on the object side 201 of the lens group 20, or on the image side 202 of the lens group 20 and located between the lens group 20 and the image sensor 30.

[0062] In one implementation, such as Figure 10 As shown, the filter 40 can be a filter film, and the filter 40 can be disposed on the light-exiting surface 22 or the light-incident surface 21 of any lens in the lens group 20. Figure 7 (As shown). When light passes through the lens group 20, the light is filtered by the filter 40 set on the light-emitting surface 22 or the light-incident surface 21 of the lens, so that the second light entering the image sensor 30 is received to generate a second image.

[0063] In another embodiment, the filter 40 can also be a standalone filter structure integrated onto the image sensor 30. (Please refer to...) Figure 8The filter 40 can be disposed on each microlens 311 of the microlens array 31, that is, the filter 40, the microlens array 31, the filter array 32 and the pixel array 33 are stacked in sequence; the filter 40 can also be integrated on the filter array 32, such as between the filter array 32 and the microlens array 31, or between the filter array 32 and the pixel array 33, that is, the microlens array 31, the filter 40, the filter array 32 and the pixel array 33 are stacked in sequence, or the microlens array 31, the filter array 32 and the pixel array 33 are stacked in sequence, so as to ensure that the light is filtered by the filter 40 before entering the pixel array 33, thereby ensuring that the light entering the pixel array 33 is the second light, so that the imaging device can acquire the second image.

[0064] Furthermore, the filter 40 can also be a standalone filter structure independent of the image sensor 30 and the lens group 20. Similarly, the filter 40 can be disposed outside or inside the module body 10, and the filter 40 is movable. Specifically, when the filter 40 is located outside the incident light path of the image sensor 30, the image sensor 30 acquires the first light ray; when the filter 40 moves into the incident light path of the image sensor 30, the image sensor 30 acquires the second light ray in the first light ray with a wavelength range outside the 530nm to 580nm range.

[0065] In one embodiment, when the filter 40 is disposed on the outside of the module body 10, or when the filter 40 is disposed on the top wall of the module body 10, such as... Figure 9 As shown in (a), the filter 40 can move relative to the top wall of the module body 10, so that the filter 40 is moved outside or inside the incident light path of the image sensor 30, thereby controlling the image sensor 30 to receive the first light or the second light, and thus acquiring the first image or the second image. For example, when the filter 40 is integrated into the protective cover plate 50, the filter 40 can move relative to the protective cover plate 50. For example, when the filter 40 is embedded within the protective cover plate 50, the filter 40 can move within the protective cover plate 50. For example, when the filter 40 is disposed on the upper or lower surface of the protective cover plate 50, the filter 40 can move relative to the protective cover plate 50 from outside the protective cover plate 50, thereby moving the filter 40 outside or inside the incident light path of the image sensor 30.

[0066] In another embodiment, when the filter 40 is housed inside the module body 10, the filter 40 may be disposed between the top wall of the module body 10 and the lens group 20 (e.g., Figure 11 As shown in (a), the filter 40 can be positioned between any two lenses in the lens group 20 (e.g., as shown in (a)). Figure 11 (b) As shown, the filter 40 can also be disposed between the lens group 20 and the image sensor 30 (e.g., as shown in [b]). Figure 1 (As shown). Thus, when light enters the imaging module 100, it is first filtered by the filter 40 before entering the pixel array 33 of the image sensor 30, so that the image sensor 30 can acquire the second light to generate a second image. In addition, the filter 40 can rotate and / or translate relative to the optical axis of the lens group 20, so that the filter 40 can selectively enter outside or within the incident light path of the image sensor 30, so as to selectively acquire the first image or the second image.

[0067] Therefore, when a user uses the imaging system 1000 to capture an image, the user can flexibly choose whether to use the filter 40 to filter the light. When the user chooses not to use the filter 40, both the imaging component 200 and the imaging module 100 acquire the first image to make the captured image more complete, clear, and realistic. When the user chooses to use the filter 40, the imaging component 200 acquires the first image, and the imaging module 100 acquires the second image. Since the second image is a more pinkish and better-looking image, the imaging system 1000 can fuse the first image based on the second image to make the skin in the captured image look better.

[0068] Imaging component 200 is used to receive first light to acquire a first image. Imaging component 200 may include a complementary metal-oxide-semiconductor (CMOS) photosensitive element or a charge-coupled device (CCD) photosensitive element. The number of pixels in the first image acquired by imaging component 200 may be greater than or equal to the number of pixels in the second image acquired by imaging module 100, so that the pixels in the second image have pixels corresponding to those in the first image.

[0069] One or more processors 300 are used to acquire a target image based on a first image and a second image. Specifically, the first image is generated by the imaging system 1000 receiving first light through the imaging component 200, while the second image is generated by the imaging system 1000 receiving second light outside the wavelength range of 530nm to 580nm from the first light through the imaging module 100 after the first light has been filtered by the filter 40. It can be seen that the difference between the first image and the second image is the different received light. Therefore, the gain of the pixels in the second image relative to the corresponding pixels in the first image at the R, G, and B pixels can be obtained. One or more processors 300 can then change the pixel values ​​of the R, G, and B pixels in the first image based on the first image and this gain, thereby acquiring the target image.

[0070] Please see Figure 12After the imaging system 2000 acquires the first image 60 and the second image 70, it can obtain the pixel values ​​of the R channel, G channel, and B channel in each pixel of the first image 60 and the second image 70, so as to... Figure 12 Taking the pixels P00, P01, P10, P11 (the first number of the subscript represents the row, and the second number represents the column) of the first image 60 and the corresponding pixels P00', P01', P10', P11' in the second image 70 as examples, for instance, in the first image 60, pixel P00 is the R channel with a pixel value of R1; P01 and P10 are the G channels with pixel values ​​of G1 and G1' respectively; and P11 is the B channel with a pixel value of B1. In the second image 70, pixel P00' is the R channel with a pixel value of R2; P01' and P10' are the G channels with pixel values ​​of G2 and G2' respectively; and P11' is the B channel with a pixel value of B2. Then, by comparing the ratio of each pixel in the second image 70 to the corresponding pixel in the first image 60, the gain of the second image 70 relative to the first image 60 can be obtained. For example, the gain value between pixel P00 and pixel P00' is R2 / R1, the gain value between pixel P01 and pixel P01' is G2 / G1, the gain value between pixel P10 and pixel P10' is G2' / G1', and the gain value between pixel P11 and pixel P11' is B2 / B1. Thus, it can be seen that after adding a filter 40, the pixel values ​​of all pixels in the generated second image 70 are changed relative to the corresponding positions of pixels in the first image 60. Based on this change, a comparison image 80 for each pixel in the first image 60 is generated.

[0071] Please refer to this again. Figure 2 (a) Figure 3 , Figure 13 and Figure 14 , Figure 13 The reflectance of hemoglobin and melanin under different wavelengths of light when two filters 40 are set is shown on the x-axis, which represents the wavelength of light and the y-axis represents the reflectance. Figure 14 The graph shows the reflectance of hemoglobin and melanin under different wavelengths of light when three filters (40°) are set. The horizontal axis represents the wavelength of light, and the vertical axis represents the reflectance. Figure 2 (a) and Figure 3 It can be seen that after adding a filter, compared to Figure 2 (a) In the wavelength range of 530nm–580nm, the difference between the reflectance of melanin and the reflectance of hemoglobin decreases. Furthermore, in combination with... Figure 13 and Figure 14It can be concluded that as the number of filters 40 increases, the melanin reflectance corresponding to this wavelength range approaches the reflectance of hemoglobin. Therefore, a greater number of filters 40 results in better skin representation in the target image. Similarly, a greater number of filters 40 leads to a larger ratio between the relative sensitivity of the red channel and the relative sensitivity of the green channel, and also a larger ratio between the relative sensitivity of the blue channel and the relative sensitivity of the green channel, resulting in a pinker skin tone in the target image.

[0072] Since the target image should be the image desired by the user, generally, the imaging module 100 contains only one filter 40. To obtain an image that is easy for the user to adjust so that the user obtains the desired image, the pixel values ​​of the R, G, and B channels of each pixel in the target image can be obtained according to the following formula. The target image is obtained by adjusting the pixel values ​​based on the first image according to the user's actual needs.

[0073]

[0074] Where Rnew is the pixel value of each R pixel channel in the target image, R1 is the pixel value of each R pixel channel in the first image, R2 is the pixel value of each R pixel channel in the second image, and N is the multiplier that the user needs to adjust according to actual needs. N can be an integer, a decimal, or a negative number. This refers to the gain data for the R pixel channel. The pixel value calculation formulas for the G and B pixel channels are similar to those for the R pixel channel, and are respectively... Therefore, we can obtain the pixel values ​​that include all pixels in the target image.

[0075] Based on the above formula, it can be seen that when the user selects N=0 on the terminal 5000 using the imaging system 1000, then Rnew=R1, meaning the target image is the first image, indicating that the target image required by the user does not need to be filtered by filter 40. When N=1 is selected, then Rnew=R2, meaning the target image is the second image, indicating that the target image required by the user needs to be filtered by one filter 40. Furthermore, the larger the N selected by the user, the more filters 40 will be applied to the target image.

[0076] Therefore, when a user needs to represent skin tone better, they can choose a larger value for N, and N can be a decimal. This allows the user to adjust the target image to a more satisfactory level. Thus, by adding a filter 40 to the imaging system 1000, the target image generated by the imaging system 1000 can exhibit the imaging effect of adding multiple filters 40.

[0077] Please see Figure 15This application also provides an imaging system 2000, which may include an imaging module 100, a movable filter 40, one or more processors 300 and a driver 600.

[0078] Please combine Figure 16 The imaging module 100 may include a module body 10, a lens group 20, and an image sensor 30. The module body 10 has a receiving space 11 for accommodating the lens group 20 and the image sensor 30. A filter 40 is used to filter a portion of the first light entering the image sensor 30 within a specific wavelength range, so that the image sensor 30 can receive a second light.

[0079] The specific wavelength range refers to the range where the reflectance of melanin is higher than that of hemoglobin, and this specific wavelength range is located within the red wavelength range. In the embodiments of this application, the specific wavelength range is 530nm to 580nm.

[0080] Please combine Figures 2 to 5 It can be concluded that when the imaging module 100 filters at least a portion of the light in the band range of 530nm to 580nm into the image sensor 30 through the filter 40, the transmittance of the light will be reduced, and the reflectivity of melanin in this band range will approach the reflectivity of hemoglobin. After filtering out part of the light in this band range, the ratio between the relative sensitivity of the red channel and the relative sensitivity of the green channel increases, and the ratio between the relative sensitivity of the blue channel and the relative sensitivity of the green channel increases, thereby causing the skin color in the final generated target image to shift to pink, resulting in better skin appearance.

[0081] Specifically, the lens group 20 includes at least one lens, and the lens group 20 has an object side 201 and an image side 202. For example... Figure 7 As shown, taking the lens group 20 as an example with only one lens, it can be seen that the positions of the object side 201 and the image side 202 are related to the incident direction of the light. The object side 201 is the side where the light-incident surface 21 is located before the light enters the lens, and the image side 202 is the side where the light-exit surface 22 of the lens is located after the light enters the lens and is converged by the lens and then exits the lens.

[0082] Please combine Figure 1 and Figure 8 The image sensor 30 is located on the image side 202 of the lens group 20. When the image sensor 30 receives the first light that has not been filtered by the filter 40, it can acquire a first image. When the image sensor 30 receives the second light that has been filtered by the filter 40, it can acquire a second image. The pixels in the second image have pixels corresponding to those in the first image.

[0083] In one example, the image sensor 30 includes a microlens array 31, a filter array 32, and a pixel array 33. The microlens array 31, the filter array 32, and the pixel array 33 are stacked sequentially.

[0084] Specifically, the microlens array 31 includes multiple microlenses 311, which can converge the light emitted through the lenses to guide more of the incident light to the filter array 32. The filter array 32 includes multiple filters 321, which are used to filter part of the light to allow red, green, and blue (or red, yellow, and blue, or other channels) light to enter the pixel array 31. The pixel array 33 includes multiple pixels 331, which are used to convert the received optical signals into electrical signals.

[0085] The filter array 32 is disposed between the microlens array 31 and the pixel array 33. Each microlens 311 corresponds to a filter 321 and a pixel 331. Along the light-receiving direction of the image sensor 30, the light passes through the microlens 311 to the filter 321, and then passes through the filter 321 to reach the corresponding pixel 331.

[0086] Please combine Figure 15 and Figure 16 The drive unit 600 is used to drive the filter 40 to rotate and / or translate relative to the optical axis 23 of the lens group 20, so that the filter 40 selectively enters or is outside the incident light path of the image sensor 30.

[0087] Specifically, the drive unit 600 may be provided with a connector 601 connected to the filter 40, and the connector 601 may rotate or translate relative to the optical axis of the lens group 20 on the drive unit 600, so as to drive the filter 40 to rotate or translate relative to the optical axis 23 of the lens group 20.

[0088] In one implementation, such as Figure 16 As shown, when the driving member 600 drives the filter 40 to translate relative to the optical axis 23 of the lens group 20, moving it away from the module body 10, the connecting member 601 moves away from the module body 10, so that the filter 40 is located outside the incident light path of the image sensor 30. At this time, the first light entering the imaging module 100 will not be filtered by the filter 40, and the light received by the image sensor 30 is the first light, so that the image sensor 30 can acquire the first image.

[0089] In another implementation, such as Figure 17As shown, when the driving member 600 drives the filter 40 to translate relative to the optical axis 23 of the lens group 20, moving it closer to the module body 10, the connecting member 601 moves closer to the module body 10, and the filter 40 enters the incident light path of the image sensor 30. At this time, the first light entering the imaging module 100 will be filtered by the filter 40 to filter a portion of the light within a specific wavelength range, so that the image sensor 30 can receive the second light and acquire the second image.

[0090] In some embodiments, the drive element 600 may also be a drive element 600 disposed within the imaging module 100 to drive the filter 40 to translate within the imaging module 100. For example... Figure 18 As shown in the left figure, when the driving unit 600 does not drive the filter 40 to rotate relative to the optical axis 23 of the lens group 20, the filter 40 can be located between the lens group 20 and the image sensor 30, and the filter 40 is located in the incident light path of the image sensor 30, so that the light entering the image sensor 30 is the second light, so that the image sensor 30 can acquire the second image. When the driving unit 600 drives the filter 40 to rotate relative to the optical axis of the lens group 20 (such as...), Figure 18 As shown in the right figure, the filter 40 can be positioned outside the incident light path of the image sensor 30, so that the light entering the image sensor 30 is the first light, enabling the image sensor 30 to acquire the first image. Similarly, the driving member 600 and the filter 40 can also be positioned between any two lens groups 20 in the lens group 20, and can selectively enter outside or within the incident light path of the image sensor 30, so that the image sensor 30 can selectively acquire the first image or the second image.

[0091] The filter 40 is disposed on the drive member 600 so that it can be moved by the drive member 600. When the filter 40 moves outside the incident light path of the image sensor 30, the image sensor 30 is used to receive the first light to obtain the first image. When the filter 40 moves into the incident light path of the image sensor 30, the image sensor 30 is used to receive the second light outside a specific wavelength range of the first light to obtain the second image. The pixels of the second image have pixels corresponding to those of the first image.

[0092] The filter 40 can be disposed on the outside of the module body 10, or the filter 40 can be housed inside the module body 10.

[0093] In one embodiment, when the filter 40 is disposed outside the module body 10, the filter 40 can be disposed on the top wall of the module body 10. Before light enters the module body 10, it will first pass through the filter 40 to filter at least a portion of the light within a specific wavelength range. Then, when the light enters the module body 10 and enters the image sensor 30, the light entering the image sensor 30 is the second light, and the image sensor 30 can acquire the second image. At this time, as... Figure 14 As shown, the driving member 600 is located on one side outside the module body 10. The driving member 600 can drive the filter 40 to translate relative to the optical axis 23 of the lens group 20, so that the filter 40 translates on the top wall of the module body 10, thereby selectively blocking the top wall of the module body 10. Then the light entering the image sensor 30 can be the first light or the second light, and the image sensor 30 can acquire the first image or the second image.

[0094] In another embodiment, generally, to ensure that the imaging module 100 is not easily damaged, a protective cover 50 is provided on the outside of the module body 10, and the protective cover 50 does not affect the light received by the imaging module 100. Therefore, when the filter 40 is provided on the outside of the module body 10, such as Figure 9 As shown, the filter 40 can also be integrated into the protective cover 50. Specifically, as... Figure 9 As shown in (b), the filter 40 can be embedded within the protective cover 50; as Figure 9 As shown in (c), the filter 40 can be disposed on the lower surface of the protective cover 50 (the surface away from the module body 10); as Figure 9 As shown in (d), the filter 40 can also be disposed on the lower surface of the protective cover 50 (the surface near the module body 10). Thus, before light enters the module body 10, it will also pass through the filter 40 to filter at least a portion of the light within a specific wavelength range, ensuring that the light entering the imaging module 100 is already the second type of light, thereby enabling the imaging module 100 to acquire a second image. It should be noted that the protective cover 50 can be a light-transmitting cover integrating a cover glass and a display screen.

[0095] Furthermore, the drive element 600 can also be disposed on the protective cover plate 50, and the drive element 600 can drive the filter 40 to rotate and / or translate relative to the optical axis of the lens group 20. For example, when the filter 40 is embedded in the protective cover plate 50, the drive element 600 can drive the filter 40 to translate within the protective cover plate 50, so that the filter 40 enters or exits the optical path of the image sensor 30. As another example, when the filter 40 is disposed on the upper or lower surface of the protective cover plate 50, the drive element 600 can drive the filter 40 to translate on the protective cover plate 50, so that the filter 40 enters or exits the optical path of the image sensor 30, thereby selectively acquiring a first image or a second image.

[0096] In another embodiment, when the filter 40 is housed inside the module body 10, the filter 40 may be disposed between the top wall of the module body 10 and the lens group 20 (e.g., Figure 11 As shown in (a), the filter 40 can be positioned between any two lenses in the lens group 20 (e.g., as shown in (a)). Figure 11 (b) As shown, the filter 40 can also be disposed between the lens group 20 and the image sensor 30 (e.g., as shown in [b]). Figure 1 (As shown). Thus, when light enters the imaging module 100, it is first filtered by the filter 40 before entering the pixel array 33 of the image sensor 30, so that the image sensor 30 can acquire the second light to generate a second image. At this time, the driving member 600 is also housed inside the module body 10 to drive the filter 40 to rotate and / or translate relative to the optical axis 23 of the lens group 20, so that the filter 40 selectively enters outside or within the incident light path of the image sensor 30, thereby selectively acquiring the first image or the second image.

[0097] Therefore, when a user takes an image using the imaging system, the user can flexibly choose whether to use filter 40 to filter the light. When the user chooses not to use filter 40, the imaging system 2000 acquires the first image to make the captured image closer to reality. When the user chooses to use filter 40, the imaging system 2000 acquires the second image. Since the second image is a pinker and better-looking image, the imaging system 2000 can also acquire the first image and the second image separately, and fuse the first image based on the second image to make the skin in the captured image look better.

[0098] One or more processors 300 are configured to generate a gain data map based on the pixel values ​​of pixels in a first image and the pixel values ​​of pixels at corresponding positions in a second image; and to obtain a target image based on the first image and the gain data map.

[0099] Please see Figure 12 After the imaging system 2000 acquires the first image 60 and the second image 70, it can obtain the pixel values ​​of the R channel, G channel, and B channel in each pixel of the first image 60 and the second image 70, so as to... Figure 12Taking the pixels P00, P01, P10, P11 (the first number of the subscript represents the row, and the second number represents the column) of the first image 60 and the corresponding pixels P00', P01', P10', P11' in the second image 70 as examples, for instance, in the first image 60, pixel P00 is the R channel with a pixel value of R1; P01 and P10 are the G channels with pixel values ​​of G1 and G1' respectively; and P11 is the B channel with a pixel value of B1. In the second image 70, pixel P00' is the R channel with a pixel value of R2; P01' and P10' are the G channels with pixel values ​​of G2 and G2' respectively; and P11' is the B channel with a pixel value of B2. Then, by comparing the ratio of each pixel in the second image 70 to the corresponding pixel in the first image 60, the gain of the second image 70 relative to the first image 60 can be obtained. For example, the gain value between pixel P00 and pixel P00' is R2 / R1, the gain value between pixel P01 and pixel P01' is G2 / G1, the gain value between pixel P10 and pixel P10' is G2' / G1', and the gain value between pixel P11 and pixel P11' is B2 / B1. Thus, it can be seen that after adding a filter 40, the pixel values ​​of all pixels in the generated second image 70 are changed relative to the corresponding positions of pixels in the first image 60. Based on this change, a comparison image 80 for each pixel in the first image 60 is generated.

[0100] Please refer to this again. Figure 2 (a) Figure 3 , Figure 13 and Figure 14 , Figure 13 The reflectance of hemoglobin and melanin under different wavelengths of light when two filters 40 are set is shown on the x-axis, which represents the wavelength of light and the y-axis represents the reflectance. Figure 14 The graph shows the reflectance of hemoglobin and melanin under different wavelengths of light when three filters (40°) are set. The horizontal axis represents the wavelength of light, and the vertical axis represents the reflectance. Figure 2 (a) and Figure 3 It can be seen that after adding a filter, compared to Figure 2 (a) In the wavelength range of 530nm–580nm, the difference between the reflectance of melanin and the reflectance of hemoglobin decreases. Furthermore, in combination with... Figure 13 and Figure 14 It can be concluded that as the number of filters 40 increases, the melanin reflectance corresponding to this wavelength range approaches the reflectance of hemoglobin. Therefore, a greater number of filters 40 results in better skin representation in the target image. Similarly, a greater number of filters 40 leads to a larger ratio between the relative sensitivity of the red channel and the relative sensitivity of the green channel, and also a larger ratio between the relative sensitivity of the blue channel and the relative sensitivity of the green channel, resulting in a pinker skin tone in the target image.

[0101] Since the target image should be the image desired by the user, generally, the imaging module 100 contains only one filter 40. To obtain an image that is easy for the user to adjust so that the user obtains the desired image, it is necessary to calculate and adjust the pixel values ​​of the R, G, and B channels of each pixel in the target image. The specific calculation method is the same as the calculation method described above, and will not be repeated here.

[0102] Therefore, by adding a filter 40 to the imaging system 2000, the target image generated by the imaging system 2000 can exhibit the imaging effect of having multiple filters 40 added.

[0103] Please combine Figure 19 This application also provides an image processing method, which includes the following steps:

[0104] 01: Receive the first light beam to obtain the first image;

[0105] 03: Receive a second light source after filtering or partially filtering a specific wavelength range of light from the first light source to obtain a second image, wherein the pixels of the second image have pixels corresponding to those of the first image;

[0106] 07: Generate a gain data map based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image; and

[0107] 09: Obtain the target image based on the first image and gain data graph.

[0108] Specifically, the first image and the second image can be generated by the imaging component 200 and the imaging module 100 of the above embodiments, respectively, or both can be generated by the imaging module 100 (when the filter 40 can be selectively moved into or outside the light-receiving optical path of the image sensor 30). Furthermore, the pixels of the second image generated by the imaging module 100 have pixels corresponding to those of the first image, meaning the number of pixels in the second image is at least equal to the number of pixels in the first image.

[0109] The first light source can be ambient light, or light that has been filtered by the filter of the image sensor 30 to produce infrared or ultraviolet light. The second light source is obtained by filtering the first light source through the filter 40, or by partially filtering light within a specific wavelength range. Therefore, the first image is the original image captured by the imaging system 1000, and the second image is the improved image obtained by the imaging system 1000 after filtering a portion of the light within a specific wavelength range through the filter 40.

[0110] The specific wavelength range refers to the range where the reflectance of melanin is higher than that of hemoglobin, and this specific wavelength range is located within the red wavelength range. In the embodiments of this application, the specific wavelength range is 530nm to 580nm.

[0111] according to Figures 2 to 5 It can be concluded that, compared to the first image, the skin tone in the second image shifts towards pink and suppresses the reflectivity of melanin in the skin, resulting in better skin appearance in the second image.

[0112] Furthermore, when the first image and the second image are generated by the imaging component 200 and the imaging module 100 of the above-described embodiments, respectively, after acquiring the first image and the second image, since the first image and the second image are acquired based on different components, the shooting positions (due to shooting parallax) and the time of generation of the first image and the second image cannot be exactly the same. Therefore, before calculating the gain data map, it is necessary to use methods such as scale-invariant feature transform (SIFT) to align the pixels in the second image with the corresponding pixels in the first image. Thus, the gain data map can be generated based on the pixel values ​​of the pixels in the first image and the pixel values ​​of the corresponding pixels in the second image.

[0113] More specifically, taking corresponding pixels in the first and second images as examples, by comparing the pixel values ​​of the R, G, and B pixel channels in the second image with those in the first image, the gain data of the pixel values ​​in the second image relative to the first image can be obtained. Thus, after comparing all corresponding pixels in the first and second images, the ratio of the pixel values ​​in the second image to those in the first image can be obtained. This ratio indicates the change in the pixel value of each pixel in the second image generated after filtering by filter 40 relative to the pixel value of the corresponding pixel in the first image generated without filtering by filter 40.

[0114] Please refer to this again. Figure 2 (a) Figure 3 , Figure 17 and Figure 18It can be concluded that as the number of filters 40 increases, the melanin reflectance corresponding to this wavelength range approaches the reflectance of hemoglobin. Therefore, the more filters 40 there are, the pinker the skin tone appears in the target image, resulting in better and healthier skin. Thus, the target image, as the image desired by the user, needs further adjustment of the aforementioned ratio, such as by increasing it exponentially or by multiples, to obtain a gain data map. Based on this gain data map, the pixel value of each pixel in the first image is adjusted to obtain the pixel value of each pixel in the target image that meets the user's expectations.

[0115] In the image processing method of this application embodiment, a first image and a second image are obtained by receiving a first light and a second light, respectively. A gain data map is then generated based on the pixel values ​​of corresponding pixels in the first and second images. Finally, a target image is obtained based on the first image and the gain data map. Since the second light is obtained by filtering or partially filtering light within a specific wavelength range based on the first light, and for the reasons mentioned above, the generated second image does not require prior identification and differentiation of skin color areas. Therefore, the skin color areas in the second image show better and healthier skin compared to the first image. Thus, by adjusting the pixel values ​​of each pixel in the first image using the gain data map, the obtained target image can show better and healthier skin.

[0116] Please combine Figure 20 In some methods, step 07: generating a gain data map based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image, further includes the following steps:

[0117] 071: Obtain the ratio of the pixel value of a pixel in the first image to the pixel value of the corresponding pixel in the second image, in order to generate a ratio map;

[0118] 073: Generate a gain data graph based on the ratios in the ratio graph and the pre-selected adjustment coefficients.

[0119] After acquiring the first and second images, a ratio map (such as...) can be generated based on the ratio of pixel values ​​in the first image to the pixel values ​​of corresponding pixels in the second image. Figure 12 (c) is shown.

[0120] Specifically, after acquiring the first image and the second image, the pixel value of each pixel in the first image and the second image can be obtained, so as to... Figure 12Taking the pixels P00, P01, P10, P11 (the first number of the subscript represents the row, and the second number represents the column) of the first image 60 and the corresponding pixels P00', P01', P10', P11' in the second image 70 as examples, for instance, in the first image 60, pixel P00 is the R channel with a pixel value of R1; P01 and P10 are the G channels with pixel values ​​of G1 and G1' respectively; and P11 is the B channel with a pixel value of B1. In the second image 70, pixel P00' is the R channel with a pixel value of R2; P01' and P10' are the G channels with pixel values ​​of G2 and G2' respectively; and P11' is the B channel with a pixel value of B2. Then, by comparing the ratio of each pixel in the second image 70 to the corresponding pixel in the first image 60, the gain of the second image 70 relative to the first image 60 can be obtained. For example, the gain value between pixels P00 and P00' is R2 / R1, the gain value between pixels P01 and P01' is G2 / G1, the gain value between pixels P10 and P10' is G2' / G1', and the gain value between pixels P11 and P11' is B2 / B1. Thus, a ratio map (such as...) can be generated. Figure 12 (c) As shown, this ratio graph can show the change in pixel value of each pixel in the first image compared to the corresponding pixel in the second image after the first light is filtered by a filter 40.

[0121] Since a single filter 40 is insufficient to accurately capture the user's desired image, adjustments are needed to each ratio in the contrast graph to obtain the target image. This can be achieved by pre-selecting adjustment coefficients based on the ratios in the contrast graph to generate a gain data graph. The specific formula is as follows:

[0122]

[0123] Where Rnew is the pixel value of each R pixel channel in the target image, R1 is the pixel value of each R pixel channel in the first image, R2 is the pixel value of each R pixel channel in the second image, and N is the pre-selected adjustment coefficient, which is the multiple that the user needs to adjust according to actual needs. N can be an integer, a decimal, or a negative number. This refers to the gain data for the R pixel channel. The pixel value calculation formulas for the G and B pixel channels are similar to those for the R pixel channel, and are respectively... Therefore, before obtaining the pixel value of each pixel in the target image, the gain data that needs to be adjusted for each pixel in the target image based on the first image can be obtained by adjusting the size of N. The gain data of multiple pixels constitute the gain data map.

[0124] It should be noted that as N gradually increases from 1, the final target image will appear as an image filtered by a larger number of filters 40, meaning the skin in the target image becomes better and the skin tone becomes more pink; conversely, as N gradually decreases from 1, the final target image will appear as an image not filtered by filters 40, or even an image with reverse optimization, meaning the skin in the target image becomes worse and the skin tone becomes more yellow. In this way, users can flexibly adjust the preset adjustment coefficients to obtain a target image that meets their own needs.

[0125] Please combine Figure 21 In some methods, the first image includes a skin-colored region, the second image includes a skin-colored region, and step 07: generating a gain data map based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image, further including the following steps:

[0126] 075: Generate a gain data map based on the pixel values ​​of the pixels in the skin-colored region of the first image and the pixel values ​​of the corresponding pixels in the skin-colored region of the second image;

[0127] Step 09: Obtain the target image based on the first image and gain data map, which also includes the following steps:

[0128] 091: Adjust the pixel values ​​of the skin color region in the first image according to the gain data map to obtain the target image.

[0129] Specifically, when both the first image and the second image contain human figures, a gain data map can be generated based solely on the pixel values ​​of the skin-colored regions of the human figures in the first image and the corresponding pixel values ​​of the skin-colored regions of the human figures in the second image. As a result, the final target image will only adjust the skin color of the skin-colored regions in the first image based on the skin-colored regions in the first image and the gain data map corresponding to the skin-colored regions, thereby making the skin color of the skin-colored regions in the first image pinker and improving the skin appearance.

[0130] Therefore, the color of the background area in the target image can be maintained without affecting the background color, thus improving the skin area in the target image while ensuring the background color.

[0131] Please combine Figure 22 In some methods, the first image includes skin-colored regions, the second image includes skin-colored regions, and the image processing method further includes the following steps:

[0132] 04: Detect skin color regions in the first and second images.

[0133] Specifically, before generating the gain data map, one or more processors 300 will first detect whether the acquired first image and second image contain skin-colored regions to determine whether to generate the gain data map, and obtain the target image based on the first image and the gain data map.

[0134] In one implementation, if it is detected that one of the first or second images does not contain a skin-colored area, it indicates that the parallax between the first and second images is large or the scene being captured is significantly different and cannot be aligned. In this case, one or more processors 300 will be unable to generate a gain data map corresponding to the skin-colored area, and one or more processors 300 will stop working to prompt the user to retake the shot.

[0135] In another implementation, if no skin color region is detected in either the first image or the second image, it indicates that no individual skin color adjustment is needed. In order to ensure the authenticity of the color of the target image, the processor 300 will not generate a gain data map and will directly output the first image or the second image as the target image.

[0136] In another embodiment, if skin-colored areas are detected in both the first image and the second image, then one or more processors 300 will continue to generate a gain data map and generate a target image based on the first image and the gain data map, thereby individually adjusting the skin-colored areas in the target image to obtain a target image with better skin appearance and a pinker color.

[0137] In another embodiment, one or more processors 300 may further calculate the pixel difference between the average pixel value of all pixels in the first image and the average pixel value of all pixels at the corresponding position in the second image. If the pixel difference is greater than a preset difference, it indicates that the parallax between the first image and the second image is large or the scene being captured is significantly different, making alignment impossible. In this case, one or more processors 300 will be unable to generate a gain data map corresponding to the skin tone area, and will stop working to prompt the user to retake the shot. For example, if the preset difference is 50, the average pixel value of all pixels in the first image is 140, and the average pixel value of all pixels at the corresponding position in the second image is 200, it indicates that the parallax between the first image and the second image is large or the scene being captured is significantly different, making alignment impossible. Alternatively, if the preset difference is 50, the average pixel value of all pixels in the first image is 160, and the average pixel value of all pixels at the corresponding position in the second image is 200, it indicates that the first image and the second image are normal. In this case, one or more processors 300 will begin executing the gain data map and generate a target image based on the first image and the gain data map, thereby individually adjusting the skin tone area in the target image to obtain a target image with better skin tone and a pinker color.

[0138] In another embodiment, if skin-colored regions are detected in both the first and second images, one or more processors 300 will further determine whether the pixel difference between the average pixel value of all pixels in the first image and the average pixel value of the corresponding position in the second image is greater than a preset threshold. If the pixel difference is greater than the preset threshold, it indicates that the skin-colored regions in the first and second images show a significant difference. In this case, the one or more processors 300 will continue to generate a gain data map and generate a target image based on the first image and the gain data map, thereby individually adjusting the skin-colored regions in the target image to obtain a target image with better skin appearance and a pinker color. If the pixel difference is less than the preset threshold, it indicates that the skin-colored regions in the first and second images are similar, and there is no need to generate a gain data map. The imaging system 1000 directly uses the second image as the target image.

[0139] The skin color area can include all areas of human skin exposed in the portrait, such as the face area, the hands, legs or feet exposed outside the clothing, etc., to ensure that when the image is a portrait, the final generated target area will only improve the skin color area in the image and will not affect the effect of areas other than the skin color area.

[0140] In the image processing method of this application, by detecting whether skin-colored areas exist in both the first image and the second image, it is ensured that the final generated target image will only adjust the skin-colored areas and will not affect the color of the non-skin-colored areas, so as to obtain a target image with better skin appearance and pinker color.

[0141] Please combine Figure 23 In some methods, image processing methods also include the following steps:

[0142] 04: Detect skin-colored regions in the first and second images; and

[0143] 06: When skin color regions exist in both the first and second images, detect the gender of the skin color in the first and second images.

[0144] Specifically, after acquiring the first image and the second image, one or more processors 300 will first detect whether the acquired first image and the second image contain skin color regions, and detect the gender to which the skin color in the first image and the second image belongs, in order to determine whether to generate a gain data map, and obtain the target image based on the first image and the gain data map.

[0145] For example, after acquiring the first image and the second image, if one or more processors 300 detect that at least one of the acquired first image and the second image does not contain a skin color area, it indicates that the first image and the second image have a large parallax or the scene being captured is significantly different and cannot be aligned. In this case, one or more processors 300 will not be able to generate a gain data map corresponding to the skin color area, and one or more processors 300 will stop working directly and will no longer determine the gender of the skin color containing the skin color area, and will prompt the user to retake the photo.

[0146] For example, after acquiring the first image and the second image, if one or more processors 300 detect that both the acquired first image and the second image contain skin-colored regions, then one or more processors 300 will continue to detect the gender to which the skin color belongs in the first image and the second image.

[0147] For example, after acquiring the first image or the second image, one or more processors 300 can also calculate the average pixel value of all pixels in the first image and the pixel difference between the average pixel value of all pixels at the corresponding position in the second image. If the pixel difference is greater than a preset difference, it indicates that the parallax between the first image and the second image is large or the scene being captured is significantly different and cannot be aligned. In this case, one or more processors 300 will be unable to generate a gain data map corresponding to the skin color area, and one or more processors 300 will stop working to prompt the user to retake the shot. For example, if the preset difference is 50, the average pixel value of all pixels in the first image is 140, and the average pixel value of the corresponding position in the second image is 200, it indicates that the parallax between the first and second images is large or the scene being captured is significantly different, making alignment impossible. Alternatively, if the preset difference is 50, the average pixel value of all pixels in the first image is 160, and the average pixel value of the corresponding position in the second image is 200, it indicates that the first and second images are normal. One or more processors 300 then begin executing the gain data map and generate a target image based on the first image and the gain data map. This allows for individual adjustment of the skin tone areas in the target image to obtain a target image with better skin texture and a pinker color.

[0148] For example, after acquiring the first and second images, if one or more processors 300 detect that both the first and second images contain skin-colored regions, the processors 300 will further determine whether the pixel difference between the average pixel value of all pixels in the first image and the average pixel value of the corresponding position in the second image is greater than a preset threshold. If the pixel difference is greater than the preset threshold, it indicates that the skin-colored regions in the first and second images show a significant difference. In this case, the processors 300 will continue to generate a gain data map and generate a target image based on the first image and the gain data map, thereby individually adjusting the skin-colored regions in the target image to obtain a target image with better skin appearance and a pinker color. If the pixel difference is less than the preset threshold, it indicates that the skin-colored regions in the first and second images are similar, and there is no need to generate a gain data map; the imaging system 1000 directly uses the second image as the target image.

[0149] Next, when at least one of the skin tones in the first and second images is identified as female, one or more processors 300 will generate a gain data map and, based on the first image and the gain data map, individually adjust the skin tones that are female to obtain a target image with better female skin appearance and a pinker color.

[0150] In the image processing method of this application, by detecting whether the first image and the second image contain skin-colored regions, and when both the first image and the second image contain skin-colored regions, it is further determined whether the gender of the skin color in the first image and the second image is female, so as to achieve skin color adjustment for target images containing female characters separately.

[0151] Please see Figure 24 In some embodiments, this application also provides a terminal 5000, which may include an imaging module 100 of any of the above embodiments. The imaging module 100 may be installed in the housing of the terminal 5000 and may be connected to the motherboard of the terminal 5000.

[0152] Please see Figure 24 In some embodiments, this application may also provide a terminal 5000, which may include the imaging system 1000 of any of the above embodiments. The imaging system 1000 may be installed in the housing of the terminal 5000 and connected to the motherboard of the terminal 5000. The imaging system 1000 is used for imaging.

[0153] Please see Figure 24In some embodiments, this application may also provide a terminal 5000, which may include the imaging system 2000 of any of the above embodiments. The imaging system 2000 may be installed in the housing of the terminal 5000 and connected to the motherboard of the terminal 5000. The imaging system 2000 is used for imaging.

[0154] Please see Figure 24 In some embodiments, this application may also provide a terminal 5000, which includes one or more processors 300. The one or more processors 300 can be used to implement the image processing method of any of the above embodiments. For example, the terminal 5000 can be used to implement one or more steps of steps 01, 03, 04, 06, 07, 071, 072, 075, 09, and 091.

[0155] Please see Figure 25 In one embodiment, after the terminal 5000 acquires an image through the imaging system 1000 or the imaging system 2000 and displays the image on the terminal 5000, the image displayed by the terminal 5000 is only the image filtered by a filter 40, i.e., the second image. At this time, the terminal 5000 may display an adjustment coefficient area 500 (i.e., N), which the user can drag from left to right to increase the value of N, thereby presenting an image filtered by multiple filters 40. When the user adjusts the image to meet their needs, the target image is obtained.

[0156] In addition, such as Figure 26 As shown, the terminal 5000 can also display an adjustment area 700. Before the user drags the adjustment coefficient area 500, the user can select the skin color area 701 in the adjustment area 700 option so that when the user adjusts the value of N, only the skin color area in the image changes.

[0157] Finally, as Figure 27 As shown, the terminal 5000 can also display a gender option 900. Before the user drags the adjustment coefficient area 500, the user can also select female 901 in the gender option 900. After the user selects the skin color area 701 in the adjustment area option 700, when adjusting the value of N, only the skin color of the female in the skin color area of ​​the image will change.

[0158] It should be noted that the user can set the adjustment coefficient area 500, adjustment 700 and gender area 900 in the terminal 5000 before the terminal 5000 acquires the image, so that the image acquired by the terminal 5000 is the image that meets the user's expectations, that is, the target image.

[0159] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0160] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0161] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0162] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An imaging system, characterized by, include: An imaging module, the imaging module including an image sensor; A movable filter is provided. When the filter moves outside the incident light path of the image sensor, the image sensor receives a first light ray to acquire a first image. When the filter moves into the incident light path of the image sensor, the image sensor receives a second light ray outside a specific wavelength range of the first light ray to acquire a second image. The pixels of the second image have pixels corresponding to those of the first image. The specific wavelength range is defined as the wavelength range where the reflectance of melanin is higher than that of hemoglobin. One or more processors are configured to: obtain a gain of all pixels in the second image relative to the pixel values ​​of corresponding pixels in the first image, based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image; and obtain a target image based on the first image and the gain.

2. The imaging system of claim 1, wherein, The specific band range is located within the red band range.

3. The imaging system of claim 1, wherein, The specific wavelength range is 530nm~580nm.

4. The imaging system of claim 1, wherein, The filter is located on the outside of the imaging module.

5. The imaging system of claim 4, wherein, The filter is disposed on the top wall of the imaging module; or The filter is integrated into the protective cover.

6. The imaging system of claim 3, wherein, The filter is housed inside the imaging module.

7. The imaging system of claim 1, wherein, The imaging module also includes: The image sensor is housed within the module body; and A lens assembly is housed within the module body, and the filter is located between the image sensor and the lens assembly.

8. The imaging system of claim 7, wherein, The imaging system also includes: A driving element for driving the filter to rotate and / or translate relative to the optical axis of the lens group, so that the filter selectively enters or exits the incident light path of the image sensor.

9. The imaging system of claim 7, wherein, The filter is disposed outside the module body and located on the object side of the lens group.

10. The imaging system of claim 7, wherein, The filter is a standalone filter structure independent of the image sensor and the lens group.

11. The imaging system according to claim 7, characterized in that, The lens group includes at least one lens, and the filter is a filter film disposed on any one of the lenses.

12. The imaging system according to claim 7, characterized in that, The filter is a single-unit filter structure integrated on the image sensor.

13. An imaging system, characterized in that, include: An imaging component, wherein the imaging component is configured to receive a first light source to acquire a first image; An imaging module is configured to receive a second ray outside a specific wavelength range of the first ray to acquire a second image, wherein the pixels of the second image have pixels corresponding to those of the first image; wherein the specific wavelength range is a wavelength range in which the reflectance of melanin is higher than that of hemoglobin; and One or more processors are configured to: obtain a gain of all pixels in the second image relative to the pixel values ​​of corresponding pixels in the first image, based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image; and obtain a target image based on the first image and the gain.

14. The imaging system according to claim 13, characterized in that, The specific band range is located within the red band range.

15. The imaging system according to claim 13, characterized in that, The specific wavelength range is 530nm~580nm.

16. The imaging system according to claim 13, characterized in that, The imaging module includes: Module body; The lens assembly is housed within the module body; An image sensor is housed within the module body and located on the image side of the lens group; and A filter for filtering at least a portion of the light entering the image sensor in the wavelength range of 530nm to 580nm.

17. The imaging system according to claim 16, characterized in that, The filter is disposed outside the module body and located on the object side of the lens group.

18. The imaging system according to claim 17, characterized in that, The filter is disposed on the top wall of the module body; or The filter is integrated into the protective cover.

19. The imaging system according to claim 16, characterized in that, The filter is housed inside the module body.

20. The imaging system according to claim 19, characterized in that, The filter is a standalone filter structure independent of the image sensor and the lens group.

21. The imaging system according to claim 19, characterized in that, The lens group includes at least one lens, and the filter is a filter film disposed on any one of the lenses.

22. The imaging system according to claim 19, characterized in that, The filter is a single-unit filter structure integrated on the image sensor.

23. The imaging system according to any one of claims 16-20, characterized in that, The filter is movable. When the filter is outside the incident light path of the image sensor, the image sensor is used to acquire the first light ray. When the filter moves into the incident light path of the image sensor, the image sensor is used to acquire the second light ray.

24. An image processing method, characterized in that, include: Receive the first light ray to obtain the first image; A second image is obtained by receiving a second light ray after filtering or partially filtering light of a specific wavelength range from the first light ray, wherein the pixels of the second image have pixels corresponding to those of the first image; wherein the specific wavelength range is the wavelength range in which the reflectance of melanin is higher than that of hemoglobin. Based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image, obtain the gain of all pixels in the second image relative to the pixel values ​​of corresponding pixels in the first image; and The target image is obtained based on the first image and the gain amount.

25. The image processing method according to claim 24, characterized in that, The step of obtaining the gain of all pixels in the second image relative to the pixel values ​​of corresponding pixels in the first image based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image includes: Obtain the ratio of the pixel value of a pixel in the first image to the pixel value of the corresponding pixel in the second image, and generate a ratio map; and A gain data graph is generated based on the ratios in the ratio graph and the pre-selected adjustment coefficients.

26. The image processing method according to claim 24, characterized in that, The first image includes a skin-colored region, and the second image includes a skin-colored region. The step of obtaining the gain of all pixels in the second image relative to the pixel values ​​of corresponding pixels in the first image, based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image, includes: A gain data map is generated based on the pixel values ​​of the pixels in the skin-colored region of the first image and the pixel values ​​of the corresponding pixels in the skin-colored region of the second image. The step of obtaining the target image based on the first image and the gain amount includes: Adjust the pixel values ​​of the skin-colored region in the first image according to the gain data map to obtain the target image.

27. The image processing method according to claim 24, characterized in that, The first image includes skin-colored regions, the second image includes skin-colored regions, and the image processing method further includes: Detect skin-colored regions in the first image and the second image; The step of obtaining the gain of all pixels in the second image relative to the pixel values ​​of the corresponding pixels in the first image based on the pixel values ​​of pixels in the first image and the pixel values ​​of corresponding pixels in the second image; and the step of obtaining the target image based on the first image and the gain are performed when skin-colored regions exist in both the first image and the second image.

28. The image processing method according to claim 24, characterized in that, The image processing method further includes: Detect skin-colored regions in the first image and the second image; When skin-colored regions exist in both the first image and the second image, detect the gender to which the skin color belongs in the first image and the second image. The step of obtaining the gain of all pixels in the second image relative to the pixel values ​​of the corresponding pixels in the first image based on the pixel values ​​of pixels in the first image and the pixel values ​​of the corresponding pixels in the second image; and the step of obtaining the target image based on the first image and the gain are performed when the gender of the skin color is female.

29. A terminal, characterized in that, The terminal includes the imaging system according to any one of claims 1-23; or The terminal includes one or more processors, which are used to implement the image processing method according to any one of claims 24-28.

Images