Imaging method and apparatus, image sensor, imaging device, and electronic device

Abstract

The application discloses an imaging method and device, an image sensor, an imaging equipment, an electronic equipment and a readable storage medium, relates to the technical field of image signal processing, and the method comprises the steps of receiving image data from an image sensor; the image sensor comprises a plurality of first sensing units and a plurality of second sensing units, the first sensing unit is used for transmitting polarized light of a set angle, the second sensing unit is used for transmitting color light of a set color, the image data comprises polarized light from the first sensing unit and color light from the second sensing unit; corresponding first interference data is acquired according to the polarized light of the first sensing unit, color image data is acquired according to the color light of the second sensing unit, second interference data in the color image data is identified according to the first interference data, the second interference data in the color image data is removed, and target imaging data is obtained; and a display image is output according to the target imaging data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image signal processing, and particularly relates to an imaging method and device, an image sensor, an imaging equipment, an electronic device, and a readable storage medium. BACKGROUND

[0002] After the lens module receives the reflected light of the object surface, the color information of the reflected light can be collected to output a color image of the object surface.

[0003] In actual situations, when the object surface is a relatively smooth surface (for example, a glass surface) or the object surface has defects (for example, scratches), the positions of the mirror reflection of the object surface or the positions of the defects in the color image can have interference information. For example, a user needs to obtain a color holographic image of the object surface, but the positions of the mirror reflection of the object surface in the color image can have image distortion (displayed as white) due to overexposure. For another example, the user wants to remove the scratches on the object surface, but currently, the scratches on the object surface cannot be removed in the process of obtaining the color image of the object surface, which brings great inconvenience to the user. SUMMARY

[0004] The embodiments of the present application provide an imaging method and device, an image sensor, an imaging equipment, an electronic device, and a readable storage medium, which can solve the problem that the interference information caused by light noise in the color image cannot be removed in the prior art.

[0005] In a first aspect, the embodiments of the present application provide an imaging method applied to an image sensor, the image sensor including a plurality of first sensing units and a plurality of second sensing units, the first sensing units being configured to transmit polarized light of a set angle, and the second sensing units being configured to transmit color light of a set color. The method includes: receiving image data from the image sensor, wherein the image data includes the polarized light from the first sensing units and the color light from the second sensing units; obtaining corresponding first interference data according to the polarized light of the first sensing units, and obtaining color image data according to the color light of the second sensing units; identifying second interference data in the color image data according to the first interference data, removing the second interference data in the color image data, and obtaining target imaging data; and outputting a display image according to the target imaging data.

[0006] In a second aspect, an embodiment of the present application provides an imaging device applied to an image sensor, the image sensor comprising a plurality of first sensing units and a plurality of second sensing units, the first sensing units being configured to transmit polarized light of a set angle, and the second sensing units being configured to transmit color light of a set color, the device comprising: a receiving module configured to receive image data from the image sensor, wherein the image data comprises the polarized light from the first sensing units and the color light from the second sensing units; a first obtaining module configured to obtain first interference data corresponding to the polarized light from the first sensing units; a second obtaining module configured to obtain color image data according to the color light from the second sensing units; an identifying module configured to identify second interference data in the color image data according to the first interference data; a removing module configured to remove the second interference data in the color image data to obtain target imaging data; and an output module configured to output a display image according to the target imaging data.

[0007] In a third aspect, an embodiment of the present application provides an image sensor comprising a plurality of first sensing units and a plurality of second sensing units, the first sensing units being configured to transmit polarized light of a set angle, and the second sensing units being configured to transmit color light of a set color, so that an imaging device receives image data from the image sensor, the image data comprising the polarized light from the first sensing units and the color light from the second sensing units, obtains first interference data corresponding to the polarized light from the first sensing units, and obtains color image data according to the color light from the second sensing units, identifies second interference data in the color image data, removes the second interference data in the color image data to obtain target imaging data, and outputs a display image according to the target imaging data.

[0008] In a fourth aspect, an embodiment of the present application provides an imaging device comprising the imaging device of the second aspect and the image sensor of the third aspect.

[0009] In a fifth aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the method of the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a readable storage medium, the readable storage medium storing a program or instructions, and the program or instructions, when executed by a processor, implement the steps of the method of the first aspect.

[0011] In a seventh aspect, an embodiment of the present application provides a chip comprising a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to run a program or instructions to implement the method of the first aspect.

[0012] In an eighth aspect, an embodiment of the present application provides a computer program product stored in a storage medium, which is executed by at least one processor to implement the method according to the first aspect.

[0013] In the embodiment of the present application, the image sensor includes a plurality of first sensing units and a plurality of second sensing units, the first sensing units are used to transmit polarized light of a set angle, and the second sensing units are used to transmit color light of a set color. The embodiment of the present application can receive image data from the image sensor; wherein the image data includes polarized light from the first sensing units and color light from the second sensing units; then corresponding first interference data is obtained according to the polarized light of the first sensing units, and color image data is obtained according to the color light of the second sensing units; second interference data in the color image data is identified according to the first interference data; the second interference data in the color image data is removed to obtain target imaging data; finally, a display image is output according to the target imaging data, thereby solving the problem that interference information due to light noise cannot be removed in the color image in the prior art, eliminating the interference information due to light noise in the color image, and improving the definition and color saturation of the image. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a sectional view of a pixel sensing unit in an image sensor in the related art;

[0015] Figure 2 is a sectional view of a pixel sensing unit in a polarization sensor in the related art;

[0016] Figure 3A is a structural schematic diagram of an image sensor according to some embodiments of the present application;

[0017] Figure 3B is a structural schematic diagram of another image sensor according to some embodiments of the present application;

[0018] Figure 3C is a structural schematic diagram of a first sensing part in a further image sensor according to some embodiments of the present application;

[0019] Figure 3D is a structural schematic diagram of a second sensing part in a further image sensor according to some embodiments of the present application;

[0020] Figure 3E is a structural schematic diagram of a further image sensor according to some embodiments of the present application;

[0021] Figure 4 is a method flowchart of an imaging method according to some embodiments of the present application;

[0022] Figure 5is a functional structural diagram of an imaging device according to some embodiments of the present application;

[0023] Figure 6 is a functional structural diagram of an imaging device according to some embodiments of the present application;

[0024] Figure 7 is a functional structural diagram of an electronic device according to some embodiments of the present application;

[0025] Figure 8 is a hardware structural schematic diagram of an electronic device according to some embodiments of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0027] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally represents a "or" relationship between the front and rear associated objects.

[0028] At present, the lens module obtains the color information of the incident light incident on it through the image sensor. For example, the Bayesian filter makes different pixel points only sensitive to one of the three primary colors of red, blue and green light. These pixel points are interleaved together, and the original full-color image is restored through demosaicing interpolation. However, in the above process, the polarization information of the incident light cannot be recorded.

[0029] For example, in the current lens module, the cross-sectional view of each pixel sensing unit (hereinafter referred to as pixel) in the image sensor is as shown in Figure 1 Each pixel is stacked by a microlens 101, a color filter 102, and a photodiode 103. Each pixel can only sense the light intensity of the light signal of one color of red (R), green (G), and blue (B). Four pixels RGGB work together to construct the full-color visual image that we finally see. However, in this imaging process, the polarization characteristics of light cannot be obtained, so the polarization characteristics of light cannot be identified or recorded.

[0030] Polarization Image Sensor (PIS) can visualize the polarization characteristics (light wave vibration direction) of light that cannot be perceived by the human eye, thus achieving tasks that current image sensors cannot do, such as object shape recognition, eliminating interfering reflected light on the surface of water or glass, deformation detection, etc., but cannot restore color images.

[0031] For example, the pixel cross-section diagram of PIS is shown in Figure 2 The core component polarization filter 104 is located between the microlens 101 and the photodiode 103, which can minimize the data error caused by pixel crosstalk problems. The polarization filter is a grating (ultra-thin metal plate, etc. arranged in an array at equal intervals) that processes metal into linear slits, which allows light in a certain electric vector vibration direction (this direction is called polarization direction, in angle units) to pass through, and absorbs (or reflects) light vibrating perpendicular to it. Similar to the current image sensor, each pixel can only record the intensity of light of one color, each pixel of PIS can only record the intensity of polarized light of one of 0°, 45°, 90°, or 135°. Four pixel blocks are needed to work together to achieve polarization imaging. Since the polarization filter 104 cannot filter and screen the wavelength information of light, the PIS can visualize the polarization, brightness and propagation direction of light, but cannot construct the full-color image we see.

[0032] In actual situations, users use mobile phone photography in a variety of scenarios, in the manufacturing industry, intelligent transportation, medical care, security and other fields, users want to obtain color images of the surface of the object on the one hand, and on the other hand, they want to eliminate the positions where mirror reflection occurs on the surface of the object, resulting in image distortion due to overexposure, or scratches and other interference factors on the surface of the object. However, the above two needs of the user cannot be met at the same time.

[0033] Based on the above problems, the embodiment of the present application provides an imaging method and device, an image sensor, an imaging equipment, an electronic equipment and a readable storage medium.

[0034] The image sensor and the imaging method provided by the embodiment of the present application will be described in detail in combination with the drawings and specific embodiments and application scenarios.

[0035] The imaging method in the embodiment of the present application is applied to an image sensor. Therefore, before introducing the imaging method in the embodiment of the present application, the image sensor in the embodiment of the present application is first introduced.

[0036] As shown in Figures 3A-3CAs shown, the image sensor 300 in the embodiment of the present application includes a plurality of first sensing units 301 and a plurality of second sensing units 302. The first sensing units 301 are configured to transmit polarized light of a set angle, and the second sensing units 302 are configured to transmit color light of a set color.

[0037] The set angle can include any one of 0°, 45°, 90°, and 135°.

[0038] The set color can include any one of a first color, a second color, a third color, and a fourth color. The first color, the second color, the third color, and the fourth color can form a Bayer array.

[0039] The first color, the second color, the third color, and the fourth color can be red, green, green, and blue, respectively, for example.

[0040] The color light of the red color is red light, with a wavelength range of 615nm-620nm; the color light of the green color is green light, with a wavelength range of 530nm-540nm; and the color light of the blue color is blue light, with a wavelength range of 460nm-470nm.

[0041] The first color, the second color, the third color, and the fourth color can be Cyan, Yellow, Green, and Magenta, respectively, for another example.

[0042] The first color, the second color, the third color, and the fourth color can be red, green, blue, and Emerald, respectively, for another example.

[0043] In some embodiments, the plurality of first sensing units and the plurality of second sensing units are arranged in a mixed manner in the same plane.

[0044] For example, as shown in Figure 3A and Figure 3B . The first color, the second color, the third color, and the fourth color can be red (marked as R in Figure 3A and Figure 3B ), green (marked as G in Figure 3A and Figure 3B ), green, and blue (marked as B in Figure 3A and Figure 3B ), respectively, for example. Figure 3A and Figure 3B The arrangement of the plurality of first sensing units and the plurality of second sensing units in

[0045] As shown in Figure 3A , a plurality of polarization unit groups 31 and a plurality of color unit groups 32 can be arranged in the image sensor.

[0046] As shown in Figure 3A Each polarization unit group 31 includes four first sensing units 301 with angles of 0°, 45°, 90°, and 135°, respectively. Each color unit group 32 includes four second sensing units 302 with colors of red, green, green, and blue, respectively, forming a Bayer array.

[0047] In the horizontal direction, the polarization unit groups 31 and the color unit groups 32 are arranged alternately; in the vertical direction, the polarization unit groups 31 and the color unit groups 32 are arranged alternately.

[0048] The alternately arranged means that the polarization unit groups 31 and the color unit groups 32 are adjacent, and any two polarization unit groups 31 are not adjacent, and any two color unit groups 32 are not adjacent.

[0049] It can be understood that the colors of the four second sensing units 302 in each color unit group 32 can also be cyan, yellow, green, and magenta, respectively, forming a Bayer array; or the colors of the four second sensing units 302 in each color unit group 32 can be red, green, blue, and turquoise, respectively, forming a Bayer array.

[0050] As shown in Figure 3B A plurality of first mixing unit groups 33 and a plurality of second mixing unit groups 34 are arranged in the image sensor.

[0051] As shown in Figure 3B The first mixing unit group 33 includes two first sensing units 301 and two second sensing units 302, and the second mixing unit group 34 includes two first sensing units 301 and two second sensing units 302.

[0052] The first mixing unit group 33 and the second mixing unit group 34 are arranged alternately only in the horizontal direction or the vertical direction. After the first mixing unit group 33 and the second mixing unit group 34 are overlapped, the colors at the positions of the second sensing units 302 form a Bayer array.

[0053] The angles of the two first sensing units 301 in the first mixing unit group 33 are 45° and 135°, and the angles of the two first sensing units 301 in the second mixing unit group 34 are 0° and 90°. That is, the first mixing unit group 33 and the second mixing unit group 34 include four first sensing units 301 with angles of 0°, 45°, 90°, and 135°.

[0054] The two second sensing units 302 in the first mixing unit group 33 are set to red and blue, and the two second sensing units 302 in the second mixing unit group 34 are set to green.

[0055] Figure 3B Only the case with two hybrid unit groups is shown. In practice, three, four, or even more hybrid unit groups can be set, as long as it is possible to obtain polarization images corresponding to 0°, 45°, 90°, and 135° based on the polarized light obtained from the multiple hybrid unit groups, and to obtain the original color image of the imaged object based on the color light obtained from the multiple hybrid unit groups.

[0056] Figure 3B The design of the mixed unit group is conducive to the application of interpolation algorithms, thereby improving the accuracy of the subsequent output image (corresponding to the image output in step S450).

[0057] In some embodiments, a plurality of first sensing units are arranged in a first plane, a plurality of second sensing units are arranged in a second plane, and the image sensor further includes a beam splitter for emitting a portion of incident light incident thereon along a first direction onto the plurality of first sensing units, and emitting another portion of incident light along a second direction onto the plurality of second sensing units.

[0058] For example, such as Figure 3C As shown, multiple first sensing units 301 are arranged in a first plane to form a first sensing section 304. The set angle of each pair of four adjacent first sensing units 301 is four different set angles, including 0°, 45°, 90°, and 135°.

[0059] like Figure 3D As shown, multiple second sensing units 302 are arranged in a second plane to form a second sensing section 305. The multiple second sensing units 302 form a Bayer array.

[0060] like Figure 3E As shown, the image sensor 300 also includes a beam splitter 303. The beam splitter 303 receives incident light reflected from the image object 310 and emits a portion of the received incident light along a first direction to the first sensing units 304 corresponding to a plurality of first sensing units. The beam splitter 303 emits another portion of the incident light along the first direction to the second sensing units 305 corresponding to a plurality of second sensing units 302.

[0061] Figure 3C The arrangement of the multiple first sensing units 301 is the same as that of the PIS structure in related technologies. Figure 3DThe arrangement of the plurality of second sensing units 302 is the same as that of the image sensor in the related art. In this case, by using the design of the light splitting element, the incident light incident to the lens module can be collected to the plurality of first sensing units 301 and the plurality of second sensing units 302 according to a certain proportion, so that the plurality of first sensing units 301 and the plurality of second sensing units 302 can simultaneously image the same imaging object. While all the necessary information is recorded, higher image resolution is also ensured. Figure 3E The design of the light splitting element can ensure that the incident light incident to the lens module can be collected to the plurality of first sensing units 301 and the plurality of second sensing units 302 according to a certain proportion, so that the plurality of first sensing units 301 and the plurality of second sensing units 302 can simultaneously image the same imaging object. While all the necessary information is recorded, higher image resolution is also ensured.

[0062] Of course, it can be understood that the arrangement of the plurality of second sensing units 302 can also be designed based on an RGBW array (one green pixel (G) in the RGGB array of the Bayer array is modified to a white pixel (W)), an RWB array, or an RYYB (Y is a yellow pixel) array, as long as the original color image of the imaging object can be obtained.

[0063] Figure 4 A method flowchart of an imaging method according to some embodiments of the present application is shown. As applied to an image sensor, as shown in FIG. 4, the method includes the following steps S410-S450. Figure 4

[0064] Step S410: receiving image data from the image sensor; wherein the image data includes polarized light from the first sensing unit and color light from the second sensing unit.

[0065] Step S420: obtaining first interference data according to the polarized light of the first sensing unit, and obtaining color image data according to the color light of the second sensing unit.

[0066] In some embodiments, the polarization angle and the first light intensity of the polarized light of the first sensing unit can be obtained first. In the case that the polarization angle and the first light intensity of the polarized light of all the first sensing units are continuous data, at least one polarization angle corresponding polarization image is obtained according to the polarization angle and the first light intensity of the polarized light of all the first sensing units, for example, four polarization images corresponding to 0° polarized light, 45° polarized light, 90° polarized light, and 135° polarized light are obtained, and finally the interference light intensity corresponding to the interference is determined according to the above polarization images as the first interference data.

[0067] The first interference data is obtained according to the interference, and the interference includes at least one of the following: data specular reflection interference, and surface flaw interference. That is, the first interference data is the data of the image formed by the data specular reflection interference, and / or the data of the image formed by the surface flaw interference. ​

[0068] The manner of determining the interference light intensity corresponding to the set interference as the first interference data according to the polarization image can be various, for example, data classification is performed on the data in the polarization image by means of an image definition evaluation method or a convolutional neural network (such as a gray entropy method, a histogram method, an energy gradient function, deep learning, etc.), to obtain the effective light intensity without interference to imaging and the interference light intensity with set interference to imaging.

[0069] The manner of data classification on the data in the polarization space by means of the image definition evaluation method or the convolutional neural network can be set by a person skilled in the art according to the actual situation, and the embodiments of the present application do not limit this.

[0070] When the color image data is obtained according to the color light of the second sensing unit, the wavelength and the second light intensity of the color light of each second sensing unit can be obtained first, and in the case that the wavelength and the second light intensity of the color light of all the second sensing units are continuous data, the color image data is obtained according to the wavelength and the second light intensity of the color light of each second sensing unit.

[0071] It should be noted that in some embodiments, the polarization angle and the first light intensity of the polarization light of all the first sensing units and the polarization angle and the second light intensity of the polarization light of all the second sensing units are continuous data, in the case that the plurality of first sensing units are arranged in the first plane and the plurality of second sensing units are arranged in the second plane.

[0072] In some embodiments, in the case that the plurality of first sensing units and the plurality of second sensing units are mixedly arranged in the same plane, the polarization angle and the first light intensity of the polarization light of all the first sensing units and the polarization angle and the second light intensity of the polarization light of all the second sensing units are discrete data. That is, the polarization angle of the polarization light of all the first sensing units is obtained, and the obtained data is discrete data; the first light intensity of all the first sensing units is obtained, and the obtained data is discrete data; the polarization angle of the polarization light of all the second sensing units is obtained, and the obtained data is discrete data; the second light intensity of all the second sensing units is obtained, and the obtained data is discrete data. In this case, before step S420 is performed, the first data of the polarization light of all the first sensing units and the second data of the color light of all the second sensing units can be obtained, and then the first data and the second data are subjected to data continuous processing to obtain the first continuous data corresponding to the first data and the second continuous data corresponding to the second data.

[0073] The above continuous processing can be, for example, that the discrete first data and the second data are supplemented with interpolation algorithm to complete the blank information, to obtain the first continuous data corresponding to the first data and the second continuous data corresponding to the second data.

[0074] In the case of obtaining the first continuous data and the second continuous data, in step S420, the first interference data can be acquired according to the first continuous data, and the color image data can be acquired according to the color light of all the second sensing units according to the second continuous data.

[0075] The corresponding first interference data is acquired according to the first continuous data, and the way of acquiring the corresponding first interference data in the case that the polarization angle of the polarized light of all the first sensing units and the first light intensity are continuous data in the above-mentioned embodiments is the same, and specific reference can be made to the corresponding description in the above-mentioned embodiments, which will not be repeated here.

[0076] The color image data is acquired according to the color light of all the second sensing units according to the second continuous data, and the way of acquiring the color image data in the case that the wavelength of the color light of all the second sensing units and the second light intensity are continuous data in the above-mentioned embodiments is the same, and specific reference can be made to the corresponding description in the above-mentioned embodiments, which will not be repeated here.

[0077] Step S430: According to the first interference data, the second interference data in the color image data is identified.

[0078] The target position and the target intensity ratio of the interference light intensity can be acquired first, and the intensity ratio of the light intensity at each position in the polarization image is the ratio of the light intensity to the set light intensity, and then the target position and the target intensity ratio are used to identify the second interference data in the color image data.

[0079] When the target position and the target intensity ratio are used to identify the second interference data in the color image data, the target position and the target intensity ratio can be input into a pre-established function model, and the output result of the function model is the target imaging data obtained by removing the second interference data in the color image data.

[0080] The above-mentioned function model can be obtained by pre-establishing an evaluation function or using a double-flow framework for learning.

[0081] Step S440: The second interference data in the color image data is removed to obtain target imaging data.

[0082] The second interference data in the color image data is removed, and the removal result is used as the target imaging data.

[0083] The way of removing the second interference data in the color image data can be set by those skilled in the art according to the actual situation, and the embodiments of the present application do not limit it.

[0084] Step S450: A display image is output according to the target imaging data.

[0085] In some embodiments, the display image can be output directly according to the target imaging data. For example, the display image needs to be able to mark the position where the mirror reflection occurs, or the display image needs to mark the flaw such as scratch on the object.

[0086] In some other embodiments, the display image can be output after the target imaging data is image reconstructed. In this case, the image in the area blocked by the mirror reflection interference and / or surface flaw interference can be recovered. For example, the part of the object behind the glass is blocked by the light line caused by the mirror reflection on the glass. In this case, after the light line caused by the mirror reflection on the glass is removed, the part blocked by the light line can be recovered according to the remaining data, so that the image output can clearly show the scene that can be presented after the light line caused by the mirror reflection is removed. For another example, the scratch on the object is removed. In this case, the image at the position of the scratch can be reconstructed according to the image data around the position of the scratch, so as to present the scene of the object without the scratch.

[0087] The way of image reconstruction of the target imaging data can be set by those skilled in the art according to actual conditions, and the embodiments of the present application do not limit this.

[0088] In the embodiments of the present application, the image sensor includes a plurality of first sensing units and a plurality of second sensing units. The first sensing units are used to transmit polarized light of a set angle, and the second sensing units are used to transmit color light of a set color. The embodiments of the present application can receive image data from the image sensor. The image data includes polarized light from the first sensing units and color light from the second sensing units. Then, corresponding first interference data is obtained according to the polarized light of the first sensing units, and color image data is obtained according to the color light of the second sensing units. According to the first interference data, second interference data in the color image data is identified. The second interference data in the color image data is removed to obtain target imaging data. Finally, a display image is output according to the target imaging data, thereby solving the problem that the interference information caused by light noise in the color image cannot be removed in the prior art, eliminating the interference information caused by light noise in the color image, and improving the clarity and color saturation of the image.

[0089] In addition, the plurality of first sensing units and the plurality of second sensing units in the embodiments of the present application can be realized by properly designing the microstructure and periodically arranging the method. It is suitable for semiconductor processing technology, and can adopt nano-imprinting technology, combined with deposition, etching and other semiconductor process technology, to realize low-cost large-area sensor processing.

[0090] The imaging method provided by the embodiments of the present application can be executed by the imaging device. The imaging device provided by the embodiments of the present application is described by taking the imaging device as an example.

[0091] Figure 5 Figure 1 is a functional block diagram of an imaging device according to some embodiments of the present application. The imaging device is applied to an image sensor including a plurality of first sensing units and a plurality of second sensing units, the first sensing units are configured to transmit polarized light of a set angle, and the second sensing units are configured to transmit color light of a set color. As shown in Figure 1, the imaging device 100 includes a receiving module 101, a first obtaining module 102, a second obtaining module 103, an identifying module 104, a removing module 105, and a displaying module 106. Figure 5

[0092] The receiving module 101 is configured to receive image data from the image sensor, wherein the image data includes polarized light from the first sensing units and color light from the second sensing units.

[0093] The first obtaining module 102 is configured to obtain corresponding first interference data according to the polarized light from the first sensing units.

[0094] The second obtaining module 103 is configured to obtain color image data according to the color light from the second sensing units.

[0095] The identifying module 104 is configured to identify second interference data in the color image data according to the first interference data.

[0096] The removing module 105 is configured to remove the second interference data in the color image data to obtain target imaging data.

[0097] The displaying module 106 is configured to output a display image according to the target imaging data.

[0098] Optionally, the first interference data includes interference light intensity, and the identifying module is configured to: obtain a target position and a target intensity ratio at which the interference light intensity occurs, the intensity ratio of the light intensity at each position in the polarized image being a ratio of the light intensity to a set light intensity; and identify the second interference data in the color image data according to the target position and the target intensity ratio.

[0099] Optionally, the first obtaining module is configured to: obtain a polarization angle and a first light intensity of the polarized light from the first sensing units; obtain at least one polarized image corresponding to the polarization angle according to the polarization angle and the first light intensity of the polarized light from the first sensing units; and determine the corresponding interference light intensity as the first interference data according to the polarized image.

[0100] Optionally, the displaying module is configured to perform image reconstruction on the target imaging data to output the display image.

[0101] Optionally, the plurality of first sensing units and the plurality of second sensing units are mixedly arranged in the same plane, and the device further includes:

[0102] ​The continuity module is used to acquire the first data of the polarized light of the first sensing unit and the second data of the color light of all the second sensing units before the first acquisition module acquires the corresponding first interference data based on the polarized light of the first sensing unit and the second acquisition module acquires the color image data based on the color light of the second sensing unit; and to perform data continuity processing on the first data and the second data to obtain the first continuous data corresponding to the first data and the second continuous data corresponding to the second data.

[0103] The first acquisition module is used to acquire corresponding first interference data based on the first continuous data, and the second acquisition module is used to acquire color image data based on the color light of the second sensing unit based on the second continuous data.

[0104] The imaging device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0105] The imaging device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0106] The imaging device provided in this application embodiment can achieve... Figure 4 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0107] Figure 6 This is a functional structural block diagram of an imaging device according to some embodiments of this application. For example... Figure 6 As shown, the imaging device 600 includes an imaging unit 500 and an image sensor 300.

[0108] In some embodiments, the first sensing units 301 and the second sensing units 302 in the image sensor 300 are arranged in the same plane. The imaging device 500 further comprises a continuous module, configured to acquire first data of the polarized light of the first sensing units and second data of the color light of all the second sensing units before the first acquisition module acquires the first interference data corresponding to the interference according to the polarized light of the first sensing units and the second acquisition module acquires the color image data according to the color light of the second sensing units; the first data and the second data are subjected to data continuous processing to obtain first continuous data corresponding to the first data and second continuous data corresponding to the second data; the first acquisition module 501 is configured to acquire the first interference data corresponding to the first continuous data, and the second acquisition module 502 is configured to acquire the color image data of the color light of the second sensing units according to the second continuous data.

[0109] In some other embodiments, the first sensing units 301 in the image sensor 300 are arranged in a first plane, and the second sensing units 302 are arranged in a second plane. The first acquisition module 502 is configured to acquire the first interference data corresponding to the polarized light of the first sensing units.

[0110] The first interference data can be acquired according to the set interference, and the set interference includes at least one of the following: specular reflection interference, and surface flaw interference.

[0111] The second acquisition module 503 is configured to acquire the color image data according to the color light of the second sensing units.

[0112] As shown in Figure 7 The embodiments of the present application further provide an electronic device 700, which comprises a processor 701 and a memory 702, and the memory 702 stores programs or instructions executable on the processor 701, and the programs or instructions are executed by the processor 701 to implement the steps of the above imaging method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0113] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0114] Figure 8 A hardware structure schematic diagram of an electronic device for implementing the embodiments of the present application.

[0115] The electronic device 800 includes but is not limited to the following components: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810, etc.

[0116] Those skilled in the art can understand that the electronic device 800 can also include a power supply (such as a battery) for powering various components, and the power supply can be logically connected to the processor 810 through a power management system, so that the power management system can realize the functions of managing charging, discharging, and power consumption management. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than shown, or combine certain components, or different component arrangements, which are not described here.

[0117] The processor 810 is applied to the image sensor, and includes a plurality of first sensing units and a plurality of second sensing units. The first sensing unit is used to transmit polarized light of a set angle, and the second sensing unit is used to transmit color light of a set color. The processor 810 is configured to: receive image data from the image sensor; wherein the image data includes polarized light from the first sensing unit and color light from the second sensing unit; obtain corresponding first interference data according to the polarized light of the first sensing unit, and obtain color image data according to the color light of the second sensing unit; identify second interference data in the color image data according to the first interference data; remove the second interference data in the color image data to obtain target imaging data; and output a display image according to the target imaging data.

[0118] In the embodiment of the present application, the image sensor includes a plurality of first sensing units and a plurality of second sensing units. The first sensing unit is used to transmit polarized light of a set angle, and the second sensing unit is used to transmit color light of a set color. The embodiment of the present application can receive image data from the image sensor; wherein the image data includes polarized light from the first sensing unit and color light from the second sensing unit; then obtain corresponding first interference data according to the polarized light of the first sensing unit, and obtain color image data according to the color light of the second sensing unit; identify second interference data in the color image data according to the first interference data; remove the second interference data in the color image data to obtain target imaging data; and finally output a display image according to the target imaging data, thereby solving the problem that the interference information appearing in the color image due to light noise cannot be removed in the prior art, eliminating the interference information appearing in the color image due to light noise, and improving the clarity and color saturation of the image.

[0119] Optionally, the first interference data includes interference light intensity, and the processor 810 is further configured to: obtain a target position and a target intensity ratio at which the interference light intensity appears, the intensity ratio of the light intensity at each position in the polarized image being a ratio of the light intensity to the set light intensity; and identify the second interference data in the color image data according to the target position and the target intensity ratio.

[0120] Optionally, the processor 810 is further configured to: acquire a polarization angle and a first light intensity of the polarized light of the first sensing unit; acquire a polarization image corresponding to each of the at least one polarization angle according to the polarization angle and the first light intensity of the polarized light of the first sensing unit; and determine the corresponding interference light intensity as the first interference data according to the polarization image.

[0121] Optionally, the processor 810 is further configured to perform image reconstruction on the target imaging data and output a display image.

[0122] Optionally, the plurality of first sensing units and the plurality of second sensing units are arranged in the same plane, and the processor 810 is further configured to: acquire first data of the polarized light of the first sensing unit and second data of the color light of all the second sensing units before acquiring the corresponding first interference data according to the polarized light of the first sensing unit and acquiring the color image data according to the color light of the second sensing unit; and perform data continuity processing on the first data and the second data to obtain first continuity data corresponding to the first data and second continuity data corresponding to the second data; and the processor 810 is further configured to acquire the corresponding first interference data according to the first continuity data and acquire the color image data according to the color light of all the second sensing units according to the second continuity data.

[0123] Optionally, the first interference data is acquired according to a set interference, and the set interference includes at least one of the following: data specular reflection interference, and surface flaw interference.

[0124] It should be understood that, in the embodiments of the present application, the input unit 804 can include a graphics processing unit (GPU) 8041 and a microphone 8042. The graphics processing unit 8041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 806 can include a display panel 8061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 can include a touch detection device and a touch controller. The other input devices 8072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, and the like, which will not be described here.

[0125] The memory 809 can be used to store software programs and various data. The memory 809 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 809 can include a volatile memory or a non-volatile memory, or the memory 809 can include both a volatile memory and a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 809 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0126] The processor 810 can include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 810.

[0127] The embodiments of the present application also provide a readable storage medium, and the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above imaging method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0128] The processor is a processor in the electronic device in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0129] The chip provided in the embodiments of the present application includes a processor and a communication interface. The communication interface is coupled with the processor. The processor is configured to execute programs or instructions to implement various processes of the imaging method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0130] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip, etc.

[0131] The embodiments of the present application provide a computer program product stored in a storage medium. The program product is executed by at least one processor to implement various processes of the imaging method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0132] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from the described order, and various steps can be added, omitted, or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0133] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0134] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. An imaging method characterized by, The application is applied to an image sensor, the image sensor comprises a plurality of first sensing units and a plurality of second sensing units, the first sensing units are used for transmitting polarized light of a set angle, the second sensing units are used for transmitting color light of a set color, and the method comprises the following steps: receiving image data from the image sensor; wherein the image data comprises polarized light from the first sensing units and color light from the second sensing units; acquiring corresponding first interference data according to the polarized light of the first sensing units, and acquiring color image data according to the color light of the second sensing units; identifying second interference data in the color image data according to the first interference data; removing the second interference data in the color image data to obtain target imaging data; outputting a display image according to the target imaging data; the first interference data comprises interference light intensity, and the identification of the second interference data in the color image data according to the first interference data comprises the following steps: acquiring a target position and a target intensity ratio of the interference light intensity, wherein the intensity ratio of the light intensity at each position in the polarized image is the ratio of the light intensity to a set light intensity; identifying the second interference data in the color image data according to the target position and the target intensity ratio; the plurality of first sensing units and the plurality of second sensing units are mixedly arranged in the same plane, and before the acquisition of corresponding first interference data according to the polarized light of the first sensing units and the acquisition of color image data according to the color light of the second sensing units, the method further comprises the following steps: acquiring first data of the polarized light of all the first sensing units and second data of the color light of all the second sensing units; performing data continuous processing on the first data and the second data to obtain first continuous data corresponding to the first data and second continuous data corresponding to the second data; wherein the continuous processing is to complete the blank information in the discrete first data and second data by using an interpolation algorithm; the acquisition of corresponding first interference data according to the first continuous data and the acquisition of color image data according to the color light of all the second sensing units according to the second continuous data. the acquisition of corresponding first interference data according to the polarized light of the first sensing units comprises the following steps:

2. The method of claim 1, wherein, acquiring a polarization angle and a first light intensity of the polarized light of the first sensing units; acquiring at least one polarized image corresponding to each polarization angle according to the polarization angle and the first light intensity of the polarized light of the first sensing units; determining corresponding interference light intensity as the first interference data according to the polarized image. the output of the display image according to the target imaging data comprises the following steps:

3. The method of claim 1, wherein, performing image reconstruction on the target imaging data to output the display image. the first interference data is acquired according to a set interference, and the set interference comprises at least one of the following: data specular reflection interference and surface flaw interference.

4. The method according to any one of claims 1 to 3, characterized in that, ​ 5. An imaging device, characterized by The application is applied to an image sensor, which comprises a plurality of first sensing units and a plurality of second sensing units, the first sensing units are used for transmitting polarized light of a set angle, the second sensing units are used for transmitting color light of a set color, and the device comprises: a receiving module, which is used for receiving image data from the image sensor; wherein the image data comprises polarized light from the first sensing units and color light from the second sensing units; a first obtaining module, which is used for obtaining corresponding first interference data according to the polarized light of the first sensing units; a second obtaining module, which is used for obtaining color image data according to the color light of the second sensing units; an identifying module, which is used for identifying second interference data in the color image data according to the first interference data; a removing module, which is used for removing the second interference data in the color image data to obtain target imaging data; a display module, which is used for outputting a display image according to the target imaging data; the first interference data comprises interference light intensity, the identifying module is used for obtaining a target position and a target intensity ratio of the interference light intensity, wherein the intensity ratio of light intensity at each position in a polarized image is the ratio of the light intensity to a set light intensity; and the second interference data in the color image data is identified according to the target position and the target intensity ratio; the plurality of first sensing units and the plurality of second sensing units are mixedly arranged in the same plane; a continuous module, which is used for obtaining first data of the polarized light of all the first sensing units and second data of the color light of all the second sensing units before the first obtaining module obtains corresponding first interference data according to the polarized light of the first sensing units and the second obtaining module obtains color image data according to the color light of the second sensing units; the first data and the second data are subjected to data continuous processing to obtain first continuous data corresponding to the first data and second continuous data corresponding to the second data; wherein the continuous processing is to complete blank information in discrete first data and second data by using an interpolation algorithm; the first obtaining module is used for obtaining corresponding first interference data according to the first continuous data; the second obtaining module is used for obtaining color image data of all the second sensing units according to the second continuous data.

6. An image sensor, comprising: The imaging device comprises a plurality of first sensing units and a plurality of second sensing units, the first sensing units are used for transmitting polarized light of a set angle, and the second sensing units are used for transmitting color light of a set color, so that the imaging device receives image data from the image sensor, the image data comprises polarized light from the first sensing units and color light from the second sensing units, corresponding first interference data is obtained according to the polarized light of the first sensing units, and color image data is obtained according to the color light of the second sensing units, second interference data in the color image data is identified, the second interference data in the color image data is removed to obtain target imaging data, and a display image is output according to the target imaging data, the first interference data comprises interference light intensity, and the identification of the second interference data in the color image data according to the first interference data comprises: obtaining a target position and a target intensity ratio of the interference light intensity, wherein the intensity ratio of the light intensity at each position in the polarization image is the ratio of the light intensity to a set light intensity; identifying the second interference data in the color image data according to the target position and the target intensity ratio; the plurality of first sensing units and the plurality of second sensing units are mixedly arranged in the same plane, and before the corresponding first interference data is obtained according to the polarized light of the first sensing units and the color image data is obtained according to the color light of the second sensing units, the method further comprises: obtaining first data of the polarized light of all the first sensing units and second data of the color light of all the second sensing units; performing data continuous processing on the first data and the second data to obtain first continuous data corresponding to the first data and second continuous data corresponding to the second data; wherein the continuous processing is to complete the blank information in the discrete first data and the second data by using an interpolation algorithm; the corresponding first interference data is obtained according to the first continuous data, and the color image data is obtained according to the second continuous data. the plurality of first sensing units are arranged in a first plane, the plurality of second sensing units are arranged in a second plane, and the image sensor further comprises a light splitting element, which is used for emitting a part of incident light incident thereon to the plurality of first sensing units in a first direction, and emitting another part of the incident light to the plurality of second sensing units in a second direction.

7. The image sensor of claim 6, wherein, The imaging device comprises the imaging device according to claim 5 and the image sensor according to claim 6.

8. An image forming apparatus characterized by comprising: The imaging device comprises a processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to realize the steps of the imaging method according to any one of claims 1-4.

9. An electronic device, comprising: ​ 10. A readable storage medium, characterized by, The program or instruction is stored on the readable storage medium, and when executed by the processor, the program or instruction implements the steps of the imaging method according to any one of claims 1-4.

Images