Image sensor and control method therefor

Abstract

An image sensor according to one embodiment of the present invention comprises: a light-emitting unit for emitting signals of one frequency; one lens and a plurality of photodiodes for receiving signals reflected from an object; and a controller. For example, the controller groups signals received by the plurality of photodiodes, and calculates the position of the object by using the difference between signals of a first group and signals of a second group.

Description

Image sensor and its control method

[0001] The present invention relates to a technology related to an image sensor, and is applicable to, for example, ToF (Time of Flight), but is not limited thereto.

[0002] Examples of methods by which image sensors acquire 3D images include stereo vision technology, structured light technology, and ToF technology. In particular, ToF technology is divided into indirect ToF (I-ToF) technology, which utilizes phase differences depending on the distance measurement method, and direct ToF (D-ToF) technology, which directly measures time differences.

[0003] Direct ToF technology requires complex hardware and higher power consumption. Indirect ToF technology, on the other hand, is relatively inexpensive and is used in a variety of commercial products.

[0004] For example, indirect ToF technology, an emitter emits light and analyzes the reflected light signal from an object. The phase shift in the received light signal occurs depending on the distance between the ToF system and the object, allowing the distance to be calculated based on the degree of phase shift.

[0005] However, since the phase shift is repeated in a 2π cycle, the measurement range of indirect ToF is expanded using multi-frequency modulation technology according to the prior art.

[0006] However, using multiple frequencies causes problems such as increased latency, which slows down the response speed, and increased power consumption.

[0007] Additionally, the performance of the depth frame rate degrades, and becomes worse when the camera moves or objects move.

[0008] Here, depth frame rate may mean, for example, the number of frames of depth data that can be captured per second using a sensor.

[0009] One embodiment of the present invention is intended to solve the above-described problem and to prevent an increase in power consumption of an image sensor such as an indirect ToF due to the use of multiple frequencies.

[0010] An image sensor according to one embodiment of the present invention for achieving the above-described technical task includes a light emitting unit that emits a signal consisting of a single frequency, a lens and a plurality of photodiodes that receive a signal reflected from an object, and a controller. For example, the controller groups the signals received by the plurality of photodiodes and calculates the position of the object using the difference between the signals of the first group and the signals of the second group.

[0011] Additionally, when the plurality of photodiodes are four, for example, the first group includes two photodiodes located on the left, and the second group includes two photodiodes located on the right.

[0012] Furthermore, the controller may compare, for example, a first amplitude of a signal of the first group with a second amplitude of a signal of the second group.

[0013] Moreover, the controller considers the object to be in an in-focus position, for example, if there is no disparity between the first amplitude and the second amplitude, and considers the object to be in an out-of-focus position, if there is a disparity between the first amplitude and the second amplitude.

[0014] And, the one frequency is characterized in that it is repeated at least twice within the maximum distance of the infinity focus of the one lens, for example.

[0015] Meanwhile, a method for controlling an image sensor according to an embodiment of the present invention includes a step of emitting a signal consisting of one frequency, a step of receiving a signal reflected from an object via one lens and a plurality of photodiodes, a step of grouping signals received by the plurality of photodiodes, and a step of calculating the position of the object using a difference between a signal of a first group and a signal of a second group.

[0016] According to one embodiment of the present invention, unlike the conventional use of multiple frequencies in indirect ToF sensors, etc., there is a technical effect of reducing power consumption by using a single frequency.

[0017] Furthermore, unlike the conventional technology that uses 8 subframes, the present invention has the technical effect of improving the frame rate by using only 4 subframes.

[0018] In addition, according to one embodiment of the present invention, since the integration time is shortened, motion blur and depth error can be reduced. Motion blur occurs when an object moves or a camera including an image sensor moves, and this appears as an error in the depth image.

[0019] Meanwhile, the integration time here refers to the exposure cycle for irradiating a light signal to an object, etc. In other words, it refers to the time for generating a charge by receiving reflected light from a pixel to obtain distance information.

[0020] Figure 1 illustrates multi-frequency used in an image sensor according to the prior art.

[0021] Figure 2 illustrates the process by which an image sensor calculates a distance using one photodiode and one tap.

[0022] Figure 3 shows a graph of a transmission signal and a reception signal obtained through the process illustrated in Figure 2.

[0023] Figure 4 illustrates the process by which an image sensor calculates distance using one photodiode and two taps.

[0024] Figure 5 illustrates one actual location and multiple possible locations of an object away from the image sensor.

[0025] FIG. 6 illustrates a plurality of photodiodes included in an image sensor according to one embodiment of the present invention.

[0026] FIG. 7 illustrates a process in which an image sensor according to one embodiment of the present invention processes signals received by a plurality of photodiodes using one frequency.

[0027] FIG. 8 illustrates a process of grouping signals received by a plurality of photodiodes by an image sensor according to one embodiment of the present invention.

[0028] FIG. 9 illustrates the difference in the amount of light of signals received by a plurality of photodiodes of an image sensor according to one embodiment of the present invention.

[0029] FIG. 10 illustrates a case where a disparity occurs between the amplitudes of signals grouped by an image sensor according to an embodiment of the present invention.

[0030] FIG. 11 illustrates a graph that can be used to estimate the position of an object using the difference between amplitudes by an image sensor according to one embodiment of the present invention.

[0031] FIG. 12 illustrates a left image and a right image generated based on pixel information grouped by an image sensor according to an embodiment of the present invention.

[0032] FIG. 13 is a block diagram illustrating components of an image sensor according to an embodiment of the present invention.

[0033] Figure 14 is a flow chart illustrating a method for controlling an image sensor according to one embodiment of the present invention.

[0034] And, FIG. 15 is a flow chart illustrating a control method of an image sensor according to another embodiment of the present invention.

[0035] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0036] Throughout the specification, identical reference numbers refer to substantially identical components. In the following description, detailed descriptions of components and functions not related to the core components of the present invention and those known in the art may be omitted.

[0037] In this specification, when the terms "includes," "has," and "consists of," are used, other parts may be added, unless "only" is used. When a component is expressed in the singular, it includes the plural unless otherwise explicitly stated.

[0038] When interpreting a component, it is interpreted as including the error range even if there is no separate explicit description.

[0039] While terms like "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, a "first" component referred to below may also be a "second" component within the technical scope of the present invention.

[0040] The term "at least one" should be understood to include all possible combinations of one or more associated items. For example, "at least one of the first, second, and third items" can mean any combination of items that can be represented by two or more of the first, second, and third items, as well as each of the first, second, and third items.

[0041] The individual features of the various embodiments of the present invention can be partially or wholly combined or combined with each other, and various technical linkages and operations are possible, and each embodiment can be implemented independently of each other or implemented together in a related relationship.

[0042] Figure 1 illustrates multi-frequency used in an image sensor according to the prior art.

[0043] In the case of an indirect ToF sensor, which is one type of image sensor, according to the prior art, as shown in FIG. 1, a single depth information (100) is obtained using a plurality of different frequencies, such as a first frequency (110) and a second frequency (120).

[0044] When designed this way, the image sensor has the advantage of being able to measure longer distances by taking advantage of the relationship between multiple frequencies.

[0045] However, since a single depth map is generated by repeating modulation and demodulation of multiple frequencies, problems such as increased power consumption, motion blur, and consequently, a decrease in the depth frame rate occur.

[0046] Figure 2 illustrates the process by which an image sensor calculates a distance using one photodiode and one tap.

[0047] Fig. 2 (a) is a graph for the irradiated light, Fig. 2 (b) is a graph for the reflected light, Fig. 2 (c) is a graph for a reference signal having a phase difference of 0 degrees with the irradiated light, Fig. 2 (d) is a graph for a reference signal having a phase difference of 180 degrees with the irradiated light, Fig. 2 (e) is a graph for a reference signal having a phase difference of 90 degrees with the irradiated light, and Fig. 2 (f) is a graph for a reference signal having a phase difference of 270 degrees with the irradiated light.

[0048] As shown in Figure 2, the reflected light may be delayed in phase by the distance that the light is reflected back after being incident on the object.

[0049] At this time, the image sensor can provide a reference signal ((c) to (f) of FIG. 2) to each pixel to derive the phase difference.

[0050] An image sensor can generate an electrical signal from reflected light based on a reference signal. In other words, the image sensor can expose a valid area corresponding to each reference signal and receive reflected light during the exposure time.

[0051] In addition, the image sensor can generate an electric signal by charging an electric charge from the reflected light when the reference signal is on (positive). Accordingly, the image sensor can generate an electric signal corresponding to the shaded portions (210, 220, 230, 240) of FIG. 2.

[0052] Q1(210) is the charge amount of the electric signal corresponding to the reference signal of the same phase as the signal of the irradiating light. Q2(220) is the charge amount of the electric signal corresponding to the reference signal whose phase is 180 degrees behind the signal of the irradiating light. Q3(230) is the charge amount of the electric signal corresponding to the reference signal whose phase is 90 degrees behind the signal of the irradiating light. Q4(240) is the charge amount of the electric signal corresponding to the reference signal whose phase is 270 degrees behind the signal of the irradiating light.

[0053] The image sensor can calculate the phase difference between the signal of the irradiated light and the signal of the reflected light using the charge amount of the electric signal generated as above, as shown in the following mathematical expression 1.

[0054]

[0055] And, the image sensor can calculate the distance to the object using the phase difference above, as in the following mathematical expression 2.

[0056]

[0057] Here, c is the speed of light and fmod is the frequency of the light.

[0058] Meanwhile, the amplitude of the reflected light can be obtained using the following mathematical formula 3.

[0059]

[0060] Figure 3 shows a graph of a transmission signal and a reception signal obtained through the process illustrated in Figure 2.

[0061] The curve (310) of the transmission signal illustrated in FIG. 3 means the amplitude change according to the time change of the irradiation light illustrated in (a) of FIG. 2, and the curve (320) of the reception signal illustrated in FIG. 3 means the amplitude change according to the time change of the reflected light illustrated in (b) of FIG. 2.

[0062] And, it is possible to estimate the distance between the image sensor and the object by using the time difference (330) between the curve (310) of the transmission signal and the curve (320) of the reception signal.

[0063] Figure 4 illustrates the process by which an image sensor calculates distance using one photodiode and two taps.

[0064] Fig. 4 (a) is a graph for the irradiated light, Fig. 4 (b) is a graph for the reflected light, Fig. 4 (c) is a graph of a reference signal having a phase difference of 0 degrees with the irradiated light, Fig. 4 (d) is a graph of a reference signal having a phase difference of 180 degrees with the irradiated light, Fig. 4 (e) is a graph of a reference signal having a phase difference of 90 degrees with the irradiated light, and Fig. 4 (f) is a graph of a reference signal having a phase difference of 270 degrees with the irradiated light.

[0065] In addition, the image sensor can generate an electric signal by charging an electric charge from the reflected light when the reference signal is on (positive). Accordingly, the image sensor can generate an electric signal corresponding to the shaded portions (410, 420, 430, 440) of FIG. 4.

[0066] Q1(410) is the charge amount of the electric signal corresponding to the reference signal of the same phase as the signal of the irradiating light. Q2(420) is the charge amount of the electric signal corresponding to the reference signal whose phase is 180 degrees behind the signal of the irradiating light. Q3(430) is the charge amount of the electric signal corresponding to the reference signal whose phase is 90 degrees behind the signal of the irradiating light. Q4(440) is the charge amount of the electric signal corresponding to the reference signal whose phase is 270 degrees behind the signal of the irradiating light.

[0067] As shown in Fig. 4, the phase shift between the incident light and the irradiated light is determined by calculating the amount of charge (410, 420, 430, 440) of the electric signal obtained in two sub-frames (0 degrees / 180 degrees and 90 degrees / 270 degrees).

[0068] For reference, in the case of the 2-tap ToF sensor illustrated in FIG. 4, it is theoretically possible to obtain the phase shift using two sub-frames, but it is also possible to use four sub-frames.

[0069] Figure 5 illustrates one actual location and multiple possible locations of an object away from the image sensor.

[0070] When the image sensor (500) measures the degree of phase shift between the irradiated light and the reflected light through the method exemplified in FIGS. 2 to 4, a plurality of expected positions (520, 510, 521, 522) can be calculated. However, when the irradiated light uses only one frequency, there are cases where the actual position (510) cannot be specified. As described above, this is because the phase shift is repeated in a 2-pi cycle.

[0071] However, in cases where multiple photodiodes, such as dual-pixel and quad-pixel, are used, examples of calculating the position of an object more accurately with only one frequency by using disparity information of a specific group will be described below with reference to FIG. 6, etc.

[0072] FIG. 6 illustrates a plurality of photodiodes included in an image sensor according to one embodiment of the present invention.

[0073] Figures 6 (a) and (b) schematically illustrate a pixel structure used, for example, in PDAF (Phase Detection Auto Focus) of dual pixels, and Figure 6 (c) schematically illustrates a pixel structure used, for example, in PDAF of quad pixels.

[0074] In particular, in the case of (a) of FIG. 6, it illustrates a case where the reflected light (610) is received equally by the first photodiode (611) and the second photodiode (612). On the other hand, in the case of (b) of FIG. 6, it illustrates a case where the reflected light (620) is received differently by the third photodiode (621) and the fourth photodiode (622).

[0075] Meanwhile, in the case of (c) of Fig. 6, reflected light can be received through four photodiodes (631, 632, 633, 634).

[0076] And, one embodiment of the present invention estimates the distance to an object by utilizing the difference between signals received through a specific group among a plurality of photodiodes using only one frequency.

[0077] FIG. 7 illustrates a process in which an image sensor according to one embodiment of the present invention processes signals received by a plurality of photodiodes using one frequency.

[0078] As shown in (a) of FIG. 7, one embodiment of the present invention uses a frame (710) consisting of one frequency, and the frame is composed of four sub-frames.

[0079] Since the conventional technology of FIG. 1 described above uses multiple frequencies, the number of required subframes (8) also increases. On the other hand, since one embodiment of the present invention is implemented with a single frequency, the number of subframes required for one depth frame is relatively small at 4. Therefore, the use of light emitters can be reduced. In particular, light emitters are also the cause of the greatest power consumption in indirect ToF, and this technical effect can solve this problem.

[0080] According to one embodiment of the present invention, disparity information is obtained from phase data. The PDAF structure described above is composed of, for example, one microlens and multiple photodiodes.

[0081] And, as shown in (b) of Fig. 7, the intensity level (e.g., amplitude, etc.) is different for each photodiode.

[0082] For example, number 720 in FIG. 7 corresponds to four photodiodes for a reference signal having a phase difference of 0 degrees from the illumination light, number 730 in FIG. 7 corresponds to four photodiodes for a reference signal having a phase difference of 90 degrees from the illumination light, number 740 in FIG. 7 corresponds to four photodiodes for a reference signal having a phase difference of 180 degrees from the illumination light, and number 750 in FIG. 7 corresponds to four photodiodes for a reference signal having a phase difference of 270 degrees from the illumination light.

[0083] Therefore, it is possible to obtain amplitude values ​​for each photodiode located on each of the top, bottom, left, and right sides.

[0084] In particular, an image sensor according to an embodiment of the present invention uses an amplitude value obtained by grouping specific diodes, and an embodiment related to this will be described later in FIG. 8.

[0085] FIG. 8 illustrates a process of grouping signals received by a plurality of photodiodes by an image sensor according to one embodiment of the present invention.

[0086] An image sensor according to one embodiment of the present invention can group signals received by a plurality of photodiodes.

[0087] For example, as illustrated in (a) of FIG. 8, the image sensor groups the photodiodes located on the left into a first group (810) and calculates the first amplitude (AL) of the signal of the first group (810). For example, the first amplitude (AL) can be the result of adding the amplitude of the photodiode (A1) located on the upper left and the amplitude of the photodiode (B2) located on the lower left, but the present invention is not necessarily limited thereto.

[0088] Furthermore, the image sensor groups the photodiodes located on the right into a second group (820) and calculates the second amplitude (AR) of the signal of the second group (820). For example, the second amplitude (AR) can be the result of adding the amplitude of the photodiode (B1) located on the upper right and the amplitude of the photodiode (A2) located on the lower right, but the present invention is not necessarily limited thereto.

[0089] Meanwhile, as illustrated in (b) of FIG. 8, the image sensor groups the photodiodes located on the upper side into a third group (830) and calculates a third amplitude (AT) of the signal of the third group (830). For example, the third amplitude (AT) can be the result of adding the amplitude of the photodiode (A1) located on the upper left and the amplitude of the photodiode (B1) located on the upper right, but the present invention is not necessarily limited thereto.

[0090] Furthermore, the image sensor groups the photodiodes located at the lower side into a fourth group (840) and calculates a fourth amplitude (AB) of the signal of the fourth group (840). For example, the fourth amplitude (AB) can be the result of adding the amplitude of the photodiode (B2) located at the lower left and the amplitude of the photodiode (A2) located at the lower right, but the present invention is not necessarily limited thereto.

[0091] In this way, one embodiment of the present invention can be used to estimate the distance of an object by comparing the result values ​​of grouped photodiodes, and this will be described later with reference to FIG. 9.

[0092] FIG. 9 illustrates the difference in the amount of light of signals received by a plurality of photodiodes of an image sensor according to one embodiment of the present invention.

[0093] When multiple photodiodes are positioned under a single micro lens, the amount of light received by each photodiode differs depending on the focus difference.

[0094] As shown in (a) of Fig. 9, when the focus is correct, the same amount of light is incident on four photodiodes (913, 914, 915, 916). Here, number 912 means reflected light, number 910 means a micro lens, and number 911 means light (ray) focused by the micro lens.

[0095] On the other hand, when the focus is not correct, for example, as shown in (b) of FIG. 9, the amount of light incident on the photodiodes (923, 925) located on the left becomes greater than the amount of light incident on the photodiodes (924, 926) located on the right.

[0096] Here, number 922 means reflected light, number 920 means micro lens, and number 921 means light (ray) focused by micro lens.

[0097] To summarize, when the focus is correct, there is no difference in the amount of light incident on the left photodiodes and the amount of light incident on the right photodiodes.

[0098] On the other hand, when the focus is not correct, there is a difference in the amount of light incident on the left photodiodes and the amount of light incident on the right photodiodes.

[0099] To be more specific, for example, if the amount of light incident on the right photodiodes is relatively greater, it may correspond to back-focus rather than in-focus. And, if the amount of light incident on the left photodiodes is relatively greater, it may correspond to front-focus rather than in-focus.

[0100] FIG. 10 illustrates a case where a disparity occurs between the amplitudes of signals grouped by an image sensor according to an embodiment of the present invention.

[0101] For reference, Fig. 9 illustrates the difference in the amount of light incident on each photodiode, while Fig. 10 illustrates the difference in the amplitude of the signal received by the group of photodiodes located on the right compared to the amplitude of the signal received by the group of photodiodes located on the left.

[0102] The curve (1000) of the transmission signal illustrated in Fig. 10 means the amplitude change according to the time change of the irradiated light, and the curve (1010) of the reception signal illustrated in Fig. 10 means the amplitude change according to the time change of the reflected light.

[0103] In particular, the present invention compares the difference in amplitude of reflected light for each of a plurality of photodiodes.

[0104] For example, as shown in Fig. 10, if there is a difference (disparity) between the amplitude of the signal (1020) applied to the photodiodes located on the left and the amplitude of the signal (1030) applied to the photodiodes located on the right, it is a case of out of focus.

[0105] On the other hand, if there is no difference (disparity) between the amplitude of the signal (1020) applied to the photodiodes located on the left and the amplitude of the signal (1030) applied to the photodiodes located on the right, then the focus is correct.

[0106] FIG. 11 illustrates a graph that can be used to estimate the position of an object using the difference between amplitudes by an image sensor according to one embodiment of the present invention.

[0107] Figure 11 (b) shows a discontinuous signal according to reflected light.

[0108] The phase of a signal is usually limited to a value between 0 and 2 pi. Wrapping occurs when the actual phase of a signal exceeds 2 pi or decreases below 0.

[0109] This causes the phase measurement to appear different from the actual phase. For example, even if the actual phase is 3 pi, the measured phase may be 1 pi. This can make it difficult for image sensors, such as indirect ToF, to accurately calculate the distance to an object.

[0110] Therefore, according to one embodiment of the present invention, as shown in (a) of FIG. 11, continuous phase information of a signal can be reconstructed and discontinuity of measured phase information can be removed.

[0111] And, (c) of Fig. 11 illustrates the change in disparity by distance. Here, disparity means, for example, the difference in amplitude of the signal received by a specific group of photodiodes.

[0112] Therefore, if there is no difference (disparity) between the amplitude of the signal applied to the photodiodes located on the left and the amplitude of the signal applied to the photodiodes located on the right, it corresponds to the focused position (1100).

[0113] Meanwhile, the depth of field becomes very deep beyond the infinity focus point (1110), so most objects beyond this distance appear sharp. Therefore, there is no need to adjust the focus from this point on.

[0114] That is, according to one embodiment of the present invention, for example, the technical effect is more effective when the frequency of the signal output by the ToF sensor is at least twice within the available range of the PDAF (up to the point corresponding to infinity focus).

[0115] For example, as shown in (b) of Fig. 11, even if the frequency phase of the signal output by the ToF sensor overlaps more than twice within the available range of the PDAF, as shown in (a) of Fig. 11, phase unwrapping is possible.

[0116] FIG. 12 illustrates a left image and a right image generated based on pixel information grouped by an image sensor according to an embodiment of the present invention.

[0117] Figure 12 (a) corresponds to the first image (Left image) generated using pixel information of photodiodes located on the left, and Figure 12 (b) corresponds to the second image (Right image) generated using pixel information of photodiodes located on the right.

[0118] And (c) of FIG. 12 is an enlarged view of a specific portion (1210) of the first image (Left image) shown in (a) of FIG. 12, and (d) of FIG. 12 is an enlarged view of a specific portion (1220) of the second image (Right image) shown in (b) of FIG. 12.

[0119] By comparing these, it can be confirmed that the position where the image is formed (the position of the same image at 1230 and 1240 is different) is different due to the difference (disparity) between the amplitude of the signal received through the photodiodes located on the left and the amplitude of the signal received through the photodiodes located on the right.

[0120] According to one embodiment of the present invention, 3D depth information can be calculated by measuring the amplitude difference (disparity) of signals received by left and right photodiodes, or by measuring the amplitude difference (disparity) of signals received by upper and lower photodiodes. In particular, it may be more suitable for a PDAF structure that uses an active light source such as a ToF sensor and has a plurality of photodiodes, but the present invention is not limited thereto.

[0121] FIG. 13 is a block diagram illustrating components of an image sensor according to an embodiment of the present invention.

[0122] As illustrated in FIG. 13, an image sensor (100) according to an embodiment of the present invention includes a lens (101), a pixel array (102), a controller (103), and a light-emitting unit (104). However, it is also possible to delete, add, or change some components according to the needs of those skilled in the art, and the scope of the present invention should be determined according to the matters described in the patent claims.

[0123] The light emitting unit (104) emits a signal consisting of one frequency. As the light source output by the light emitting unit (104), for example, a flash type or dot type can be used.

[0124] However, when a dot-type light source is used, since a certain pattern is formed, it is easier to obtain disparity information between groups of photodiodes used in the present invention, and there is a technical effect of improving the accuracy of depth information.

[0125] Then, a signal reflected from an object is received by a plurality of photodiodes (120) included in a pixel array (102) via a lens (101) (e.g., a micro lens, etc.).

[0126] The controller (103) groups signals received by a plurality of photodiodes (120) and calculates the position of the object by using the difference between the signals of the first group and the signals of the second group.

[0127] For example, if there are four photodiodes constituting one pixel, the first group includes two photodiodes located on the left, and the second group includes two photodiodes located on the right.

[0128] Furthermore, the controller (103) compares the first amplitude of the signal of the first group with the second amplitude of the signal of the second group. For example, reference may be made to the graphs illustrated in FIGS. 9 and 10.

[0129] The controller considers the object to be in focus if there is no disparity between the first and second amplitudes, for example. Conversely, if there is a disparity between the first and second amplitudes, the controller considers the object to be out of focus.

[0130] Meanwhile, one frequency used in the present invention is characterized by being repeated at least twice within the maximum distance of the infinity focus of the lens (101), for example. FIG. 14 is a flow chart illustrating a method for controlling an image sensor according to one embodiment of the present invention.

[0131] First, a process of initializing an image sensor according to an embodiment of the present invention is performed (S211).

[0132] Furthermore, the image sensor sets the modulation sequence of the PLL (Phase-Locked Loops) according to the distance range to be measured (S212).

[0133] In addition, the image sensor sets, for example, a sensor core and an IR light source (S213), and starts the operation of the image sensor while transmitting the IR light source (S214). Here, the sensor core refers to, for example, a controller that controls the operation of components within the image sensor. Furthermore, if the image sensor is an indirect ToF, it controls the phase shift change for each subframe.

[0134] Then, the image sensor emits IR light and acquires an image for the modulation sequence set in step S212 (e.g., 0 degrees, 90 degrees, 180 degrees, 270 degrees) (S215). A related embodiment was described above in FIG. 7.

[0135] Furthermore, the image sensor generates disparity and phase information of photodiodes located above, below, left, and right of a unit pixel (S216). A related embodiment was described above in FIG. 10.

[0136] Finally, the image sensor performs phase unwrapping and filtering operations using the disparity information generated in step S216 (S217) and calculates the depth of the 3D map (S218). The filtering operation in step S217 described above means, for example, invalidating data outside the distance range using the disparity information generated in step S216 and preset modulation frequency information.

[0137] Also, Fig. 15 is a flowchart illustrating a method for controlling an image sensor according to another embodiment of the present invention. However, supplementary interpretation is also possible by combining the previous embodiments.

[0138] An image sensor according to one embodiment of the present invention emits a signal consisting of one frequency (S311) and receives a signal reflected from an object via one lens and a plurality of photodiodes (S312).

[0139] Furthermore, the image sensor groups signals received by a plurality of photodiodes and calculates the position of the object using the difference between the signals of the first group and the signals of the second group (S314).

[0140] For example, if there are four photodiodes, the first group includes the two photodiodes located on the left, and the second group includes the two photodiodes located on the right. However, the present invention is not limited to this, and grouping the photodiodes in an up-down direction is also within the scope of the present invention.

[0141] Although not shown in Fig. 15, the image sensor can additionally compare the amplitude of the signal of the first group with the amplitude of the signal of the second group.

[0142] If there is no disparity between the first and second amplitudes, the image sensor considers the object to be in focus. On the other hand, if there is a disparity between the first and second amplitudes, the image sensor considers the object to be out of focus.

[0143] Accordingly, the image sensor according to one embodiment of the present invention can more accurately detect the location of an object. As described above, the image sensor herein may correspond to, for example, an indirect ToF sensor, but the present invention is not necessarily limited thereto.

[0144] Those skilled in the art will appreciate that the present invention described above can be implemented in other specific forms without changing the technical idea or essential features thereof.

[0145] Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included within the scope of the present invention.

[0146] Since the present invention can be applied to various image sensors such as a ToF sensor, its industrial applicability is recognized.

Claims

1. In the image sensor, A light emitting element that emits a signal consisting of one frequency; A lens and a plurality of photodiodes for receiving a signal reflected from an object; and Controller Including, but not limited to, The above controller, An image sensor that groups signals received by the plurality of photodiodes and calculates the position of the object by using the difference between the signals of the first group and the signals of the second group.

2. In paragraph 1, In the case where there are four of the above photodiodes, The first group above includes two photodiodes located on the left side, The second group is an image sensor including two photodiodes located on the right.

3. In paragraph 1, The above controller, An image sensor that compares a first amplitude of a signal of the first group with a second amplitude of a signal of the second group.

4. In paragraph 3, The above controller, If there is no disparity between the first amplitude and the second amplitude, the object is considered to be at the in-focus position. An image sensor that considers an object to be in an out-of-focus position when there is a disparity between the first amplitude and the second amplitude.

5. In paragraph 1, The above one frequency, An image sensor that is repeated at least twice within the maximum distance of infinity focus of the above one lens.

6. In a method for controlling an image sensor, A step of emitting a signal consisting of one frequency; A step of receiving a signal reflected from an object via one lens and a plurality of photodiodes; A step of grouping signals received by the above plurality of photodiodes; ㄱ mfl A step of calculating the position of the object by using the difference between the signal of the first group and the signal of the second group. A method for controlling an image sensor including a .

7. In paragraph 6, In the case where there are four of the above photodiodes, The first group above includes two photodiodes located on the left side, The second group is a control method for an image sensor including two photodiodes located on the right side.

8. In paragraph 6, A step of comparing the amplitude of the signal of the first group with the amplitude of the signal of the second group. A method for controlling an image sensor further comprising:

9. In paragraph 8, a step of considering the object as being at the in-focus position when there is no disparity between the first amplitude and the second amplitude; and A step of considering the object as being in an out-of-focus position when there is a difference (disparity) between the first amplitude and the second amplitude. A method for controlling an image sensor further comprising:

10. In paragraph 6, The above one frequency, A method of controlling an image sensor, wherein the control is repeated at least twice within the maximum distance of infinity focus of the above one lens.

Images