Apparatus for measuring biological information and electronic device including the same

Abstract

An apparatus for measuring biological information and an electronic device including the same are provided. An apparatus for measuring biological information includes a light source including a first light emitter that emits first light of a first wavelength and a second light emitter that emits second light of a second wavelength, an image sensor including a first pixel area including first pixels that detect the first light in response to an object and a second pixel area including second pixels that detect the second light in response to the object, a light source controller that controls the first light emitter to emit the first light of the first wavelength when the first pixels are executed with a first exposure operation and controls the second light emitter to emit the second light of the second wavelength when the second pixels are executed with a second exposure operation, and a processor that obtains a biological signal of the object from data detected by the image sensor.

Description

[0001] This application claims priority to Korean Patent Application No. 10-2020-0122111, filed with the Korean Intellectual Property Office on September 22, 2020, the entire disclosure of which is incorporated herein by reference. Technical Field

[0002] The apparatus and methods consistent with the example embodiments involve measuring biological information (such as blood pressure) by using a light source and an image sensor. Background Technology

[0003] With the advancement of medical science and the increase in average lifespan, interest in medical devices continues to grow. Not only are large medical devices being developed for hospitals or medical examination facilities, but also small, portable personal medical devices. Medical devices used to measure biometric information are generally divided into invasive and non-invasive devices. Non-invasive devices have the advantage of detecting biometric information without causing pain to the subject, but they also have the disadvantage of low accuracy in measurement results. Therefore, various studies are being conducted to overcome these shortcomings. Summary of the Invention

[0004] The exemplary embodiments at least address the above-described problems and / or disadvantages, as well as other disadvantages not described above. Furthermore, the exemplary embodiments do not need to overcome the above-described disadvantages, and may not overcome any of the above-described problems.

[0005] According to one aspect of an example embodiment, an apparatus for measuring bioinformation is provided, the apparatus comprising: a light source including a first light emitter and a second light emitter, the first light emitter being configured to emit first light of a first wavelength and the second light emitter being configured to emit second light of a second wavelength; an image sensor including a first pixel region and a second pixel region, the first pixel region including a plurality of first pixels configured to detect the first light emitted by the first light emitter and react with an object, the second pixel region including a plurality of second pixels configured to detect the second light emitted by the second light emitter and react with an object; a light source controller configured to: control the first light emitter to emit the first light of the first wavelength when the plurality of first pixels are subjected to a first exposure operation, and control the second light emitter to emit the second light of the second wavelength when the plurality of second pixels are subjected to a second exposure operation; and a processor configured to: obtain a biosignal of an object from data detected by the image sensor when the light source is operated by the light source controller, wherein the image sensor includes ten or more pixel rows, wherein the first pixel rows included in the first pixel region are different from the second pixel rows included in the second pixel region.

[0006] The first pixel region may include ten or more pixel rows.

[0007] The image sensor may include: a pixel controller configured to generate pixel control signals and timing signals for controlling the plurality of first pixels and the plurality of second pixels; and a light source controller further configured to control the light source based on the pixel control signals or timing signals.

[0008] The pixel control signal may include a reset signal, the timing signal may include a horizontal synchronization signal, and the light source controller may also be configured to control the light source based on the reset signal or the horizontal synchronization signal.

[0009] Image sensors can operate at sampling rates ranging from 15 Hz to 1000 Hz.

[0010] The light source may include a third light emitter configured to emit a third wavelength of light.

[0011] The light source may include: a sub-first light emitter configured to emit a fourth light of a first wavelength; and a sub-second light emitter configured to emit a fifth light of a second wavelength.

[0012] The device may include a force sensor disposed on the image sensor to measure the force applied to the device.

[0013] Biometric information can include blood pressure.

[0014] The image sensor may include: a third pixel region comprising a plurality of third pixels configured to detect first light emitted by a first light emitter and reacting with an object; and a fourth pixel region comprising a plurality of fourth pixels configured to detect second light emitted by a second light emitter and reacting with an object.

[0015] The plurality of first pixels in the first pixel region, the plurality of second pixels in the second pixel region, the plurality of third pixels in the third pixel region, and the plurality of fourth pixels in the fourth pixel region may be arranged in different pixel rows from each other.

[0016] According to one aspect of an example embodiment, a method for measuring bio-information is provided, the method comprising: driving a first light emitter to emit first light of a first wavelength based on a pixel control signal or a timing signal generated by an image sensor; detecting the first light of the first wavelength in response to an object by using a plurality of first pixels included in a first pixel region of the image sensor; driving a second light emitter to emit second light of a second wavelength based on a pixel control signal or a timing signal generated by the image sensor; detecting the second light of the second wavelength in response to an object by using a plurality of second pixels included in a second pixel region of the image sensor; and measuring bio-information based on data obtained from the first pixel region and the second pixel region of the image sensor.

[0017] An image sensor may include ten or more rows of pixels, and the first row of pixels included in the first pixel region may be different from the second row of pixels included in the second pixel region.

[0018] The first time period during which a first exposure operation is performed on the first pixel region does not overlap with the second time period during which a second exposure operation is performed on the second pixel region.

[0019] The method may include measuring the force exerted by an object on a device comprising a first light emitter, a second light emitter, and an image sensor.

[0020] The method may include: detecting first light emitted by a first light emitter and reacting with an object by using a plurality of third pixels included in a third pixel region of an image sensor; and detecting second light emitted by a second light emitter and reacting with an object by using a plurality of fourth pixels included in a fourth pixel region of an image sensor.

[0021] The plurality of first pixels in the first pixel region, the plurality of second pixels in the second pixel region, the plurality of third pixels in the third pixel region, and the plurality of fourth pixels in the fourth pixel region may be arranged in different pixel rows from each other.

[0022] Biometric information can include blood pressure.

[0023] The timing signal can be a horizontal synchronization signal, and the steps of driving the first light emitter and driving the second light emitter can respectively include driving the first light emitter based on the horizontal synchronization signal and driving the second light emitter based on the horizontal synchronization signal.

[0024] The pixel control signal can be a pixel reset signal, and the steps of driving the first emitter and driving the second emitter can respectively include driving the first emitter based on the pixel reset signal and driving the second emitter based on the pixel reset signal.

[0025] According to one aspect of an example embodiment, an electronic device is provided, the electronic device comprising: a means for measuring biometric information; a processor configured to control operation of the means; and a sound output means or a display means configured to output information measured by the means. The means may include: a light source including a first light emitter and a second light emitter, the first light emitter being configured to emit first light of a first wavelength and the second light emitter being configured to emit second light of a second wavelength; an image sensor including a first pixel region and a second pixel region, the first pixel region including a plurality of first pixels configured to detect the first light emitted by the first light emitter and react to an object, the second pixel region including a plurality of second pixels configured to detect the second light emitted by the second light emitter and react to an object; a light source controller configured to: control the first light emitter to emit the first light of the first wavelength when the plurality of first pixels undergo a first exposure operation, and control the second light emitter to emit the second light of the second wavelength when the plurality of second pixels undergo a second exposure operation. The processor is configured to: obtain biometric signals of an object from data detected by the image sensor when the light source is operated by the light source controller. The image sensor may include ten or more pixel rows. The first pixel row included in the first pixel region may be different from the second pixel row included in the second pixel region.

[0026] The first pixel region may include ten or more pixel rows.

[0027] The image sensor may include: a pixel controller configured to generate pixel control signals and timing signals for controlling the plurality of first pixels and the plurality of second pixels, and a light source controller further configured to control the light source based on the pixel control signals or timing signals.

[0028] Pixel control signals may include reset signals, and timing signals may include horizontal synchronization signals. The light source controller may also be configured to control the light source based on either the reset signal or the horizontal synchronization signal.

[0029] Image sensors can operate at sampling rates ranging from 15 Hz to 1000 Hz.

[0030] The light source may also include a third light emitter configured to emit third light of a third wavelength.

[0031] The light source may also include: a sub-first light emitter configured to emit a fourth light of a first wavelength; and a sub-second light emitter configured to emit a fifth light of a second wavelength.

[0032] The electronic device may further include a force sensor disposed on the image sensor to measure the force applied to the device or electronic device.

[0033] Biometric information can include blood pressure.

[0034] The image sensor may include: a third pixel region comprising a plurality of third pixels configured to detect first light emitted by a first light emitter and reacting with an object; and a fourth pixel region comprising a plurality of fourth pixels configured to detect second light emitted by a second light emitter and reacting with an object.

[0035] The plurality of first pixels in the first pixel region, the plurality of second pixels in the second pixel region, the plurality of third pixels in the third pixel region, and the plurality of fourth pixels in the fourth pixel region may be arranged in different pixel rows from each other.

[0036] According to one aspect of an example embodiment, an apparatus for measuring biometric information is provided, the apparatus comprising: a light source configured to emit first light of a first wavelength and second light of a second wavelength toward an object; an image sensor including: a first pixel row including a plurality of first pixels exposed together to the first light during a first time period, and a second pixel row including a plurality of second pixels exposed together to the second light during a second time period; a light source controller configured to: control the light source to emit the first light during the first time period and control the light source to emit the second light during the second time period; and a processor configured to: obtain biometric information based on data collected from the first pixel row and the second pixel row of the image sensor.

[0037] The image sensor can be configured to generate a reset signal and provide the reset signal to the light source controller, and the light source controller can also be configured to control the light emission time of the first light and the second light based on the reset signal.

[0038] The image sensor can be configured to generate a horizontal synchronization signal and provide the horizontal synchronization signal to the light source controller, which can also be configured to control the light emission time of the first light and the second light based on the horizontal synchronization signal. Attached Figure Description

[0039] The above and / or other aspects will become clearer by referring to the accompanying drawings, which describe specific example embodiments in which:

[0040] Figure 1 This is a schematic diagram illustrating the configuration of a device for measuring bioinformation according to an embodiment of the present disclosure;

[0041] Figure 2A It is shown that it includes Figure 1 A diagram illustrating the structure of a unit pixel within a pixel segment;

[0042] Figure 2B The structure of a data output device according to an example embodiment is shown;

[0043] Figure 2C and Figure 2D The structure of a light source controller 500 according to an example embodiment is shown;

[0044] Figure 3 This is an explanation Figure 1 A diagram illustrating the operation of the pixel unit and the light source;

[0045] Figure 4 This is an explanation provided. Figure 3 A diagram showing the pixel control signals for the first and second pixel rows;

[0046] Figure 5A This is a diagram showing the first PPG signal for light of the first wavelength. Figure 5B This is a diagram showing the second PPG signal for light of the second wavelength;

[0047] Figure 6 It is a diagram illustrating the principle of generating component waveforms included in the unit waveform of the PPG signal;

[0048] Figure 7 It is a diagram illustrating the component waveforms included in the unit waveform of the PPG signal;

[0049] Figure 8 This is a diagram illustrating the operation of a device for measuring biological information that uses a horizontal synchronization signal to drive a light source;

[0050] Figure 9 This is a diagram illustrating the operation of a device for measuring biometric information that includes two or more rows of pixels in a pixel region;

[0051] Figure 10 This is a block diagram showing a device for measuring biological information, including a third light emitter;

[0052] Figure 11 This is an explanation Figure 10 A diagram illustrating the operation of the pixel section and light source of a device used to measure biological information;

[0053] Figure 12 This is a block diagram illustrating a device for measuring biological information, comprising two or more light emitters that emit light of the same wavelength;

[0054] Figure 13A This describes the driver. Figure 12 An example diagram illustrating the method of using a light source;

[0055] Figure 13B This describes the driver. Figure 12 A diagram illustrating another example of the light source method;

[0056] Figure 14AThis is a block diagram illustrating a device for measuring biological information, including a force sensor positioned above an image sensor. Figure 14B This is a diagram showing a device for measuring biological information, including a force sensor positioned below an image sensor;

[0057] Figure 15A and Figure 15B It shows the use Figure 14A A diagram showing the PPG signal measured by the image sensor. Figure 15C It shows the use Figure 14A A diagram showing the force intensity measured by the force sensor;

[0058] Figure 16 It shows the use Figures 15A to 15C The diagram shows the oscilloscope envelope obtained from the data shown.

[0059] Figure 17 This is a block diagram illustrating a device for measuring biological information, including an image signal processor;

[0060] Figure 18 This is a block diagram showing an electronic device including an apparatus for measuring biological information;

[0061] Figure 19 This is an illustration showing an example of a wristwatch-type electronic device that includes a device for measuring biometric information;

[0062] Figure 20 This is an illustration showing an example of a mobile electronic device including a device for measuring biometric information; and

[0063] Figure 21 This is a diagram illustrating an example of an ear-worn device that includes equipment for measuring biological information. Detailed Implementation

[0064] The following describes an example embodiment in detail with reference to the accompanying drawings.

[0065] In the following description, the same reference numerals are used for the same elements, even in different drawings. The contents defined in the description (such as detailed constructions and elements) are provided to aid in a comprehensive understanding of the exemplary embodiments. However, it is clear that the exemplary embodiments can be practiced without those specific definitions. Furthermore, well-known functions or constructions are not described in detail because they would obscure the description with unnecessary detail.

[0066] In this disclosure, the expressions “A or B” or “A and / or B” may include all possible combinations of the listed items. It will be understood that although the terms “first” and “second” may be used herein to describe various elements without regard to order and / or importance, these elements should not be limited by these terms. These terms are used only to distinguish one element from another.

[0067] When a statement such as "at least one of..." follows a column of elements, the statement modifies the entire column of elements rather than individual elements within the column. For example, the statement "at least one of a, b, and c" should be understood to include: only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or any variation of the above examples.

[0068] In the following description of the embodiments, when a particular element is combined with / attached to or connected to another element, it should be understood that the particular element may be directly connected to the other element or connected to the other element via an intermediate element. Conversely, when a particular element is directly connected to or directly linked to another element, it should be understood that there are no other elements in between. Furthermore, unless explicitly stated otherwise, expressions such as “comprising” or “including” will be understood to implicitly include the stated elements but do not exclude any other elements. However, the term “comprising” should not be construed as including all components or steps stated in this disclosure.

[0069] The embodiments of this disclosure described below relate to the art of devices for measuring biological information (e.g., devices for measuring blood pressure). Detailed descriptions of technical features widely known to those skilled in the art will be omitted below.

[0070] Figure 1 This is a schematic diagram illustrating the configuration of a device for measuring bioinformation according to an exemplary embodiment of the present disclosure. Figure 2A It is shown that it includes Figure 1 A diagram illustrating the structure of a unit pixel within a pixel segment.

[0071] Biometric information can be biological / medical information obtainable from the object OBJ to be measured, and examples of biometric information may include at least one of blood pressure, blood glucose, body fat, heart rate, blood oxygen saturation level, vascular compliance, blood flow rate, and arterial stiffness. The object OBJ can be a body part from which biometric information can be readily measured, and can be, for example, the area on the inside of the wrist adjacent to the radial artery, the upper part of the wrist where veins or capillaries are located, or the peripheral part of the body where blood vessels are dense (such as fingers, toes, etc.).

[0072] Reference Figure 1The device 1000 for measuring biological information includes: a light source 100, an image sensor 300, a light source controller 500, and a processor 700. The light source 100 is used to emit light toward an object OBJ, the image sensor 300 is used to detect the light in response (such as light scattered or reflected from or transmitted through the object OBJ), the light source controller 500 is used to control the light source 100, and the processor 700 is used to measure biological information from the signal detected by the image sensor 300.

[0073] Light source 100 can emit two or more beams of light to object OBJ. The two or more beams can have different wavelengths and can reach different depths of object OBJ. (See reference...) Figure 1 The light source 100 may include a first light emitter 110 for emitting light of a first wavelength and a second light emitter 120 for emitting light of a second wavelength. The first wavelength of light may be short-wavelength light and can reach a shallower depth in the object OBJ compared to the second wavelength of light. For example, the first wavelength of light may be green light in the wavelength range of 500 nm to 565 nm and can penetrate into capillaries. The second wavelength of light may be long-wavelength light and can reach a greater depth in the object OBJ. For example, the second wavelength of light may be infrared light in the wavelength range of 750 nm to 2500 nm and can penetrate into arterioles. The first light emitter 110 and the second light emitter 120 may be light-emitting diodes (LEDs), laser diodes (LDs), phosphors, and / or combinations thereof. For example, both the first light emitter 110 and the second light emitter 120 may be LED light sources, or the first light emitter 110 may be an LED light source and the second light emitter 120 may be an LD light source.

[0074] Image sensor 300 is an electronic device for detecting light reacting with an object OBJ and generating an electrical signal, and may include a pixel unit 310, a pixel controller 320, and a data output unit 330. Image sensor 300 may be a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0075] The pixel unit 310 can accumulate charge by reacting with light reflected from the object OBJ according to the drive signal provided by the pixel controller 320, generate a pixel signal by detecting the potential change caused by the accumulated charge, and send the generated pixel signal to the data output unit 330. Figure 1 As shown, pixel section 310 may include pixels arranged in a matrix (e.g., pixels arranged in a 6×6 matrix). Pixels arranged in the same row may be referred to as a pixel row, and pixel section 310 may include first pixel row 311 to sixth pixel row 316. Although due to limited space in the figures, Figure 1Six (6) pixel rows are shown, but image sensor 300 may include ten or more pixel rows, allowing light source 100 to be controlled at a lower speed than the speed at which a single pixel row can be controlled. A unit pixel may include a photoelectric conversion device and multiple transistors for processing photocharge output from the photoelectric conversion device; an example of a unit pixel is shown in Figure 2. The photoelectric conversion device may be a photodiode, phototransistor, photogate, pinned photodiode, etc. Pixel section 310 may include 100 to 5000 pixel rows and 80 to 2000 pixel columns.

[0076] The pixel unit 310 may include superpixels having horizontal and / or vertical edges, the length of which is twice that of the other pixels, such that the superpixels are arranged in two columns and / or two rows. Furthermore, the pixel unit 310 may include filters formed in part or the entire area for allowing only light within a specific wavelength range to pass through.

[0077] Figure 2A Showing includes Figure 1 An example of a unit pixel in the pixel section having a four-transistor (4T) structure.

[0078] Reference Figure 2A Each unit pixel PX may include a photodiode PD and pixel circuitry. Pixel circuitry may include a floating diffuser FD, a transfer transistor TX, a reset transistor RX, a drive transistor DX, and a select transistor SX.

[0079] The floating diffuser FD is a triple junction of the transfer transistor TX, the reset transistor RX, and the drive transistor DX, and can be the section where the charge converted by the photodiode PD is accumulated and converted into voltage. The transfer transistor TX is turned on to transfer the charge converted by the photodiode PD to the floating diffuser FD. The reset transistor RX is turned on to convert the voltage of the floating diffuser FD to the supply voltage (VDD) and remove the charge accumulated in the floating diffuser FD. The drive transistor DX amplifies the voltage (i.e., the signal of the charge accumulated in the floating diffuser FD). The select transistor SX is turned on to output the pixel signal (i.e., the voltage amplified by the drive transistor DX) to the column line COL. One end of the select transistor SX is connected to the column line COL through node PN. Among the signals used to control the on / off state of transistors TX, RX, and SX, the signal used to control the transfer transistor TX is called the transfer signal TG, the signal used to control the reset transistor RX is called the reset signal RG, and the signal used to control the select transistor SX is called the select signal SEL.

[0080] Return to reference Figure 1The pixel controller 320 can generate pixel control signals (TG, SG, and SEL) or timing signals used in the image sensor 300 based on a master clock signal provided from the light source controller 500 or processor 700 (e.g., a central processing unit (CPU) or the like), and / or a clock signal generated by a separate clock generator. When the master clock signal is generated from the light source controller 500 or processor 700, the light source controller 500 or processor 700 may include clock circuitry or a clock generator (e.g., an electronic oscillator). The pixel controller 320 can provide the generated pixel control signals external to the image sensor 300.

[0081] The timing signal generated by the pixel controller 320 can be a pixel clock used as a reference for a series of operations performed by the image sensor 300 (including photoelectric conversion operations of the pixel unit 310, analog-to-digital conversion operations of the data output unit 330, etc.). Optionally, the timing signal can be a reference signal used to distinguish pixel signals output from the image sensor 300 on a row or frame basis, and can be, for example, a horizontal synchronization signal H-sync indicating the completion of sampling of pixel signals read from a pixel row, or a vertical synchronization signal V-sync indicating the completion of sampling of pixel signals in a frame. Multiple pixels in the same pixel row can be exposed simultaneously, and the term "pixel row" can also be referred to as "pixel line".

[0082] The pixel control signals generated by the pixel controller 320 can be the transmission signal TG, the reset signal RG, and the selection signal SEL as described above, and the pixel controller 320 can drive the pixel unit 310 through a rolling shutter method in which pixel rows are exposed and read out sequentially. For example, the rolling shutter can expose the frame line by line, and the number of exposures can be equal to the number of lines in the frame.

[0083] The pixel controller 320 can send timing signals and / or pixel control signals TG, RG and SEL to external devices (e.g., light source 100, light source controller 500 and / or display device) through the interface of the image sensor 300, so that the external devices can interact with the image sensor 300.

[0084] like Figure 2BAs shown, the data output unit 330 may include a multiplexer 330A and an analog-to-digital converter (ADC) 330B. The multiplexer 330A may receive pixel signals in analog form from the pixel unit 310 and may combine the received pixel signals into one or more analog signals. The ADC 330B may convert one or more analog signals into digital signals. Specifically, the ADC 330B may compare the amplitude of an analog pixel signal that has undergone correlated double sampling with the amplitude of a ramp signal to generate a comparison signal corresponding to the amplitude difference between the correlated double sample signal and the ramp signal, and may convert the pixel signal into a digital signal by counting the comparison signal. The data output unit 330 may include a memory 330C for storing digital signals and a buffer 330D including an amplifier for sensing and amplifying digital signals.

[0085] Return to reference Figure 1 The light source controller 500 can receive timing signals or pixel control signals TG, RG, and SEL generated by the pixel controller 320, and can synchronize the operation of the light source 100 with the image sensor 300. For example, the light source controller 500 can control the light source 100 in such a way that during the exposure operation EXP of the first pixel row 311, the light source controller 500 can turn on the first light emitter 110 and turn off the second light emitter 120, so that the pixels included in the first pixel row 311 can detect light of a first wavelength; and during the exposure operation EXP of the second pixel row 312, the light source controller 500 can turn off the first light emitter 110 and turn on the second light emitter 120, so that the pixels included in the second pixel row 312 can detect light of a second wavelength.

[0086] In one example embodiment, the light source controller 500 may include one or more switching elements 500A and 500B to turn the first light emitter 110 and the second light emitter 120 on and off. Examples of switching elements 500A and 500B include relays, bipolar junction transistors (BJTs), field-effect transistors (FETs), and metal-oxide-semiconductor field-effect transistors (MOSFETs).

[0087] like Figure 2C As shown, during the exposure operation EXP of the first pixel row 311, the light source controller 500 can turn on the first switch 500A connected to the first emitter 110 and turn off the second switch connected to the second emitter 120. During the exposure operation EXP of the second pixel row 312, the light source controller 500 can turn off the first switch 500A connected to the first emitter 110 and turn on the second switch connected to the second emitter 120.

[0088] like Figure 2DAs shown, during the exposure operation EXP of the first pixel row 311, the light source controller 500 can control the first switch 500A to connect to the first emitter 110 and disconnect it from the second emitter 120. On the other hand, during the exposure operation EXP of the second pixel row 312, the light source controller 500 can control the first switch 500A to disconnect from the first emitter 110 and connect to the second emitter 120.

[0089] The processor 700 can measure biological information using data output from the image sensor 300. The output data from the image sensor 300 received by the processor 700 includes data about the light intensity detected by the pixels of the pixel unit 310, and the processor 700 can analyze or extract biological information by analyzing the received data. The following will refer to... Figure 3 Describe the operation of processor 700.

[0090] In the following text, reference will be made to Figure 3 and Figure 4 Description by using Figure 1 The device 1000 for measuring bioinformation measures blood pressure as an example of a bioinformation method.

[0091] Figure 3 This is an explanation Figure 1 A diagram illustrating the pixel rows and the timing of the light emitter's operation. Figure 4 This is an explanation of what is used to provide application to Figure 3 A diagram showing the timing of the pixel control signals for the first and second pixel rows.

[0092] Reference Figure 3 Pixel rows 311 to 316 can sequentially execute electronic shutter operation ES, exposure operation EXP, and readout operation RO according to the pixel control signals TX, RX, and SEL of the pixel controller 320. For example, the first pixel row 311 can execute electronic shutter operation ES in the first time period t1, exposure operation EXP in the second time period t2, and readout operation RO in the third time period t3; subsequently, the second pixel row 312 can execute electronic shutter operation ES, exposure operation EXP, and readout operation RO in the fourth time period t4 to the sixth time period t6. Figure 3 In this diagram, the horizontal axis represents the sequence of operations performed by the pixel unit 310, and the number of time periods or the intervals between time periods do not indicate the length of time. For example, the second time period t2 may be longer than the first time period t1 and the third time period t3.

[0093] More specifically, refer to Figure 4To reset the first pixel row 311, the pixel controller 320 can first send a reset signal RG and a transmission signal TG simultaneously to the pixels included in the first pixel row 311 during a first time period t1, and can remove the remaining charge in the photodiode PD and floating diffuser FD of pixel PX. The operation of removing the remaining charge in the photodiode PD and floating diffuser FD can be referred to as electronic shutter operation ES. The pixel controller 320 can also send the reset signal RG not only to the first pixel row 311 but also to the light source controller 500, and the light source controller 500 can turn on the first light emitter 110. Figure 3 In the second time period t2, among the photons emitted by the first light emitter 110, photons that react with the object OBJ can be incident on the photodiode PD for photoelectric conversion. As described above, the operation of accumulating charge by incident photons after the electronic shutter operation ES can be referred to as the exposure operation EXP. When the first light emitter 110 is turned on / off in response to the rising edge of the reset signal RG, if the reset signal RG is in a high state, no charge is accumulated, making the actual exposure operation EXP period from the falling edge of the reset signal RG in the first time period t1 to the rising edge of the reset signal RG in the third time period t3 longer than the second time period t2. However, the general period of the exposure operation EXP is much longer than the first time period t1 or the third time period t3, so for ease of explanation, the second time period t2 can be referred to as the period of the exposure operation EXP.

[0094] Then, in the third time period t3, the pixel controller 320 can send a reset signal RG and a selection signal SEL to the first pixel row 311 to sample the reset voltage, and can send a transmission signal TG at the end of the reset signal RG to sample the exposure voltage generated by the charge accumulated in the photodiode PD. The reason for sampling the reset voltage first is that each pixel can have different voltages in the reset state, so that in order to accurately measure only the signal generated by the incident photons during the exposure operation EXP, both the reset voltage and the exposure voltage are sampled, and the difference between the two voltages can then be used as the signal of the exposure operation EXP. As shown in the third time period t3, the operation of reading out the signal generated by the charge accumulated in the exposed photodiode PD and sending the signal to the data output unit 330 can be called the readout operation RO. In this case, the rising edge of the reset signal RG that the first light emitter 110 can send in the third time period t3 is turned off. Subsequently, the electronic shutter operation ES of the second pixel row 312 can be performed in the fourth time period t4, and the second light emitter 120 can be turned on by the reset signal RG in the fourth time period t4. In the fifth time period t5, among the photons emitted by the second light emitter 120, the photons that react with the object OBJ are incident on the second pixel row 312 for photoelectric conversion. In the sixth time period t6, the readout operation RO of the second pixel row 312 is performed, and the second light emitter 120 can be turned off at the rising edge of the reset signal RG of the second pixel row 312.

[0095] Although Figure 4 Only the control signals for the first pixel row 311 and the second pixel row 312 are shown, but the electronic shutter operation ES, exposure operation EXP, and readout operation RO for the third to sixth pixel rows 313, 314, 315, and 316 can also be shown. Figure 3 The order shown is executed similarly. For example... Figure 3 As shown in the timing diagram, once the readout operation RO up to the sixth pixel row 316, which is the last pixel row, is completed, the pixel signal of the first frame F1 is considered to have been sent to the data output unit 330. Once the first frame F1 ends, the operation can proceed to the second frame F2, and the operation can be repeated for the frames until the desired number of pixel signals are output. Figure 3 and Figure 4 The timing diagram up to the second frame F2 is shown only, but pixel signals from the third frame onwards can be output continuously (e.g., pixel signals from 100 to 5000 frames can be output). The output pixel signals can be converted into digital data by the data output unit 330 and sent to the processor 700.

[0096] Although Figure 3An example is shown where the light source controller 500 controls the light source using a reset signal RG in the pixel control signals. However, the light source controller 500 can control the light source using another pixel control signal or reference signal. For example, the light source controller 500 can control the light source using a transmission signal TS in the pixel control signals or by using a horizontal synchronization signal H-sync in the timing signals.

[0097] In the following text, the method for measuring blood pressure will be described as using... Figure 1 An example of a device used to measure biological information and analyze biological information.

[0098] Figure 5A The diagram shows a first PPG signal, PPG1, obtained by using light within a first wavelength range obtained from pixel signals of image sensor 300. Figure 5B The second PPG signal PPG2 is shown, obtained by using light in the second wavelength range.

[0099] The photoplethysmography (PPG) signal can be a signal obtained by emitting light within a specific wavelength range toward a body part and detecting the light response of the body part, and can be a signal representing the pulsating component generated by the heartbeat. The processor 700 can obtain the PPG signal for the first wavelength of light and the PPG signal for the second wavelength of light by using data sent from the data output unit 330 of the image sensor 300.

[0100] Reference Figure 5A and Figure 5B The points forming the first PPG signal PPG1 and the second PPG signal PPG2 (e.g., G1 to G6 of PPG1, IR1 to IR6 of PPG2, etc.) can correspond to the signals sampled during the readout operation RO of each pixel row 311 to 316 as described above. For example, Figure 5A G1 can correspond to in Figure 3 The pixel signal sampled from the first pixel row 311 in the third time period t3; G2 can correspond to the pixel signal sampled from the third pixel row 313 in the ninth time period t9; G6 can correspond to the pixel signal sampled from the fifth pixel row 315 in the thirty-third time period t33. Similarly, Figure 5B IR1 can correspond to in Figure 3The pixel signal sampled from the second pixel row 312 in the sixth time period t6; IR2 may correspond to the pixel signal sampled from the fourth pixel row 314 in the twelfth time period t12; IR6 may correspond to the pixel signal sampled from the sixth pixel row 316 in the thirty-sixth time period t36. The pixel signals sampled from each pixel row include unit pixel signals output from multiple unit pixels included in the pixel row, and the processor 700 can obtain the PPG signal by processing the unit pixel signals in various ways. For example, the processor 700 can obtain the value of G1 by averaging the unit pixel signals read from six unit pixels included in the first pixel row 311 in the third time period t3, or by averaging the signals of only two to four unit pixels located in the central portion of the pixels included in the first pixel row 311. In another example, if the object OBJ does not contact the entire area of ​​the pixel portion 310, the processor 700 can measure the biosignal by using only the pixel signals of the pixels in contact with the object OBJ.

[0101] In the case where each of the G1 value, IR1 value, etc., is defined as a biosignal, the sampling rate (or frame rate) of the biosignal measured by the device 1000 for measuring bioinformation can be in the range of 10Hz to 5000Hz or 15Hz to 1000Hz. Therefore, since the sampling rate (frame rate) is higher than that of conventional image sensors, the device 1000 according to the embodiments of this disclosure has the advantage of providing a fast response time.

[0102] When the pixel row performing the exposure operation is defined as a pixel region whenever light emitters 100 and 120 are turned on, Figure 3 In the example implementation, each pixel row may indicate a pixel region. Two or more pixel rows may be included within a single pixel region, as will be discussed later. Figure 9 Describe it.

[0103] Figure 6 This is a diagram illustrating the principle of generating the unit waveform of the PPG signal. Figure 7 This is a diagram illustrating an example of dividing the unit waveform P of a PPG signal into multiple component waveforms.

[0104] Reference Figure 6 and Figure 7The unit waveform P is formed by superimposing a forward wave P1 that moves from the heart to a branch point in a blood vessel in the periphery of the body (e.g., the iliac artery) and reflected waves P2 and P3 that reflect from the branch point in the periphery. Specifically, the component waveforms P1, P2, and P3 may include the forward wave P1 generated by cardiac contraction, a first reflected wave P2 mainly reflected from the renal artery, and a second reflected wave P3 mainly reflected from the iliac artery. The amplitude of the forward wave P1 is highly correlated with cardiac activity, while the reflected waves P2 and P3 are highly correlated with vascular characteristics. Therefore, blood pressure can be measured by dividing the unit waveform P of the PPG signal into individual component waveforms P1, P2, and P3 and by analyzing the intensity and duration of each component waveform P1, P2, and P3, the interval between component waveforms P1, P2, and P3, and the ratio between the intensities of component waveforms P1, P2, and P3. For example, blood pressure can be measured by analyzing the time interval and / or the ratio between the peak point of the forward wave P1 and the peak points of the reflected waves P2 and P3. Specifically, this can be achieved by... Figure 5A The first PPG signal PPG1 and Figure 5B The second PPG signal, PPG2, is divided into component waveforms P1, P2, and P3, and these component waveforms are analyzed. Blood pressure is then measured by averaging these values; or based on... Figure 5B The second PPG signal PPG2 is used to correct the blood pressure measurement by analyzing the value obtained from the first PPG signal PPG1, thereby improving the accuracy of blood pressure measurement.

[0105] Although Figure 3 The method of measuring blood pressure using green light and infrared wavelengths is shown, but... Figure 1 The device 1000 for measuring bioinformation can use light of other wavelengths (such as blue wavelengths) to measure other types of bioinformation besides blood pressure (such as blood glucose, body fat, heart rate, blood oxygen saturation level, vascular compliance, blood flow rate, or arterial stiffness).

[0106] Figure 8 This is a diagram illustrating the method of driving a light source using a horizontal synchronization signal H-sync.

[0107] The horizontal synchronization signal H-sync is provided whenever a pixel signal is sampled for a pixel row, and can also be provided when a readout operation RO of a pixel row is performed. For example, the horizontal synchronization signal H-sync can be provided during the time period when the readout operation RO is performed (e.g., the third time period t3 when the readout operation RO of the first pixel row 311 is performed, the fifth time period t5 when the readout operation RO of the second pixel row 312 is performed, etc.). Among the timing signals or pixel control signals TG, RG, and SEL generated or used by the image sensor 300, there may be signals that are not sent outside the image sensor 300. However, in many cases, the horizontal synchronization signal H-sync is sent to the outside through the interface of the image sensor 300, allowing methods for controlling the light source based on the horizontal synchronization signal H-sync to be effectively used.

[0108] Will Figure 8 and Figure 3 Comparison, Figure 8 Example embodiments and Figure 3 The difference between the example embodiments is that: Figure 3 In an example embodiment, the reset signal RG among the pixel control signals TG, RG, and SEL is used to drive the light source 100, but in Figure 8 In an example embodiment, a horizontal synchronization signal H-sync, which is one of the timing signals, is used to drive the light source 100. Furthermore, Figure 8 Example embodiments and Figure 3 The difference between the example embodiments also lies in: Figure 3 In the example embodiment, the electronic shutter operation ES for the subsequent pixel row is performed after the readout operation RO of the previous pixel row is executed; in contrast, Figure 8 In an example embodiment, the readout operation RO of the previous pixel row and the electronic shutter operation ES of the next pixel row are executed simultaneously within the same time period. For example, in Figure 3 In the example embodiment, the readout operation R0 of the first pixel row 311 in the first frame F1 is performed in the third time period t3, and the electronic shutter operation ES of the second pixel row 312 is performed in the fourth time period t4; however, Figure 8 In an example embodiment, the electronic shutter operation ES of the second pixel row 312 can be executed simultaneously with the readout operation RO of the first pixel row 311 of the first frame F1 during the third time period t3.

[0109] The light source controller 500 can control the on / off state of the first light emitter 110 and the second light emitter 120 using a horizontal synchronization signal H-sync. For example, in response to the horizontal synchronization signal at a third time period t3, the light source controller 500 can turn on the first light emitter 110; in response to the horizontal synchronization signal at a fifth time period t5, the light source controller 500 can turn off the first light emitter 110 and turn on the second light emitter 120. Figure 8 During the exposure operation EXP period of the first pixel row 311 in the first frame F1, both the first light emitter 110 and the second light emitter 120 are turned off, so that the pixel signal of the first pixel row read out in the third time period t3 cannot be used to measure bio-information or can be used to measure ambient light not generated by light emitters 110 and 120. The measured ambient light not generated by light emitters 110 and 120 can be used to remove noise from the signal received from light emitters 110 and 120. Figure 8 Other operations in the example embodiments are similar to Figure 3 The operations described in the example embodiments are omitted here, as their redundant descriptions will be omitted.

[0110] Figure 9 This is a diagram illustrating another example of a method for driving a light source using a horizontal synchronization signal H-sync.

[0111] Will Figure 9 and Figure 8 The difference lies in the comparison: Figure 8 In the example embodiment, each pixel row corresponds to a pixel region, but in Figure 9 In the example embodiment, two adjacent pixel rows correspond to one pixel region. As briefly described above, when the pixel rows that perform the exposure operation whenever light emitters 110 and 120 are turned on are defined as pixel regions, when Figure 9 When the first light emitter 110 is turned on, the second pixel row 312 and the third pixel row 313, which perform the exposure operation EXP during the fourth time period t4 to the sixth time period t6, can correspond to a pixel region. Similarly, when the second light emitter 120 is turned on, the fifth pixel row 315 and the sixth pixel row 316, which perform the exposure operation EXP during the tenth time period t10 to the twelfth time period t12, can correspond to a pixel region.

[0112] A pixel signal output from a pixel region can be considered a signal used for analyzing biological signals. For example, by analyzing the signal in... Figure 9 The average sampled value obtained by averaging the value sampled from the second pixel row 312 in the fifth time period t5 of the first frame F1 and the sampled value sampled from the third pixel row 313 in the seventh time period t7 can be used to measure bioinformation. However, from Figure 9The pixel signals output from the first pixel row 311 and the fourth pixel row 314 of the first frame F1 (the first pixel row 311 and the fourth pixel row 314 only perform the exposure operation EXP during the period when both light emitters 110 and 120 are turned off) cannot be used to measure biometric information. For example, the pixel signal of the first pixel row 311 sampled in the third time period t3 and the pixel signal of the fourth pixel row 314 sampled in the ninth time period t9 cannot be used to measure biometric information. As described above, by placing the pixel rows for which pixel signals are not sampled between the pixel regions for which biometric information obtained using the first light emitter 110 is sampled and the pixel regions for which biometric information obtained using the second light emitter 120 is sampled, noise caused by the turn-off delay of the light emitters can be reduced. For example, it is necessary to use... Figure 9 The horizontal synchronization signal H-sync in the seventh time period t7 immediately turns off the first light emitter 110, but the turn-off timing is delayed, keeping the first light emitter 110 on until the eighth time period t8, in which the fourth pixel row 314 performs an exposure operation. Even in this case, the bio-information obtained using the second light emitter 120 is sampled starting from the tenth time period t10 instead of the eighth time period t8, thereby preventing noise caused by the turn-off delay of the first light emitter 110.

[0113] Figure 9 The light source controller 500 can control the on / off state of the first light emitter 110 and the second light emitter 120 using a horizontal synchronization signal H-sync. For example, the light source controller 500 can turn on the first light emitter 110 in response to the horizontal synchronization signal H-sync in response to the third time period t3, and can turn off the first light emitter 110 in response to the horizontal synchronization signal H-sync in response to the seventh time period t7. Furthermore, the light source controller 500 can turn on the second light emitter 120 in response to the horizontal synchronization signal H-sync in response to the ninth time period t9, and can turn off the second light emitter 120 in response to the horizontal synchronization signal H-sync in response to the thirteenth time period t13.

[0114] Although Figure 9 The example embodiment shows an example of a pixel region including two pixel rows, but the number of pixel rows included in a pixel region can be three or more, or it can be 10 to 500 or 10 to 100.

[0115] Figure 10 This is a schematic diagram illustrating an example of the structure of a device for measuring biological information, including three light emitters. Figure 11 This is an explanation Figure 10 A diagram illustrating the operation of the pixel unit and the light source.

[0116] Will Figure 10 and Figure 1 Comparison, Figure 10 Light source 101 and Figure 1 The differences between the 100 light sources are: Figure 10 The light source 101 also includes a third light emitter 130. The third light emitter 130 can emit light of a third wavelength (e.g., a red wavelength in the range of 625 nm to 740 nm) that is different from the wavelength of the light emitted from the first light emitter 110 and the second light emitter 120.

[0117] Reference Figure 11 , Figure 10 The light source controller 500 can sequentially drive the first light emitter to the third light emitter 110, 120 and 130; for example, the light source controller 500 can turn on the first light emitter 110 in the first time period t1 and the second time period t2, turn on the second light emitter 120 in the fourth time period t4 and the fifth time period t5, and turn on the third light emitter 130 in the seventh time period t7 and the eighth time period t8. Figure 10 The device 1001 for measuring biological information can measure biological information by analyzing biological signals with light having a third wavelength.

[0118] Although Figure 10 An example of a structure including three light emitters 110, 120 and 130 that emit light of three different wavelengths is shown, but four or more light emitters (e.g., four to nine light emitters) that emit light of four or more wavelengths may be included in the light source 101.

[0119] Figure 12 This is a schematic diagram showing the structure of a device for measuring biological information, comprising two or more light emitters that emit light of the same wavelength; Figure 13A and Figure 13B This is an explanation Figure 12 A diagram illustrating the operation of the pixel portion and the light source.

[0120] Will Figure 12 and Figure 1 Comparison, Figure 12 Light source 102 and Figure 1 The difference between the light source 100 and the light source 100 is that: Figure 12 The light source 102 also includes a sub-first light emitter 112 that emits light of a first wavelength, and a sub-secondary light emitter 122 that emits light of a second wavelength. The wavelength of the light emitted from the sub-first light emitter 112 may be the same as or substantially the same as the wavelength of the light emitted from the first light emitter 110. The wavelength of the light emitted from the sub-secondary light emitter 122 may be the same as or substantially the same as the wavelength of the light emitted from the second light emitter 120. Figure 12In the light source 102, light emitters 110, 120, 112 and 122 are arranged on both sides of the image sensor 300, so that the deviation of the amount of emitted light generated within the distance between the light emitters 110, 120, 112 and 122 and the pixel can be controlled.

[0121] Reference Figure 13A , Figure 12 The light source controller 500 can control the simultaneous on / off of the secondary first light emitter 112 and the first light emitter 110, and can also control the simultaneous on / off of the secondary second light emitter 122 and the second light emitter 120. The light source controller 500 can synchronize the on-time of the secondary first light emitter 112 and the first light emitter 110, and can also synchronize the on-time of the secondary second light emitter 122 and the second light emitter 120. For example, the light source controller 500 can control the first light emitter 110 and the secondary first light emitter 112 to be simultaneously on during the first time period t1 and the second time period t2, and can control the second light emitter 120 and the secondary second light emitter 122 to be simultaneously on during the fourth time period t4 and the fifth time period t5.

[0122] Reference Figure 13B , Figure 12 The light source controller 500 can control the first light emitter 110 and the secondary first light emitter 112 to be alternately turned on and off, and can also control the second light emitter 120 and the secondary second light emitter 122 to be alternately turned on and off. For example, the light source controller 500 can control only the first light emitter 110 to be turned on during the first time period t1 and the second time period t2, and can control only the secondary first light emitter 112 to be turned on during the seventh time period t7 and the eighth time period t8.

[0123] Although Figure 12 The example shown is of a light source 102 comprising two light emitters that emit light of the same wavelength, but there may be two or more light emitters that emit light of the same wavelength (e.g., three to eight).

[0124] Figure 14A and Figure 14B This is a schematic diagram showing the structure of a device for measuring biological information, which also includes a force sensor.

[0125] Will Figure 14A and Figure 14B and Figure 1 Comparison, Figure 14A and Figure 14B The device for measuring biological information may also include a force sensor disposed on top of or at the bottom of the image sensor.

[0126] Force sensor 800 can be like Figure 14A The image sensor 300 is positioned on top of the sensor shown in the diagram, or it may be positioned as shown in the diagram. Figure 14B The force sensor 800 shown is located at the bottom of the image sensor 300 and can measure the intensity of the pressure applied to the object OBJ. The force sensor 800 can be a voltage-resistance sensor, ultrasonic force sensor, load sensor, capacitive force sensor, thermoelectric sensor, strain gauge force sensor, electrochemical force sensor, optical force sensor, or magnetic force sensor.

[0127] Figure 14A and Figure 14B The devices 1003a and 1003b for measuring bioinformation can measure bioinformation by combining information provided by the force sensor 800 with information provided by the image sensor 300. In the following text, reference will be made to... Figure 15A , Figure 15B and Figure 16 Description via Figure 14A Device 1003a for measuring biological information, and method for measuring blood pressure.

[0128] Figure 15A and Figure 15B This shows that when object OBJ is pressed by gradually increasing the pressing force... Figure 14A The image shows the PPG signals PPG1 and PPG2 measured by the image sensor 300 when the force sensor 800 is used. Figure 15C This is a diagram showing the intensity of the pressing force measured by the force sensor 800.

[0129] Figure 15A The first PPG signal PPG1 obtained for light of the first wavelength is shown. Figure 15B The diagram illustrates a second PPG signal, PPG2, obtained from light of a second wavelength. Since the first wavelength, being short, cannot penetrate deep into the skin, the first PPG signal, PPG1, can include signals reflected from capillaries. In contrast, the second wavelength, being long, can penetrate into arterioles located at greater depths than capillaries, allowing the second PPG signal, PPG2, to include signals from both capillaries and arterioles. Because blood pressure-related information is more abundantly included in the arteriole signal compared to the capillary signal, the arteriole signal can be obtained by removing the capillary signal from the second PPG signal, PPG2; and the first PPG signal, PPG1, can be used to remove the capillary signal from the second PPG signal, PPG2.

[0130] Figure 15C The values ​​measured when object OBJ presses the pressure sensor by gradually increasing the pressing force are shown, and it can be seen that the pressing force gradually increases over time.

[0131] The processor 700 can normalize the first PPG signal PPG1 and the second PPG signal PPG2 to obtain second-order differential signals of the first PPG signal PPG1 and the second PPG signal PPG2, and can normalize the second-order differential signals again and subtract the second-order differential signal of the first PPG signal PPG1 from the second-order differential signal of the second PPG signal PPG2 to obtain the subtracted differential signal. Peak-to-peak points can be extracted using the waveform envelope of the subtracted differential signal, and the peak-to-peak amplitude can be plotted relative to the force intensity to obtain... Figure 16 The oscilloscope waveform envelope OW is shown. However, the example embodiment is not limited to a second-order differential signal, and the oscilloscope waveform envelope OW can also be obtained using a first PPG signal PPG1 and a second PPG signal PPG2. The force intensity (or contact pressure) MP at the maximum peak point (MA) can be calculated as the mean arterial pressure (MAP), and the force intensities SP and DP located at the right and left points of the maximum peak point and having a preset ratio with the force intensity at the maximum peak point can be calculated as the systolic blood pressure SBP and diastolic blood pressure DBP, respectively.

[0132] Figure 17 This is a schematic diagram showing the structure of a device for measuring biological information, including an image signal processor.

[0133] Image signal processor 900 can receive pixel signals from image sensor 300 to generate image information, and this image information can be used in various ways to measure biometrics. Image signal processor 900 can be implemented using a graphics processing unit (GPU) and can be integrated into processor 700 or implemented as a separate chip or component from processor 700. For example, the image information may include information about the contact area between object OBJ and image sensor 300, and can be used to calculate the pressure exerted by object OBJ based on the contact area between object OBJ and image sensor 300 and the intensity of the pressure applied by object OBJ; the calculated pressure can be used to measure blood pressure. In another example, the image information can be used to determine whether object OBJ is in contact with or near a device for measuring biometrics in a manner sufficient to measure biometrics, and the device for measuring biometrics can remain in standby mode without performing a measurement until object OBJ is within a distance where the biometrics can be measured. In another example, where object OBJ needs to be in the correct position for measuring biometrics, the device for measuring biometrics can determine the current position of object OBJ and can provide information about the direction in which the user moves object OBJ.

[0134] Figure 18 It is a block diagram of an electronic device used to measure biological information.

[0135] Electronic device 1801 includes a processor 1820, a memory 1830, an input device 1850, a sound output device 1855, a display device 1860, an audio module 1870, a sensor module 1876, an interface 1877, a haptic module 1879, a camera module 1880, a power management module 1888, a battery 1889, a communication module 1890, a user identification module 1896, and / or an antenna module 1897. In some embodiments, at least one of the components (e.g., the display device 1860, etc.) may be omitted from electronic device 1801, and one or more other components may be added to electronic device 1801. Figure 1 , Figure 10 , Figure 12 , Figure 14A , Figure 14B and Figure 17 The aforementioned devices 1000, 1001, 1002, 1003a, 1003b, and 1004 shown for measuring biological information can be implemented as a single integrated circuit to be installed in the sensor module 1876 of the electronic device 1801, or they can be distributed in different components. For example, the image sensor 300 and / or the light source 100 may be included in the sensor module 1876, and the light source controller 500, the processor 700, and / or the image signal processor 900 may be included in the processor 1820.

[0136] Processor 1820 can execute, for example, software (e.g., program 1840, etc.) to control at least one or more other components (e.g., hardware or software components, etc.) of electronic device 1801 connected to processor 1820, and can perform various data processing or calculations. According to an example embodiment, as part of data processing or calculation, processor 1820 can load commands or data received from another component (e.g., sensor module 1876, communication module 1890, etc.) into volatile memory 1832, process the commands or data stored in volatile memory 1832, and store the resulting data in non-volatile memory 1834. Processor 1820 can generate a synchronous master clock for the operation of the components and can provide the master clock to, for example, the aforementioned... Figure 1 , Figure 10 , Figure 12 and Figure 17 The pixel controller 320.

[0137] Processor 1820 may include a main processor 1821 (e.g., a central processing unit (CPU) or application processor (AP)) and an auxiliary processor 1823 (e.g., a graphics processing unit (GPU), image signal processor (ISP), sensor hub processor, or communication processor (CP)) that may operate independently of or in conjunction with the main processor 1821. The auxiliary processor 1823 may be adapted to consume less power than the main processor 1821 or be dedicated to a specific function. The auxiliary processor 1823 may, when the main processor 1821 is inactive (e.g., in a sleep state), control, in place of the main processor 1821, at least some of the functions or states associated with at least one component of the electronic device 1801 (e.g., display device 1860, sensor module 1876, communication module 1890, etc.), or, when the main processor 1821 is active (e.g., in an application execution state), control, together with the main processor 1821, at least some of the functions or states associated with at least one component of the electronic device 1801 (e.g., display device 1860, sensor module 1876, communication module 1890, etc.). The auxiliary processor 1823 (e.g., an image signal processor, a communication processor, etc.) may be implemented as part of another component (e.g., a camera module 1880, a communication module 1890, etc.) functionally associated with the auxiliary processor 1823.

[0138] In response to a user's request to measure biological information, processor 1820 may send control signals to image sensor 300, light source controller 500, processor 700, and / or image signal processor 900 of the aforementioned devices 1000, 1001, 1002, 1003a, 1003b, and 1004. The light source controller 500, processor 700, and / or image signal processor 900 may be implemented as independent processors or integrated into the main processor 1821 or auxiliary processor 1823 of electronic device 1801.

[0139] Memory 1830 may store various data required by at least one component of electronic device 1801 (e.g., processor 1820, sensor module 1876, etc.). The various data may include, for example, software (e.g., program 1840, etc.) and input or output data of commands associated therewith. Memory 1830 may include volatile memory 1832 and / or non-volatile memory 1834. Non-volatile memory 1834 may include internal memory 1836 and external memory 1838.

[0140] Program 1840 may be stored as software in memory 1830 and may include, for example, an operating system (OS) 1842, middleware 1844 and / or application 1846.

[0141] Input device 1850 can receive commands or data from an external source (e.g., a user, etc.) of electronic device 1801 that will be used by another component of electronic device 1801 (e.g., processor 1820, etc.). Input device 1850 may include, for example, a microphone, mouse, keyboard, or digital pen (e.g., stylus, etc.).

[0142] The sound output device 1855 can output sound signals to the outside of the electronic device 1801. The sound output device 1855 may include, for example, a speaker and / or a receiver. The speaker can be used for general purposes (such as playing multimedia or playing records), and the receiver can be used for incoming calls. The receiver may be implemented separately from the speaker or as part of the speaker.

[0143] Display device 1860 can visually provide information to the outside of electronic device 1801. Display device 1860 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a corresponding one of the display, holographic device, and projector. Display device 1860 may include touch circuitry adapted to detect touch and / or sensor circuitry adapted to measure the intensity of the force caused by touch (e.g., pressure sensor, etc.).

[0144] Audio module 1870 can convert sound into electrical signals, or vice versa. Audio module 1870 can acquire sound via input device 1850, or output sound via sound output device and / or headphones that are directly or wirelessly connected to external electronic devices (e.g., electronic device 1802, etc.) of electronic device 1801.

[0145] Sensor module 1876 can detect the operating state of electronic device 1801 (e.g., power, temperature, etc.) or the state of the external environment (e.g., the user's state, etc.), and can generate electrical signals or data values ​​corresponding to the detected state. Sensor module 1876 may include, for example, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor, an accelerometer, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor. Figure 1 , Figure 10 , Figure 12 , Figure 14A , Figure 14B and Figure 17 The aforementioned devices 1000, 1001, 1002, 1003a, 1003b and 1004 may be one of the biometric sensors included in the sensor module 1876.

[0146] Interface 1877 may support one or more specified protocols used by electronic device 1801 for direct or wireless connection to other electronic devices (e.g., electronic device 1802, etc.). Interface 1877 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, and / or an audio interface.

[0147] Connection terminal 1878 may include a connector via which electronic device 1801 can be physically connected to an external electronic device (e.g., electronic device 1802, etc.). Connection terminal 1878 may include, for example, an HDMI connector, a USB connector, an SD card connector, and / or an audio connector (e.g., a headphone connector, etc.).

[0148] The haptic module 1879 can convert electrical signals into mechanical stimuli (e.g., vibration, movement, etc.) or electrical stimuli that can be recognized by a user through touch or kinesthesia. The haptic module 1879 may include, for example, a motor, a piezoelectric element, and / or an electrical stimulator.

[0149] Camera module 1880 can capture still or moving images. Camera module 1880 may include a lens assembly having one or more lenses, an image sensor, an image signal processor, and / or a flash. The lens assembly included in camera module 1880 can collect light emitted from the subject to be imaged.

[0150] The power management module 1888 manages the power supplied to the electronic device 1801. The power management module 1888 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0151] Battery 1889 can supply power to at least one component of electronic device 1801. Battery 1889 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0152] Communication module 1890 supports the establishment of direct (e.g., wired) and / or wireless communication channels between electronic device 1801 and other electronic devices (e.g., electronic device 1802, electronic device 1804, server 1808, etc.) within network environment 1800, and supports communication via the established communication channels. Communication module 1890 may include one or more communication processors that can operate independently of processor 1820 (e.g., application processor, etc.) and support direct and / or wireless communication. Communication module 1890 may include wireless communication module 1892 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) and / or wired communication module 1894 (e.g., local area network (LAN) communication module, power line communication (PLC) module, etc.). Among these communication modules, a corresponding communication module may be connected via a first network 1898 (e.g., a short-range communication network such as Bluetooth). TM The communication module 1892 can communicate with other electronic devices via Wi-Fi Direct, Infrared Data Association (IrDA) or a second network 1899 (e.g., a long-distance communication network such as a cellular network, the Internet, or a computer network such as a LAN, a wide area network (WAN), etc.). These various types of communication modules can be implemented as a single component (e.g., a single chip, etc.) or as multiple components that are separate from each other (e.g., multiple chips). The wireless communication module 1892 can use user information (e.g., International Mobile Subscriber Identity (IMSI), etc.) stored in the user identification module 1896 to identify and authenticate electronic devices 1801 in a communication network (e.g., a first network 1898 or a second network 1899).

[0153] Antenna module 1897 can transmit signals and / or power to or from external devices (e.g., other electronic devices, etc.). Antenna module 1897 may include an antenna comprising a transmitting element formed of a conductive pattern formed on a substrate (e.g., a PCB, etc.). Antenna module 1897 may include one or more antennas. In the case where antenna module 1897 includes multiple antennas, communication module 1890 can select at least one antenna from the multiple antennas that is suitable for a communication scheme used in a communication network (such as a first network 1898 and / or a second network 1899). Signals or power can be transmitted or received between the communication module and other electronic devices via the selected antenna. In addition to the antenna, other components (e.g., radio frequency integrated circuits (RFICs), etc.) may also be included as part of antenna module 1897.

[0154] At least some of the above components can be interconnected and can transmit signals (e.g., commands, data, etc.) between them via peripheral communication schemes (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), mobile industrial processor interface (MIPI), etc.).

[0155] Commands or data can be sent or received between electronic device 1801 and external electronic device 1804 via server 1808 connected to the second network 1899. Other electronic devices 1802 and 1804 can be devices of the same or different types as electronic device 1801. All or some of the operations to be performed by electronic device 1801 can be performed by one or more of the other electronic devices 1802, 1804, and server 1808. For example, if electronic device 1801 needs to automatically perform a function or service, electronic device 1801 can request one or more other electronic devices to perform at least a portion of the function or service, instead of electronic device 1801 performing the function or service. The one or more other electronic devices receiving the request can perform at least a portion of the requested function or service or additional functions or services related to the request, and can send the results of the performed function or service to electronic device 1801. For this purpose, cloud computing, distributed computing, and / or client-server computing technologies can be used.

[0156] Figures 19 to 21 This is a diagram illustrating an example of an electronic device in which an instrument for measuring biological information is installed.

[0157] Reference Figure 19 , Figure 18 The electronic device 1801 can be implemented as a wristwatch-type wearable device 1801a and may include a main body and a wristband. A display is disposed on the front surface of the main body and can display various application screens (including time information, received message information, etc.). Devices 1000, 1001, 1002, 1003a, 1003b, and 1004 for measuring bio-information can be disposed on the rear surface of the main body. Devices 1000, 1001, 1002, 1003a, 1003b, and 1004 for measuring bio-information can output optical signals to a body part (such as the user's wrist) in contact with the rear surface of the main body, and can measure bio-information by detecting reflected light. By analyzing the bio-information measured by devices 1000, 1001, 1002, 1003a, 1003b, and 1004, the wristwatch-type wearable device 1801a can measure the user's bio-information (such as blood pressure, vascular age, arterial stiffness, aortic pressure waveform, pressure index, etc.).

[0158] Reference Figure 20 , Figure 18The electronic device 1801 may be implemented as a mobile device (such as a smartphone) 1801b and may include a housing and a display panel.

[0159] The housing forms the outer surface of the electronic device 1801b. The housing has a first surface, a second surface facing the first surface, and side surfaces surrounding the space between the first and second surfaces. A display panel and a cover glass are sequentially disposed on the first surface of the housing, and the display panel is exposed to the outside through the cover glass. Devices 1000, 1001, 1002, 1003a, 1003b, and 1004 for measuring bioinformation, a camera module, and an infrared sensor are disposed on the second surface of the housing. When a user sends a request for bioinformation by executing an application stored in the electronic device 1801b, the electronic device 1801b can measure the bioinformation using devices 1000, 1001, 1002, 1003a, 1003b, and 1004, and can provide the measured bioinformation to the user as an image and / or sound.

[0160] Reference Figure 21 , Figure 18 The electronic device 1801 can be implemented as an ear-worn device 1801c and may include a body and an ear strap.

[0161] Users can wear the ear straps by hanging them on their earlobes. Figure 21 The ear-worn device 1801c can be inserted into the external auditory canal. Devices 1000, 1001, 1002, 1003a, 1003b and 1004 for measuring bioinformation can be installed in the main body and can output optical signals to a body part in contact with the main body (such as the wall of the external auditory canal), and can measure bioinformation by detecting the reflected light. Figure 21 The electronic device 1801c can provide the user with biological information measured by the devices 1000, 1001, 1002, 1003a, 1003b and 1004 for measuring biological information as sound, or can transmit the measured biological information to an external device (e.g., a mobile device, a tablet PC, etc.) via a communication module installed in the main body.

[0162] While not limited thereto, the exemplary embodiments can be implemented as computer-readable code on a computer-readable recording medium. A computer-readable recording medium is any data storage device capable of storing data that can subsequently be read by a computer system. Examples of computer-readable recording media include read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices. The computer-readable recording medium can also be distributed across a networked computer system, such that the computer-readable code is stored and executed in a distributed manner. Furthermore, the exemplary embodiments can be written as computer programs that are transmitted via a computer-readable transmission medium (such as a carrier wave) and received and executed in a general-purpose or special-purpose digital computer executing the program. Moreover, it should be understood that in the exemplary embodiments, one or more units of the above-described devices and apparatus may include circuits, processors, microprocessors, etc., and are capable of executing computer programs stored on computer-readable media.

[0163] The foregoing example embodiments are merely illustrative and should not be construed as limiting this disclosure. The description of the example embodiments is intended to be illustrative and not to limit the scope of the disclosure as defined by the appended claims, and alternatives, modifications, and variations will be apparent to those skilled in the art.

Claims

1. A device for measuring bioinformatics, the device comprising: The light source includes a first light emitter and a second light emitter, the first light emitter being configured to emit first light of a first wavelength and the second light emitter being configured to emit second light of a second wavelength; An image sensor includes a first pixel region, a second pixel region, and a pixel controller. The first pixel region includes a plurality of first pixels configured to detect first light emitted by a first light emitter and react to an object. The second pixel region includes a plurality of second pixels configured to detect second light emitted by a second light emitter and react to an object. The pixel controller is configured to generate a horizontal synchronization signal for controlling the plurality of first pixels and the plurality of second pixels. The horizontal synchronization signal is provided when a readout operation of a pixel row is performed. The light source controller is configured to control the turning on and off of the light source based on a horizontal synchronization signal. The light source controller is configured to: when the plurality of first pixels undergo a first exposure operation, control the first light emitter to emit first light of a first wavelength and turn off the second light emitter; and when the plurality of second pixels undergo a second exposure operation, turn off the first light emitter and control the second light emitter to emit second light of a second wavelength; and The processor is configured to: acquire the object's biosignals from data detected by the image sensor when the light source is operated by the light source controller. The image sensor includes at least ten rows of pixels. The first pixel row included in the first pixel region is different from the second pixel row included in the second pixel region.

2. The device according to claim 1, wherein, The first pixel region comprises at least ten pixel rows.

3. The device according to claim 1, wherein, The image sensor operates at sampling rates ranging from 15 Hz to 1000 Hz or from 10 Hz to 5000 Hz.

4. The device according to claim 1, wherein, The light source also includes a third light emitter, configured to emit third light of a third wavelength.

5. The device according to claim 1, wherein, The light source also includes: The secondary first optical emitter is configured to emit a fourth light of a first wavelength; and The secondary light emitter is configured to emit a fifth light of a second wavelength.

6. The device according to any one of claims 1 to 5, further comprising: A force sensor is mounted on the image sensor to measure the force applied to the device.

7. The device according to any one of claims 1 to 5, wherein, Bioinformation includes at least one of the following: blood pressure, blood glucose, body fat, heart rate, blood oxygen saturation level, vascular compliance, blood flow rate, and arterial stiffness.

8. The device according to any one of claims 1 to 5, wherein, Image sensors also include: The third pixel region includes a plurality of third pixels configured to detect first light emitted by the first light emitter and react with the object; and The fourth pixel region includes a plurality of fourth pixels configured to detect second light emitted by the second light emitter and react with the object.

9. The device according to claim 8, wherein, The plurality of first pixels in the first pixel region, the plurality of second pixels in the second pixel region, the plurality of third pixels in the third pixel region, and the plurality of fourth pixels in the fourth pixel region are arranged in different pixel rows.

10. A method for measuring bioinformation, the method comprising: Based on the horizontal synchronization signal generated by the image sensor, the first light emitter is driven to emit first light of a first wavelength, and the second light emitter is turned off; A first wavelength of light reacting to an object is detected by using a plurality of first pixels included in a first pixel region of an image sensor; Based on the pixel horizontal synchronization signal generated by the image sensor, the second light emitter is driven to emit second light of a second wavelength, and the first light emitter is turned off; A second wavelength of light reacting to an object is detected by using a plurality of second pixels included in a second pixel region of an image sensor; as well as Biometric information is measured based on data obtained from the first and second pixel regions of an image sensor. The image sensor includes at least ten rows of pixels. The first pixel row included in the first pixel region is different from the second pixel row included in the second pixel region. The horizontal synchronization signal is provided when the readout operation of the pixel row is performed.

11. The method according to claim 10, wherein, The first time period during which a first exposure operation is performed on the first pixel region does not overlap with the second time period during which a second exposure operation is performed on the second pixel region.

12. The method of claim 10, further comprising: The force exerted by an object on a device comprising a first light emitter, a second light emitter, and an image sensor is measured.

13. The method according to any one of claims 10 to 12, further comprising: The first light emitted by the first light emitter and reacting with the object is detected by using multiple third pixels included in the third pixel region of the image sensor; as well as The second light emitted by the second light emitter and reacting with the object is detected by using multiple fourth pixels included in the fourth pixel region of the image sensor.

14. The method according to claim 13, wherein, The plurality of first pixels in the first pixel region, the plurality of second pixels in the second pixel region, the plurality of third pixels in the third pixel region, and the plurality of fourth pixels in the fourth pixel region are arranged in different pixel rows.

15. The method according to any one of claims 10 to 12, wherein, Bioinformation includes at least one of the following: blood pressure, blood glucose, body fat, heart rate, blood oxygen saturation level, vascular compliance, blood flow rate, and arterial stiffness.

16. An electronic device comprising: Devices used for measuring biological information; The processor is configured to control the operation of the device; as well as At least one of the sound output device and the display device is configured to output information measured by said device. The device includes: The light source includes a first light emitter and a second light emitter, the first light emitter being configured to emit first light of a first wavelength and the second light emitter being configured to emit second light of a second wavelength; An image sensor includes a first pixel region, a second pixel region, and a pixel controller. The first pixel region includes a plurality of first pixels configured to detect first light emitted by a first light emitter and react to an object. The second pixel region includes a plurality of second pixels configured to detect second light emitted by a second light emitter and react to an object. The pixel controller is configured to generate a horizontal synchronization signal for controlling the plurality of first pixels and the plurality of second pixels. The horizontal synchronization signal is provided when a readout operation of a pixel row is performed. The light source controller is configured to control the turning on and off of the light source based on a horizontal synchronization signal. The light source controller is configured to: when the plurality of first pixels undergo a first exposure operation, control the first light emitter to emit first light of a first wavelength and turn off the second light emitter; and when the plurality of second pixels undergo a second exposure operation, turn off the first light emitter and control the second light emitter to emit second light of a second wavelength; and The processor is configured to: acquire the object's biosignals from data detected by the image sensor when the light source is operated by the light source controller. The image sensor includes at least ten rows of pixels. The first pixel row included in the first pixel region is different from the second pixel row included in the second pixel region.

17. The electronic device according to claim 16, wherein, The first pixel region comprises at least ten pixel rows.

18. The electronic device according to claim 16, wherein, The image sensor operates at sampling rates ranging from 15 Hz to 1000 Hz or from 10 Hz to 5000 Hz.

19. The electronic device according to any one of claims 16 to 18, wherein, The light source also includes a third light emitter, configured to emit third light of a third wavelength.

20. The electronic device according to any one of claims 16 to 18, wherein, The light source also includes: The secondary first optical emitter is configured to emit a fourth light of a first wavelength; and The secondary light emitter is configured to emit a fifth light of a second wavelength.

21. The electronic device according to any one of claims 16 to 18, further comprising: A force sensor is mounted on an image sensor to measure the force applied to the device or electronic apparatus.

22. The electronic device according to any one of claims 16 to 18, wherein, Bioinformation includes at least one of the following: blood pressure, blood glucose, body fat, heart rate, blood oxygen saturation level, vascular compliance, blood flow rate, and arterial stiffness.

23. The electronic device according to any one of claims 16 to 18, wherein, Image sensors also include: The third pixel region includes a plurality of third pixels configured to detect first light emitted by the first light emitter and react with the object; and The fourth pixel region includes a plurality of fourth pixels configured to detect second light emitted by the second light emitter and react with the object.

24. The electronic device according to claim 23, wherein, The plurality of first pixels in the first pixel region, the plurality of second pixels in the second pixel region, the plurality of third pixels in the third pixel region, and the plurality of fourth pixels in the fourth pixel region are arranged in different pixel rows.

25. An apparatus for measuring bioinformation, the apparatus comprising: The light source is configured to emit a first wavelength of light and a second wavelength of light toward the object. Image sensors, including: The first pixel row includes multiple first pixels that are exposed to the first light together in the first time period, and The second pixel row includes multiple second pixels that are exposed to the second light together in the second time period; A light source controller is configured to: control the light source to emit a first light during a first time period, and control the light source to emit a second light during a second time period; and The processor is configured to: acquire biometric information based on data collected from the first and second pixel rows of the image sensor. The first pixel row is different from the second pixel row. The image sensor is configured to generate a horizontal synchronization signal for controlling the plurality of first pixels and the plurality of second pixels, the horizontal synchronization signal being provided when a readout operation of a pixel row is performed. The light source controller is further configured to control the light emission time of the first light and the second light based on the horizontal synchronization signal. When the plurality of first pixels are subjected to a first exposure operation, the light source controller turns on the first light emitter and turns off the second light emitter. When the plurality of second pixels are subjected to a second exposure operation, the light source controller turns off the first light emitter and turns on the second light emitter.

26. A computer-readable storage medium storing a program, which, when executed by a processor, causes the processor to perform the method according to any one of claims 10 to 15.

Images