Biological monitoring device, method and non-transitory computer-readable medium

Abstract

To capture information by making use of a multispectral sensor device.SOLUTION: A multispectral sensor device may include a sensor array comprising a plurality of channels and one or more processors to determine that a time-sensitive measurement is to be performed, wherein the time-sensitive measurement is to be performed using data collected by one or more channels of the plurality of channels; cause the data to be collected by a proper subset of channels of the plurality of channels, wherein the proper subset of channels includes the one or more channels; and determine the time-sensitive measurement based on the data.SELECTED DRAWING: Figure 1B

Description

Background Art

[0001] Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element. Information can be captured by utilizing a multispectral sensor device. For example, a multispectral sensor device can acquire information regarding a set of electromagnetic fields in a certain frequency band. The multispectral sensor device can include a set of sensor elements (e.g., an optical sensor, a spectral sensor, and / or an image sensor) for acquiring information. For example, by utilizing an array of sensor elements, information regarding a plurality of frequencies can be acquired. A specific sensor element of the sensor element array can be associated with a filter that restricts the frequency band directed towards the specific sensor element. The filter can be associated with a specific band corresponding to the width of the spectral range that the filter transmits towards the specific sensor element.

Summary of the Invention

[0002] In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps. In some possible embodiments, a multispectral sensor device can include: a sensor array having a plurality of channels and one or more processors, the step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more of the plurality of channels, the step of collecting data by an appropriate subset of the plurality of channels, where the appropriate subset of channels includes one or more channels, and the step of determining time-dependent measurements based on the data, and a processor for performing these steps.

[0003] In some possible embodiments, the method can include the following steps: a multispectral sensor A step of determining that measurement should be performed by a device, where the measurement is to be performed using data collected by one or more channels of a multi-spectral sensor device, and the measurement is to be related to time-dependence, and a step of collecting data by an appropriate subset for a channel among a plurality of channels of the multi-spectral sensor device, where the appropriate subset for the channel includes one or more channels, and a step of determining a measurement based on the data by the multi-spectral sensor device. In some possible embodiments, the non-transitory computer-readable medium may store one or more instructions: When executed by one or more processors of the multi-spectral sensor device, a step of determining that the one or more processors should perform a first measurement and a second measurement, where the first measurement is to be performed using first data collected by one or more first channel groups among a plurality of channels of the multi-spectral sensor device, and the second measurement is to be performed using second data collected by one or more second channel groups among the plurality of channels, and the first measurement is related to a higher degree of time-dependence than the second measurement, and a step of collecting the first data by an appropriate subset for a channel among the plurality of channels, where the appropriate subset for the channel includes one or more first channel groups, and a step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of collecting data by an appropriate subset for a channel among a plurality of channels of the multi-spectral sensor device, where the appropriate subset for the channel includes one or more channels. A step of determining a measurement based on the data by the multi-spectral sensor device. In some possible embodiments, the non-transitory computer-readable medium may store one or more

[0004] instructions: When executed by one or more processors of the multi-spectral sensor device, a step of determining that the one or more processors should perform a first measurement and a second measurement, where the first measurement is to be performed using first data collected by one or more first channel groups among a plurality of channels of the multi-spectral sensor device, and the second measurement is to be performed using second data collected by one or more second channel groups among the plurality of channels, and the first measurement is related to a higher degree of time-dependence than the second measurement, and a step of collecting the first data by an appropriate subset for a channel among the plurality of channels, where the appropriate subset for the channel includes one or more first channel groups, and a step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. instructions: When executed by one or more processors of the multi-spectral sensor device, a step of determining that the one or more processors should perform a first measurement and a second measurement, where the first measurement is to be performed using first data collected by one or more first channel groups among a plurality of channels of the multi-spectral sensor device, and the second measurement is to be performed using second data collected by one or more second channel groups among the plurality of channels, and the first measurement is related to a higher degree of time-dependence than the second measurement, and a step of collecting the first data by an appropriate subset for a channel among the plurality of channels, where the appropriate subset for the channel includes one or more first channel groups, and a step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of determining that the one or more processors should perform a first measurement and a second measurement, where the first measurement is to be performed using first data collected by one or more first channel groups among a plurality of channels of the multi-spectral sensor device, and the second measurement is to be performed using second data collected by one or more second channel groups among the plurality of channels, and the first measurement is related to a higher degree of time-dependence than the second measurement. A step of collecting the first data by an appropriate subset for a channel among the plurality of channels, where the appropriate subset for the channel includes one or more first channel groups. A step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of collecting the first data by an appropriate subset for a channel among the plurality of channels, where the appropriate subset for the channel includes one or more first channel groups. A step of collecting the first data by an appropriate subset for a channel among the plurality of channels, where the appropriate subset for the channel includes one or more first channel groups. A step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. A step of collecting the second data, where the multi-spectral sensor device is configured to activate all channels of the plurality of channels to collect the second data. a step, a step of determining a first measurement based on first data, and a second step of determining a second measurement based on the data, and causing the execution of the steps.

Brief Description of Drawings

[0005]

Figure 1A

Figure 1B

Figure 1C

Figure 1D

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0006] The following detailed description of the exemplary embodiments refers to the accompanying drawings. The same reference numerals in different drawings can identify the same or similar elements.

[0007] The frame rate used for time-dependent optical measurements in health monitoring applications (e.g., heart rate, blood pressure, etc.) is sometimes 250 - 500 samples per second (sps, samp) ​(in frames per second). In a multispectral sensor that utilizes a plurality of pixel regions of a single image sensor, the high readout speed for the full sensor may be limited by the maximum data transfer speed achievable in the imaging system. This may be due to issues with the readout architecture of the image sensor or issues with the system bus. High-resolution, high-speed sensors with high bit depth require complex circuitry, increasing the cost and size of the device. When seeking a sensor with good size, cost, bit depth, and responsiveness, it may be difficult to achieve 250 frames per second (fps) at full resolution. In spatially constrained consumer electronics applications where size and cost are design considerations, it may be difficult to achieve a high frame rate at high resolution and high bit depth. In a multispectral sensor, the high readout speed for the full sensor may be limited by the maximum data transfer speed achievable in the imaging system. This may be due to issues with the readout architecture of the image sensor or issues with the system bus. High-resolution, high-speed sensors with high bit depth require complex circuitry, increasing the cost and size of the device. When seeking a sensor with good size, cost, bit depth, and responsiveness, it may be difficult to achieve 250 frames per second (fps) at full resolution. In spatially constrained consumer electronics applications where size and cost are design considerations, it may be difficult to achieve a high frame rate at high resolution and high bit depth. The embodiments described herein can maintain high resolution, high bit depth, and high frame rate without exceeding the data transfer speed of the imaging system by processing a view of a specific region of interest (ROI) from the sensor image. For example, specific time-sensitive spectral channel measurements can be obtained at a high frame rate (e.g., at full ROI resolution and / or bit depth). For example, time-dependent measurements can be used to process time-dependent parameters such as specific health state parameters.

[0008] Full-spectrum sensors can operate at a lower rate for measurements that require a full set of spectral channels and / or at an intermediate frame rate for any data parameter mixing mode that does not exceed the data bus speed of the spectrometer. ROI processing can be used to achieve high-resolution, high-bit-depth, and high-frame-rate imaging while keeping the data transfer rate within the capabilities of the imaging system. For example, time-sensitive spectral channel measurements can be obtained at a high frame rate (e.g., at full ROI resolution and / or bit depth). For example, time-dependent measurements can be used to process time-dependent parameters such as specific health state parameters. Full-spectrum sensors can operate at a lower rate for measurements that require a full set of spectral channels and / or at an intermediate frame rate for any data parameter mixing mode that does not exceed the data bus speed of the spectrometer. ROI processing can be used to achieve high-resolution, high-bit-depth, and high-frame-rate imaging while keeping the data transfer rate within the capabilities of the imaging system. Full-spectrum sensors can operate at a lower rate for measurements that require a full set of spectral channels and / or at an intermediate frame rate for any data parameter mixing mode that does not exceed the data bus speed of the spectrometer. ROI processing can be used to achieve high-resolution, high-bit-depth, and high-frame-rate imaging while keeping the data transfer rate within the capabilities of the imaging system. For example, time-sensitive spectral channel measurements can be obtained at a high frame rate (e.g., at full ROI resolution and / or bit depth). The principle can be achieved by the sensor of the camera, (relating to a charge-coupled device (CCD) system device by) partial scanning, or (relating to a complementary metal-oxide-semiconductor (CMOS) system device by) windowing. For measurements that are time-dependent or made frequently, by performing ROI processing using partial scanning or windowing, it may be possible to avoid exceeding the data bus speed of the multispectral sensor, thereby maintaining the time dimension of the time-dependent measurement and thereby improving the accuracy of the measurement. Further, some of the embodiments described herein can be made on the chip of the multispectral sensor (e.g., before passing the data to the control device), thereby reducing latency and also improving the accuracy of the time-of-measurement of the measurement.

[0009] Figures 1A - 1D are schematic diagrams of exemplary embodiment 100 described herein. As shown in Figure 1A, exemplary embodiment 100 can be made by a multispectral sensor device such as a multispectral sensor device using a CMOS device or a CCD (e.g., the multispectral sensor device 220 of Figure 2). In some embodiments, certain operations regarding embodiment 100 can be made by another device in the environment 200 of Figure 2, such as control device 210.

[0010] As shown in Figure 1, the multispectral sensor device may include a sensor array 105. As shown, the sensor array 105 may include channels 110-1 through 110-64. For example, the sensor array may include a plurality of sensor elements configured to obtain information regarding a plurality of corresponding frequency bands. Additionally or alternatively, the sensor array may be a single including a plurality of sensor elements configured to obtain information associated with a frequency band obtained. The sensor element may correspond to channel 110.

[0011] As shown in FIG. 1B and indicated by reference numeral 120, the multispectral sensor device may perform measurements based on region 115. Regarding how to perform the measurements, FIG. 1 will be described in more detail in relation to FIGS. 1C to 1D. As indicated by reference numeral 125, the multi spectral sensor device may perform measurement 1 by using channels 10, 11, 18, 19 of the sensor array 105. As further shown, the multispectral sensor device may perform measurement 2 by using all channels of the multispectral sensor device. Here, in measurement 1, four channels are used, and these may be collectively referred to as a pixel region or ROI. As shown in the figure, in measurement 2, all channels of the sensor array 10 5 are used. In some embodiments, in measurement 2, a number of channels less than all channels of the sensor array 105 may be used.

[0012]

[0012] In relation to exemplary embodiment 100, assume that measurement 1 is a time-dependent measurement and measurement 2 is not a time-dependent measurement. Here, a time-dependent measurement may mean a measurement associated with a threshold frame rate or threshold data rate, a measurement that requires accurate timing measurement for accuracy, and / or other similar measurements. A time non-time-sensitive measurement may mean a measurement not associated with a threshold frame rate or data rate, a measurement that does not require accurate timing measurement, and / or other similar measurements. can mean a measurement. In some embodiments, the time-dependent measurement may be a multi-spectrum associated with a specific frame rate and / or resolution that would exceed the data rate of the bus of the multispectral sensor device. Exceeding the data rate of the bus can cause data to be queued, thereby corrupting the time dimension of the data. As a result, the accuracy of some time-dependent measurements can be reduced. / or resolution that can be associated. Exceeding the data rate of the bus can cause data to be queued, thereby corrupting the time dimension of the data. As a result, the accuracy of some time-dependent measurements can be reduced. This can cause data to be queued, thereby corrupting the time dimension of the data. As a result, the accuracy of some time-dependent measurements can be reduced. This can reduce the accuracy of some time-dependent measurements.

[0013] As shown in FIG. 1C and indicated by reference numeral 130, the multispectral sensor device can determine that measurement 1 is a time-dependent measurement. Further shown, the multispectral sensor device can collect data only for the channels (channels 10, 11, 18, 19 indicated by diagonal hatching) associated with measurement 1. In some embodiments, the multispectral sensor device can collect data using the ROI windowing described below. In some embodiments, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105, as detailed below. This can cause data to be queued, thereby corrupting the time dimension of the data. As a result, the accuracy of some time-dependent measurements can be reduced. As shown in FIG. 1C and indicated by reference numeral 130, the multispectral sensor device can determine that measurement 1 is a time-dependent measurement. Further shown, the multispectral sensor device can collect data only for the channels (channels 10, 11, 18, 19 indicated by diagonal hatching) associated with measurement 1. In some embodiments, the multispectral sensor device can collect data using the ROI windowing described below. In some embodiments, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105, as detailed below. As shown in FIG. 1C and indicated by reference numeral 130, the multispectral sensor device can determine that measurement 1 is a time-dependent measurement. Further shown, the multispectral sensor device can collect data only for the channels (channels 10, 11, 18, 19 indicated by diagonal hatching) associated with measurement 1. In some embodiments, the multispectral sensor device can collect data using the ROI windowing described below. In some embodiments, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105, as detailed below. In some embodiments, the multispectral sensor device can collect data using the ROI windowing described below. In some embodiments, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105, as detailed below. In some embodiments, the multispectral sensor device can collect data using the ROI windowing described below. In some embodiments, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105, as detailed below. In some embodiments, such as when the multispectral sensor device includes a CCD-based device, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105. For example, to perform a partial scan, the following can be done: Make several (e.g., consecutive) vertical shifts to the read register to avoid or eliminate unwanted (e.g., channels associated with channels other than 10, 11, 18, 19). This can cause data to be queued, thereby corrupting the time dimension of the data. As a result, the accuracy of some time-dependent measurements can be reduced.

[0014] In some embodiments, such as when the multispectral sensor device includes a CCD-based device, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105. For example, to perform a partial scan, the following can be done: Make several (e.g., consecutive) vertical shifts to the read register to avoid or eliminate unwanted (e.g., channels associated with channels other than 10, 11, 18, 19). In some embodiments, such as when the multispectral sensor device includes a CCD-based device, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105. For example, to perform a partial scan, the following can be done: Make several (e.g., consecutive) vertical shifts to the read register to avoid or eliminate unwanted (e.g., channels associated with channels other than 10, 11, 18, 19). In some embodiments, such as when the multispectral sensor device includes a CCD-based device, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105. For example, to perform a partial scan, the following can be done: Make several (e.g., consecutive) vertical shifts to the read register to avoid or eliminate unwanted (e.g., channels associated with channels other than 10, 11, 18, 19). In some embodiments, such as when the multispectral sensor device includes a CCD-based device, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105. For example, to perform a partial scan, the following can be done: Make several (e.g., consecutive) vertical shifts to the read register to avoid or eliminate unwanted (e.g., channels associated with channels other than 10, 11, 18, 19). In some embodiments, such as when the multispectral sensor device includes a CCD-based device, the multispectral sensor device can collect data by performing a partial scan on the sensor array 105. For example, to perform a partial scan, the following can be done: Make several (e.g., consecutive) vertical shifts to the read register to avoid or eliminate unwanted (e.g., channels associated with channels other than 10, 11, 18, 19). discarding unwanted or unnecessary charges. Outputting each pixel in a row Since it is not necessary, vertical transfer can be performed faster than reading an entire row, which increases the frame rate. This is because fewer rows are output by the sensor for each frame. Once the ROI scan for Measurement 1 is achieved, the sensor array 105 can be operated normally, and pixels can be output from appropriate rows (detailed in FIG. 1D described later).

[0015] In some embodiments, such as when the multispectral sensor device includes a CMOS-based device, the multispectral sensor device can collect data using ROI windowing. For example, for some CMOS sensor architectures, both vertical and horizontal windowing can be done. This allows for an increase in the corresponding frame rate in some embodiments. This is because the following is done: That is, the pixel signals are sent in parallel through a bank of column amplifiers, then towards a column A / D converter, and finally the digitized data is sent out of the chip towards a high-speed multiplexer. The integration of parallel A / D converters in the CMOS chip can enable a high pixel clock with a high frame rate.

[0016] In some embodiments, windowing for the CMOS sensor can be extended from a single window to multiple windows by appropriately addressing the correct rows and columns of interest. This can be done by using multiple windows or ROIs. With multiple windows or ROIs, the multispectral sensor device can, without exceeding the data rate of the bus, useful​​​ The utilization rate of the sensor output band for information can be improved. In this way, a multi-spectrum sensor device can improve the measurement frequency and accuracy with respect to time-dependent measurements.

[0017] As shown in FIG. 1D and indicated by reference numeral 135, in some embodiments the multi-spectrum sensor device can determine that Measurement 2 is not time-dependent. Then, the multi-spectrum sensor device can collect data using the full sensor array 105 and determine Measurement 2 based on the collected data. For example, the multi-spectrum sensor device can collect data for each channel of the sensor array 105 . In some embodiments, the multi-spectrum sensor device can collect data for channels 10, 11, 18, and / or the remaining channels other than 19, and can save the resources that would have been used to collect unnecessary data from channels 10, 11, 18, and / or 19. In some embodiments, the multi-spectrum sensor device can collect data at full resolution for all channels of the sensor array 105, enabling more accurate determination with respect to time-independent measurements.

[0018] Examples of operations described in connection with FIGS. 1A - 1D include biometric monitoring for the purpose of measuring heart rate, blood pressure, SpO2, blood glucose levels, hydration, and / or other health state parameters. Consider the case of a 64-channel multi-spectrum sensor as a biometric monitoring device, where the sensor is obtained by integrating a monolithic multi-spectrum filter on a pixelated sensor (such as a normal silicon CMOS image sensor). Heart rate, blood pressure, and SpO2, etc.For the cardiopulmonary function parameters, time-dependent measurements (e.g., made at more than 250 sps) of the time-dependent spectral signals at a few wavelengths may be required. By utilizing the multispectral ROI windowing technique, data in specific channels corresponding to a few wavelengths can be sampled at a speed that meets the timing requirements for sampling and the necessary measurements can be calculated. Once the time-dependent measurements are completed, the multispectral sensor can perform a full sensor readout (e.g., for all 64 channels), which captures the remaining channels regarding the data within the full readout. Using this information, other spectral health state parameters such as blood glucose level and hydration can be determined, which are time-independent but may require high-resolution spectral content. In this way, the multispectral ROI windowing technique achieves high resolution, high bit depth, and high frame rate, and without this technique, a complex architecture that would impose a considerable cost and size burden on the device would be required. Other techniques such as wafer stacking for integrating specialized readout circuits into each pixel or creating dedicated circuits for performing ultra-high-speed data collection may not be suitable for achieving low cost and high manufacturability. Also, without the ROI technique for discarding additional useless data, a large amount of data needs to be processed before calculating the useful signal and returning it to the user, and the time-dependence of the measurement will be impaired.

[0019]

[0020] ​​​​​​​​​​​​​​​​​Exemplary embodiment 100 is described in connection with a two-dimensional sensor array. However, the embodiments described in this specification may also be applicable to a three-dimensional sensor array. For example, the ROI for such a sensor array can be one-dimensional (e.g., a single channel or a line for several channels), two-dimensional (e.g., a layer for several channels), or three-dimensional (e.g., two or more layers for one or more channels).

[0021] As described above, FIGS. 1A - 1D are presented only as examples. There may be other examples and they may differ from those described in connection with FIGS. 1A - 1D.

[0022] FIG. 2 is a diagram of an exemplary environment 200 in which the systems and / or methods described in this specification may be implemented. As shown in FIG. 2, the environment 200 may include a control device 210, a multispectral sensor device 220, and a network 230. The various devices of the environment 200 may be interconnected via a wired connection, a wireless

[0023] connection, or a combination of wired and wireless connections. The control device 210 includes one or more devices capable of storing, processing, and / or routing information related to multispectral sensing. For example, the control device 210 may include a server, a computer, a wearable device, a cloud computing device, and / or other similar ones. In some embodiments, the control device 210 can be associated with a specific In some embodiments, the control device 210 can receive information from and / or transmit information to other devices within the environment 10, such as the multispectral sensor device 220.

[0024] The multispectral sensor device 220 includes a device capable of measuring light directed thereto. For example, the multispectral sensor device 220 can include an image sensor, a multispectral sensor, and / or the like, which can perform sensor measurements on the light directed to the multispectral sensor device 220. The multispectral sensor device 220 can utilize one or more sensor technologies such as CMOS technology, CCD technology, and / or the like. The multispectral sensor device 220 can include a plurality of sensor elements (e.g., an array of sensor elements, hereinafter referred to as a sensor array), each configured to acquire information. The sensor element can correspond to some channel, such as channel 115 in FIG. 1A.

[0025] The network 230 includes one or more wired and / or wireless networks. For example, the network 230 can include the following: cellular networks (e.g., LTE network, CDMA network, 3G network, 4G network, 5G network, another type of next-generation network, etc.), public land mobile networks (PLMN, public land mobile network), LAN, WAN, metropolitan area networks (MAN, me ​​​​​​​​​​​​​​​metropolitan area network), telephone network (public switched telephone network (PTSN, Public Switched Tel ephone Network)), private network, ad-hoc network, intranet, i nternet, optical fiber network, cloud computing network, and the like, and / or combinations of these or other types of networks.

[0026] The number and arrangement of the devices and networks shown in FIG. 2 are presented as an example. In actual operation there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged in a different manner in relation to FIG. 2. Furthermore, two or more devices shown in FIG. 2 can be implemented within a single device, and a single device shown in FIG. 2 can be implemented in a distributed manner using multiple devices. Additionally or alternatively, a set of devices in environment 200 (e.g., one or more devices) may provide one or more of the functions described as being performed by another set of devices in environment 200.

[0027] FIG. 3 is a diagram of exemplary components of device 300. Device 300 may correspond to control device 210 and / or multispectral sensor device 220. In some embodiments control device 210 and / or multispectral sensor device 220 may include one or more devices 300 and / or one or more components of device 300. As shown in FIG. 3 , device 300 includes bus 310, processor 320, memory 330, storage component 3 40. An input component 350, an output component 360, and a communication interface may include 370.

[0028] The bus 310 includes components that enable communication between the components of the device 300. The processor 320 is implemented in hardware, firmware, or a combination of hardware and software. The processor 320 can take the following forms: a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a micro processor, a microcontroller, an FPGA, an ASIC, or other types of processing components. In some embodiments, the processor 320 includes one or more processors that can be programmed to perform certain functions. The memory 330 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) for storing information and / or instructions for the processor 320. The storage component 340 stores information and / or software related to the operation and use of the device 300. For example, the storage component 340 may include the following: a hard

[0029] disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or an SSD ), a CD, a DVD, a floppy (registered trademark) disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium, as well as the corresponding drive. )

[0030] The input component 350 enables the device 300 to receive information, for example, via user input. including components that can enable this (e.g., touch screen display, ke yboard, keypad, mouse, button, switch, and / or microphone). Additionally or alternatively, the input component 350 may include sensors for sensing information (e.g., GPS component, accelerometer, gyroscope, and / or actuator). The output component 360 includes components for providing output information from the device 300 (e.g., display, speaker, and / or one or more LEDs) .

[0031] The communication interface 370 includes transceiver components (e.g., transceiver and / or separate receiver and transmitter) that enable the device 300 to communicate with other devices , and communication can be made via, for example, a wired connection, a wireless connection, or a combination of wired and wireless connections . The communication interface 370 can enable the device 300 to receive information from another device and / or provide information to another device. For example , the communication interface 370 may include: Ethernet (registered trademark) interface , optical interface, coaxial interface, infrared interface, radio frequency ( RF) interface, Universal Serial Bus (USB) interface, Wi-Fi (registered trademark) interface, cellular network interface, etc .

[0032] The device 300 can perform one or more of the processes described herein. The device 300 can perform these processes by a processor 320 that executes software instructions stored in a non-transitory computer-readable medium such as the memory 330 and / or the storage component 340 . ​ As used herein, a computer-readable medium is defined as a non-transitory memory device. Memory devices include: a memory area within a single physical storage device or memory areas dispersed among multiple physical storage devices.

[0033] Software instructions can be read into the memory 330 and / or the storage component 340 from another computer-readable medium or from another device via the communication interface 370. During execution, the software instructions stored in the memory 330 and / or the storage component 340 can cause the processor 320 to perform the processes described herein. Additionally or alternatively, instead of or in combination with the software instructions, the processes described herein can be performed using hardwired circuitry. Accordingly, the embodiments described herein are not limited to any specific combination of hardware circuitry and software.

[0034] The number and arrangement of components shown in FIG. 3 are presented by way of example. In actual operation, device 300 may include additional components, fewer components, different components, or components arranged in a different manner compared to those shown in FIG. 3. Additionally or alternatively, a set of components of device 300 (e.g., one or more components) may perform one or more functions performed by another set of components of device 300.

[0035] FIG. 4 illustrates an exemplary process regarding ROI windowing for multispectral measurement. A flowchart for 400. In some embodiments, one or more processing blocks of FIG. 4 may be , performed by the multispectral sensor device 220. In some embodiments, FIG. 4 one or more processing blocks of may be different from the device or the multispectral sensor device 220 performed by another device group or a device group including the multispectral sensor device 220 obtained (e.g., the control device 210).

[0036] As shown in FIG. 4, the process 400 is a step of determining to perform time-dependent measurements, where the time-dependent measurements are to be performed using data collected by one or more channels of a plurality of channels, and may include steps (S410). For example, the multispectral sensor device 220 can determine (e.g., using a processor 320 and / or other similar ones) to perform time-dependent measurements. The time-dependent measurements can be made using data collected by one or more channels of the sensor array (e.g., within the ROI associated with the time-dependent measurements). In some embodiments, the determination of the measurement can be made automatically by the multispectral sensor device 220 (e.g., based on feedback). In some embodiments, the measurement can be subject to some settings (e.g., requiring 250 sps for a particular measurement).

[0037] As further shown in FIG. 4, the process 400 is a step of collecting data by an appropriate subset of channels of a plurality of channels, where the appropriate subset of channels includes one or more channels, and may include steps (S420). For example For example, the multispectral sensor device 220 may (e.g., using the processor 320) A suitable subset of the channels (e.g., more than all channels) The data can be collected by using fewer channels. The relevant subset may include one or more channels in the ROI. The multispectral sensor device 220 may employ an ROI windowing technique or partial scanning. Data collection can be accomplished using techniques detailed elsewhere herein. There are.

[0038] As further shown in FIG. 4, a process 400 determines a time-dependent measurement based on the data. For example, the multispectral sensor device 220 may include a step (S430). For example, the processor 320 may be used to determine a time-dependent measurement based on the data. In this way, the data bus transfer of the multispectral sensor device 220 for time-dependent measurements is In some embodiments, the multi-spectral sensor device 220 may include a The data can be provided to a device (eg, controller 210) where the data can be determined.

[0039] The process 400 may be implemented, for example, in any single embodiment or as described below and / or elsewhere herein. Additional embodiments may be included, such as combinations of the described embodiments.

[0040] In some embodiments, the appropriate subset of channels may include one or more channels. In some embodiments, the appropriate subset of channels includes only Each row contains one or more rows, and each row contains one or more channels. In some embodiments, the multispectral sensor device 220 can discard data other than the data collected by one or more channels. In some embodiments, the multispectral sensor device 220 can cause data collection for an appropriate subset of channels based on the time dependence of the time-dependent measurement. In some embodiments, the time-dependent measurement is the first measurement, and the data is the first data. The multispectral sensor device 220 can determine that a second measurement should be made, where the second measurement can be associated with a reduced time dependence compared to the first measurement, and the second data can be collected by all of the plurality of channels, and at least a portion of the second data can be used to make the second measurement. In some embodiments, the multispectral sensor device 220 can perform multiple iterations for the first measurement and the second measurement, and the first measurement can be performed more frequently than the second measurement. In some embodiments, the first measurement is determined with less latency than the second measurement. In some embodiments, the first measurement is performed more frequently than the second measurement. In some embodiments, the sensor array includes at least one of a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) device. In some embodiments, the time-dependent measurement is made about a value related to a living body or a medical value. In some embodiments, the multispectral sensor device 220 includes a CMOS device. The multispectral sensor device 220 can perform vertical and horizontal windowing to allow data to be collected by only one or more channels. In some embodiments

[0041]

[0042] ​​​​​​​ The multispectral sensor device 220 includes a CCD. The multispectral sensor device 22 0 can perform one or more consecutive vertical shifts to a read register and discard data other than the data to be collected. In some embodiments, certain data from one or more rows are not associated with one or more channels and are dropped when determining measurements.

[0043] FIG. 4 presents exemplary blocks for process 400. In some embodiments, process 4 00 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks compared to those illustrated in FIG. 4. Additionally or alternatively, two or more blocks of process 400 can be performed in parallel.

[0044] FIG. 5 is a flowchart for another exemplary process 500 regarding ROI windowing for multispectral measurements. In some embodiments, one or more blocks of FIG. 5 can be performed by the multispectral sensor device 220. In some embodiments, one or more blocks of FIG. 5 can be performed by another device or a group of devices including or separate from the multispectral sensor device 220 (e.g., control device 210).

[0045] As shown in FIG. 5, process 500 may include a step of determining to perform a first measurement and a second measurement, where the first measurement is associated with a stronger time-dependence than the second measurement (S510). For example, the multispectral sensor device 220 ​​​​​​​​​​​, determining (e.g., using processor 320) that a first measurement and a second measurement should be taken. The first measurement is associated with a stronger time dependence than the second measurement. In some embodiments, the first measurement may be performed at a higher data rate, frame rate, or the like than the second measurement. The frame rate and / or resolution may be associated with the image.

[0046] As shown in FIG. 5, a process 500 may include: collecting first data by a suitable subset of the channels, The appropriate subset may include one or more first channels (S520 For example, the multispectral sensor device 220 may (e.g., using the processor 320 (c) extracting first data from a suitable subset of channels of the plurality of channels; The appropriate subset of channels can be collected by one or more first The ROIs may include ROIs corresponding to the channels.

[0047] As further shown in FIG. 5, the process 500 includes the step of collecting second data. , the multispectral sensor device 220 activates all of the multiple channels. and causing the second data to be collected (S530). For example, the multispectral sensor device 220 may (e.g., using the processor 320 ) the second data can be collected. All channels of the channel can be activated to collect second data. .

[0048] As further shown in FIG. 5, the process 500 includes determining a first measurement based on the first data. may include a step of (S540). For example, the multi-spectral sensor device 220 may determine a first measurement based on the first data (using, for example, the processor 320). In some embodiments, the multi-spectral sensor device 220 can provide the first data to another device (e.g., the control device 210) for determination of the first measurement.

[0049] As further shown in FIG. 5, the process 500 may include a step of determining a second measurement based on the second data (S550). For example, the multi-spectral sensor device 220 may determine a second measurement based on the second data (using, for example, the processor 320). In some embodiments, the multi-spectral sensor device 220 can provide the second data to another device (e.g., the control device 210) for determination of the second measurement.

[0050] The process 500 may include additional embodiments, such as combinations of any single embodiment or the embodiments described hereinafter and / or elsewhere in this specification.

[0051] In some embodiments, the multi-spectral sensor device 220 can perform multiple repetitions for the first measurement and the second measurement, and the first measurement can be performed more frequently than the second measurement. In some embodiments, the first measurement is determined with less latency than the second measurement. In some embodiments, the multi-spectral sensor device includes a CCD or a CMOS device.

[0052] FIG. 5 presents exemplary blocks for the process 500. However, in some embodiments, the process 500 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently as compared to those illustrated in FIG. 5. Additionally or alternatively, two or more blocks of process 500 can be performed in parallel. In this way, the multi-spectral ROI windowing technique achieves high resolution, high bit depth, and high frame rate, and without this technique, a complex architecture that would impose a significant cost and size burden on the device would be required. For example, other techniques such as wafer stacking for integrating specialized readout circuits into each pixel or creating dedicated circuits for performing ultra-high-speed data collection may not be suitable for achieving low cost and high manufacturing ease. Also, without an ROI technique for discarding useless additional data, a large amount of data needs to be processed before calculating useful signals and returning them to the user, which would compromise the time-dependency of the measurement.

[0053]

[0054] The above disclosure provides examples and explanations, but does not cover everything and is not intended to strictly limit the embodiments to the disclosed forms. Changes and modifications can be made in light of the above disclosure or obtained from the practice of the embodiments.

[0055] As used herein, the term component is to be understood broadly as referring to hardware, firmware, and / or a combination of hardware and software.

[0056] Some of the embodiments described in this specification relate to thresholds. As described in this specification, satisfying a threshold value refers to a value greater than the threshold value, a value exceeding the threshold value, a value higher than the threshold value, a value equal to or greater than the threshold value, a value less than the threshold value, a value smaller than the threshold value, a value lower than the threshold value, a value equal to or less than the threshold value, a value equal to the threshold value, and the like.

[0057] It is clear that the systems and / or methods described in this specification can be implemented in various forms as hardware, firmware, or a combination of hardware and software. It is understood that the actual specialized control hardware or software code used to implement these systems and / or methods does not limit the embodiments. Therefore, the operations and behaviors of the systems and / or methods have been described without reference to specific software code, and it should be noted that based on the description in this specification, software and hardware can be designed to implement the systems and / or methods.

[0058] Although special combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible embodiments. In fact, many of these features can be combined in ways not specifically recited in the claims and / or not disclosed in the specification. Each of the dependent claims described below can be directly dependent on only one

[0059] claim, but the disclosure of possible embodiments includes combinations of each dependent claim with all other claims within the scope of the claims.

[0059]

[0059] Elements, acts, or instructions used in this specification shall not be construed as important or essential unless explicitly stated. Also, the articles "a" and "an" as used in this specification shall be construed to include one or more items and may be used interchangeably with "one or more". Furthermore, the term "set" as used in this specification shall be construed to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with "one or more". If only one item is intended, the term "one" or a similar word shall be used. Also, terms such as "has", "have", "having" as used in this specification shall be construed as open - ended terms. Furthermore, the phrase "based on" shall be construed to mean "at least partially based on" unless otherwise specified.

Description of Reference Numerals

[0060] 100 Exemplary Embodiment 200 Exemplary Environment 210 Control Device 220 Multispectral Sensor Device 230 Network 300 Device 310 Bus 320 Processor 330 Memory 340 Storage Component 350 Input Component 360 Output Component 370 Communication Interface

Claims

1. A multispectral sensor device, comprising: A sensor array having a plurality of channels; One or more processors that perform steps of: Determining to perform a time-dependent measurement, wherein the time-dependent measurement is to be performed using data collected by one or more of the plurality of channels; Collecting the data by an appropriate subset of the plurality of channels, wherein the appropriate subset of the channels includes the one or more channels; And determining the time-dependent measurement based on the data. A multispectral sensor device comprising the one or more processors.

2. The multispectral sensor device according to claim 1, wherein the appropriate subset of the channels includes only the one or more channels.

3. The multispectral sensor device according to claim 1, wherein the appropriate subset of the channels includes one or more rows of sensors, and the one or more rows include the one or more channels.

4. The multispectral sensor device according to claim 3, wherein the one or more processors perform a step of discarding data other than the data collected by the one or more channels.

5. 、 The multispectral sensor device according to claim 1, wherein when collecting the data by the appropriate subset of the channels, the one or more processors further perform a step of collecting the data by the appropriate subset of the channels based on the time-dependency of the time-dependent measurement.

6. The multispectral sensor device according to claim 1, wherein the time-dependent measurement is a first measurement, the data is first data, and the one or more processors perform steps of: Determining to perform a second measurement, wherein the second measurement is associated with a time-dependency having less strictness than the first measurement; Collecting second data by all of the plurality of channels; And performing the second measurement using at least a portion of the second data. ​ ​ ​ ​ ​ , Multispectral sensor device.

7. In the multispectral sensor device according to claim 6, the one or more processors 、 perform a step of repeating the first measurement and the second measurement a plurality of times, wherein the first measurement is performed more frequently than the second measurement, and further perform the step, Multispectral sensor device.

8. In the multispectral sensor device according to claim 1, the sensor array includes at least one of a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) device, Multispectral sensor device.

9. In the multispectral sensor device according to claim 1, the time-dependent measurement is the first measurement, and the data is the first data, the one or more processors perform a step of determining to perform a second measurement, wherein the second measurement is associated with a stricter time dependence than the first measurement, step, and collect second data by a channel set for the plurality of channels, the channel set including a number of channels less than an appropriate subset for the channels, step, and perform a step of performing the second measurement using at least a part of the second data, Multispectral sensor device.

10. A step of determining to perform a measurement by a multispectral sensor device, wherein the measurement is to be performed using data collected by one or more channels among the plurality of channels of the multispectral sensor device, the measurement is associated with time dependence, step, and a step of collecting the data by an appropriate subset for the channels among the plurality of channels by the multispectral sensor device, wherein the appropriate subset for the channels includes the one or more channels, step, and a step of determining the measurement based on the data by the multispectral sensor device, Method.

11. In the method according to claim 10, the multispectral sensor device includes a complementary metal oxide semiconductor (CMOS) device, the step of collecting the data further includes performing vertical and horizontal windowing so that the data is collected only by the one or more channels, Method. ​ ​ ​ ​

12. The method according to claim 10, wherein the multispectral sensor device includes a charge coupled device ( CCD), the step of collecting the data includes the step of performing one or more consecutive vertical shifts to a readout register and the step of discarding data other than the data to be collected, a method.

13. The method according to claim 12, wherein specific data from one or more rows is not associated with the one or more channels, and the specific data is dropped when determining the measurement, a method.

14. The method according to claim 10, wherein the measurement is a first measurement and the data is first data, and the method includes a step of determining to perform a second measurement, wherein the second measurement is associated with a time dependence with reduced strictness compared to the first measurement , a step of collecting second data by all channels of the plurality of channels , and a step of performing the second measurement using at least a part of the second data, further including a method.

15. The method according to claim 14, wherein the first measurement is determined with less latency than the second measurement, a method.

16. The method according to claim 14, wherein the first measurement and the second measurement are repeated a plurality of times , and the first measurement is performed more frequently than the second measurement, a method.

17. A non-transitory computer-readable medium storing instructions, the instructions comprising one or more instructions to be executed by one or more processors of a multispectral sensor device, causing the one or more processors to determine to perform a first measurement and a second measurement, wherein the first measurement is to be performed using first data collected by one or more first channel groups of a plurality of channels of the multispectral sensor device , the second measurement is to be performed using second data collected by one or more second channel groups of the plurality of channels , 、 the first measurement is associated with a higher time dependence than the second measurement , , a step, a step of collecting the first data by an appropriate subset of channels among the plurality of channels , An appropriate subset of the channels includes the one or more first channel groups , steps, and a step of collecting the second data, wherein the multispectral sensor device is configured to activate all channels of the plurality of channels to collect the second data, the step and a step of determining the first measurement based on the first data and a step of determining the second measurement based on the second data, a non-transitory computer-readable medium.

18. In the non-transitory computer-readable medium according to claim 17, when the one or more instructions are executed by the one or more processors a step of performing multiple repetitions of the first measurement and the second measurement, wherein the first measurement is performed more frequently than the second measurement, the step is further performed by the one or more processors, a non-transitory computer-readable medium.

19. In the non-transitory computer-readable medium according to claim 17, the first measurement is determined with less latency than the second measurement, a non-transitory computer-readable medium.

20. In the non-transitory computer-readable medium according to claim 17, the multispectral sensor device includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) device , a non-transitory computer-readable medium. ​ ​ ​ ​

Images