Encoding and decoding the spectral distribution of light

Abstract

The specific spectral distribution (310) of light (210) is encoded into a spectral distribution identifier (250 / 270). Light (210) is individually filtered by a set of filters (120-n) that meet conditions such as uniqueness and / or efficiency. The filtered light (220-n) is measured (130) to obtain a provisional intensity vector ({B}, 230-n). To compensate for variations, the calculation function (140, 150, 160, 170) absorbs light variations using an intensity reference value (L_DATA) and absorbs filter variations using predetermined calibration data ({CAL}). Thereafter, the calculation function converts the provisional intensity vector into a spectral distribution identifier (250 / 270).

Description

[Technical Field]

[0001] Generally, this disclosure relates to data representing light, and more specifically, to methods, computers, and computer program products for encoding the spectral distribution of light into data. [Background technology]

[0002] Physical objects can provide light by emitting, reflecting, transmitting, refracting, or otherwise. By examining the light emanating from an object, humans can conclude at least some physical phenomena or properties (i.e., object properties hereafter) concerning the object. Very simply put, such conclusions are possible because object properties influence optical properties. Optical properties include, for example, spectral distribution and light intensity.

[0003] Many objects have a spatial configuration, so their spectral distribution and light intensity can change, for example, along the surface of the object. Since object properties can change over time, optical properties such as spectral distribution and light intensity may also change.

[0004] Humans can investigate the properties of an object by visually inspecting it. The human eye perceives changes in the spectral distribution as changes in color and the intensity of light as brightness. For example, a human observer can inspect a tree (i.e., an object) and recognize that apples are ready for harvest at a particular location on the tree (i.e., a property). This is possible because the apples appear in a color (such as red) that contrasts with the color of the leaves (such as green) and is characteristic of mature apples.

[0005] The limitations on visual inspection are well known, and it is sufficient to list only a few of them. ● Different examiners may perceive light with the same spectral distribution (and the same light intensity) differently. ● The human eye usually examines light and cannot draw conclusions in an absolute sense. ● There are limitations in space, and it may be difficult for a human inspector to visually inspect relatively large objects. Ultimately, the human inspector must move along the object, move close to the object, or move away from the object. ● There is a time limit for a human inspector to be present to view an object, and a human inspector cannot see the past. ● There are spectral limitations, and humans can only investigate object characteristics from (visible) light, but some of the more interesting object characteristics can only be investigated through non-visible light.

[0006] Such limitations can be at least partially overcome when optical information is processed in the form of digital data. Developers design applications for technical systems, and human inspectors become users of such applications. Developers perform at least some of the following activities. (i) The developer identifies an object and at least one object characteristic of interest. (ii) The developer specifies the technical parameters of the system. For example, the parameters can describe the spectral distribution of light expected to reach from the object, the spread of the object, the maximum allowable dimensions of the technical system or its components, and the time interval during which light from the object is available. The developer selects the technical parameters so that the (interesting) characteristics can be investigated. (iii) The developer then designs a technical system (with components of that system) to implement the application. Thereby, the developer selects the technical system components according to the technical parameters.

[0007] In the first example, the user is located far away from the tree. A visual inspection in front of the tree is impossible. Regarding the object and object characteristics, the user is interested in whether the tree provides red apples that are ready for harvest. Regarding the technical parameters, the system processes data on the visible wavelength range and data on the spectral resolution to distinguish at least red from green. The system also needs to process data related to the tree's leaves (at least where the tree can have apples). A suitable technical system comprises a handheld digital camera, a data network, and a screen.

[0008] In the second example, the user needs to (a) determine the type of apple and (b) detect potential deviations from normal, such as spots on the surface. The technical parameters include (i) the spectral range covering the light from the apple's surface and (ii) the spectral resolution enabling the distinction of minute changes on the surface. Regarding the technical system, the developer holds a handheld digital camera but pays special attention to the range and resolution and, in some cases, applies computer-aided image processing.

[0009] In the third example, the user needs to identify the chemicals within the apple. Such substances can be residues from fertilizers, pesticides, etc. The parameter range (λ_min, λ_max) and resolution (Δλ) are specified to identify the spectral lines typical for such substances. Even if the spectrometer is limited to the laboratory and the apple has to be transported to the laboratory, using spectroscopy (with the spectrometer) is a suitable choice. Once the spectrum is measured, the user can compare it to a reference, usually assisted by a computer.

[0010] While the examples may seem very simple, selecting technical components according to technical parameters typically imposes various technical constraints on the developer. It should also be noted that system components are arranged in a one-way chain from a data provider to a data consumer via a data link. In the first example, the chain goes from a digital camera to a screen via a network (and / or storage). In the second example, an image processing computer is added as a data consumer. In the third example, a spectrometer acts as the data provider, and a computer acts as the data consumer.

[0011] Since users are typically located at the end of the chain, i.e., as data consumers, object property detection is only possible if the data they receive contains sufficient detail.

[0012] Therefore, technical constraints usually arise in combination, and the following may only be a non-exclusive overview of some of these constraints. ●Data accuracy parameters (including spectral resolution) are related to the hardware requirements for system components and the amount of data being processed. ● Light in different wavelength ranges (such as visible light and infrared light) is typically captured by devices designed to be different.

[0013] International Publication No. 2018 / 056976(A1) describes a method for acquiring spectral data in the context of recognizing known spectra characteristic of a particular substance in so-called sparse conditions. This document presents multiple filters that are moved on a wheel so that filtered light reaches a single sensor. The filters and their combinations are described as being specific to sparse use cases. The filters typically reject a portion of the spectral range, which may hinder the identification of a particular substance. The document further describes data processing using transmission matrices to enable the recognition of known spectra. [Overview of the project]

[0014] A method for encoding a specific spectral distribution of light into a spectral distribution identifier is disclosed. Light has wavelength-specific intensity in a specific spectral distribution within a wavelength range from the minimum wavelength to the maximum wavelength, which has wavelength resolution.

[0015] The step of receiving light is followed by filtering the received light. The light is filtered individually using each filter in a filter set of at least N=3 filters. The filter set satisfies the following conditions: (1) Each filter in the filter set has a filter-specific transmission function that describes the wavelength-specific transmission of the filter. (2) In the combined transmission function for the filter set, the transmission vector, which is the transmission concatenation, is unique for any wavelength and unique for each spectral distribution of light within the wavelength range. (3) Each filter in the filter set is a single-state filter.

[0016] This method continues by individually measuring the intensity of the filtered light across the wavelength range for each of the N filters to obtain provisional intensity values. These values ​​are the N elements of the provisional intensity vector.

[0017] This method is computer-implemented in at least several steps. A calculation function provides the computer with a spectral distribution identifier that encodes a specific spectral distribution of received light having N elements. This calculation function absorbs light variations using intensity reference values ​​and filter variations using predetermined calibration data.

[0018] Optionally, providing a spectral distribution identifier is performed by an operational function implemented according to either the first or second alternative form.

[0019] The alternative forms simply distinguish absorption by their temporal sequence. However, since it is inappropriate to cover both alternative forms in a single claim, the alternative forms are presented in two independent claims.

[0020] In the first alternative form, the first calculation function maps provisional intensity values ​​to calibrated intensity values, which are elements of a calibrated intensity vector, according to predetermined calibration data. The second calculation function calculates normalized calibrated intensity values, which are elements of a normalized calibrated intensity vector, by individually dividing the calibrated intensity values ​​of the calibrated intensity vector by the intensity reference value. Thus, the spectral distribution identifier is provided as a normalized calibrated intensity vector.

[0021] In the second alternative form, the first calculation function calculates normalized intensity values, which are elements of a normalized intensity vector, by individually dividing the provisional intensity values ​​by the intensity reference values. The second calculation function maps the normalized intensity values ​​to calibrated normalized intensity values, which are elements of a calibrated normalized intensity vector, according to predetermined calibration data. Thus, the spectral distribution identifier is provided as a calibrated normalized intensity vector.

[0022] Optionally, filtering the received light is performed by a filter set, in which the conditions are checked for suitability by simulation based on coding parameters that define each spectral distribution.

[0023] Optionally, in the first alternative form of the steps provided, the second calculation function obtains the intensity reference value by processing a provisional intensity vector or by processing a calibrated intensity vector.

[0024] Optionally, in the first alternative form, the second calculation function obtains the intensity reference value by a calculation selected from the following: (i) calculating the intensity reference value as the sum of provisional intensity values ​​or the sum of calibrated intensity values; (ii) calculating the intensity reference value as the average of provisional intensity values ​​or the average of calibrated intensity values; and (iii) calculating the intensity reference value as the median of provisional intensity values ​​or the median of calibrated intensity values.

[0025] Optionally, in a second alternative form of the steps provided, the third calculation function obtains the intensity reference value by processing the provisional intensity vector.

[0026] Optionally, the third calculation function obtains the intensity standard value by a calculation selected from the following: (i) calculating the intensity standard value as the sum of the provisional intensity values, (ii) calculating the intensity standard value as the average of the provisional intensity values, and (iii) calculating the intensity standard value as the median of the provisional intensity values.

[0027] Optionally, the intensity reference value is obtained by measuring the intensity of the received light using a sensor. In the first method, the received light proceeds directly to the sensor. In the second method, the received light proceeds indirectly to the sensor via a neutral photoconductor, and the sensor measures the intensity of the received light after correcting for the loss introduced by the neutral photoconductor.

[0028] Optionally, filtering the received light with a set of filters is performed by a combination of multiple filters and sensors positioned in a plane perpendicular to the direction of the received light. Measuring the intensity of the received light is then performed by directing a portion of the received light directly to at least one additional sensor, thereby bypassing the filtering of the filter set.

[0029] In both alternative configurations, the intensity reference value is obtained by measuring the intensity of the received light, and filtering the received light with a set of filters and measuring the intensity of the filtered light is performed in an arrangement where the filters of the filter set are combined with corresponding sensors. These filter-sensor combinations are positioned in a plane perpendicular to the direction of the received light. Measuring the intensity of the received light and obtaining the intensity reference value can be done by a combination of conductors and sensors positioned in the same plane. The filter-sensor combinations and the conductor-sensor combinations form a mosaic pattern in the plane.

[0030] In both alternative forms, the mapping step is performed using calibration data derived from filter-specific deviations measured before light is received. Filter-specific deviations are measured by comparing the characteristics of the filters in the filter set with the characteristics of the reference filters in the reference filter set.

[0031] Optionally, prior to the mapping step, calibration data may be determined by training a neural network using simulated intensity vectors as ground truth and simulated filtering with a simulated reference filter, using simulation data for a predefined programmable spectral distribution. The weights within the network then serve as calibration data for the mapping step, either transferring provisional intensity values ​​to a neural network that outputs calibrated intensity values ​​in a first alternative configuration, or transferring normalized intensity values ​​to a preliminary neural network that outputs calibrated normalized intensity values ​​in a second alternative configuration.

[0032] Optionally, the filtering step is performed using a filter such that wavelength-specific transmission exceeds a transmission threshold of at least 50% for all wavelengths within the wavelength range.

[0033] Optionally, the filtering step is performed using filters, and therefore, each wavelength-specific filter-specific combination of transmissions is unique for each wavelength.

[0034] Optionally, the measurement step is implemented by at least one photosensor element that converts the quantity of photons into a readable electrical signal. The photosensor element is selected from the group consisting of photodetectors, photodiodes, phototransistors, photomultiplier tubes, photomultiplier tube cameras, and photometers.

[0035] Optionally, performing filtering and measurement steps is implemented by providing a set of filters attached to multiple photosensor elements in an array having surfaces perpendicular to the received light.

[0036] Furthermore, a computer implementation method for decoding spectral distribution identifiers (obtained by performing an encoding method) is disclosed. The decoding method includes the steps of: receiving a spectral distribution identifier; and mapping the spectral distribution identifier to a specific spectral distribution. The mapping is performed by one of the following: (i) accessing a library representing a predefined relationship between a specific spectral distribution identifier and a specific spectral distribution; (ii) simulating the encoding of a plurality of known input distributions to a plurality of corresponding spectral distribution identifiers, thereby identifying a particular spectral distribution as a specific simulated input distribution for which the corresponding simulated distribution identifier fits the received spectral distribution identifier; and (iii) processing the received spectral distribution identifier with a pre-trained neural network that classifies the received spectral distribution identifier into one of the predefined distributions.

[0037] When a computer program product is loaded into a computer's memory and executed by at least one processor of the computer, the computer is caused to perform the steps of a computer-implemented decoding method.

[0038] The device is adapted to encode a specific spectral distribution of light. The device comprises a filter layer attached to the sensor layer. The filter layer and the sensor layer are planar layers. The filter layer is adapted to receive light on its surface and transmit the received light to the sensor layer. The filter layer and the sensor layer are divided into separate pixel positions corresponding to filter positions and corresponding to sensors. The sensor is adapted to quantify the intensity of the transmitted light by integrating the intensity of the transmitted light during a measurement time interval and provide a sensor-specific provisional intensity value representing the transferred light. For a continuous combination of M pixel positions, hereinafter referred to as an area, when N < M, the following applies. N filter positions hold a filter set of N≧3 filters above the corresponding sensor such that the corresponding sensor provides a sensor-specific intensity value as a provisional intensity value. M - N filter positions are either empty filters or holding filters having a wavelength-nonspecific transmission function, and thus the corresponding sensor provides a sensor-specific intensity value as a luminance value. Each of the N filters in the filter set has a filter-specific transmission function that describes the wavelength-specific transmission of the filter. In the combined transmission function for the filter set, the concatenation of transmissions, which is a transmission vector, is unique for any wavelength and for each spectral distribution of light within the wavelength range.

[0039] The device further comprises a calculation module that processes the provisional intensity value into a normalized intensity value that is an element of a normalized intensity vector corresponding to a spectral distribution identifier encoding a specific spectral distribution of the received light.

[0040] Optionally, this calculation module is implemented such that the mapper is adapted to map provisional intensity values ​​to calibrated intensity values ​​which are elements of a calibrated intensity vector using predetermined device-specific calibration data, and the divider is adapted to individually divide the calibrated intensity values ​​of the calibrated intensity vector by the luminance values.

[0041] Optionally, the calculation module is implemented such that the divider is adapted to obtain an intermediate normalized intensity vector by individually dividing the provisional intensity values ​​by the luminance values, and the mapper is adapted to map the intermediate normalized intensity values ​​to the calibrated normalized intensity values ​​of the normalized intensity vector using predetermined device-specific calibration data.

[0042] Optionally, the filter layer and sensor layer are positioned in a plane perpendicular to the direction of the received light.

[0043] Optionally, wavelength-specific transmission exceeds at least 50% of the transmission threshold for all wavelengths within the wavelength range.

[0044] A computer program product, when loaded into the computer's memory and executed by at least one of the computer's processors, causes the computer to perform the step of computing a function by providing a spectral distribution identifier.

[0045] A computer implementation method for decoding spectral distribution identifiers is disclosed. The identifiers are obtained in advance by performing a coding method. The coding method includes the steps of: receiving a spectral distribution identifier; and mapping the spectral distribution identifier to a specific spectral distribution. The mapping is performed by one of the following: (i) accessing a library representing a predefined relationship between a specific spectral distribution identifier and a specific spectral distribution; (ii) simulating the coding of multiple known input distributions to multiple corresponding spectral distribution identifiers, thereby identifying a specific spectral distribution as a specific simulated input distribution for which the corresponding simulated distribution identifier fits the received spectral distribution identifier; or (iii) processing the received spectral distribution identifier by a pre-trained neural network that classifies the received spectral distribution identifier into one predefined distribution.

[0046] A computer program product, when loaded into the computer's memory and executed by at least one of the computer's processors, causes the computer to perform steps of a computer-implemented decryption method. [Brief explanation of the drawing]

[0047] [Figure 1] This provides a simplified overview of a data processing system that includes a data provider, data link, and data consumer. [Figure 2] This document outlines the implementation of a filtering technique for encoding specific spectral distributions of light into spectral distribution identifiers, including its deployment to data providers. [Figure 3] This document presents filtering techniques with details for both hardware and software. [Figure 4] Some of the hardware components are shown, but it also has an additional sensor for measuring brightness. [Figure 5]A flowchart is shown for a method of encoding the spectral distribution of light into a spectral distribution identifier. [Figure 6] An extended flowchart illustrating the method in the context of design time and calibration time is shown. [Figure 7] Details for filtering and measurement steps are provided, taking into account different implementation options for filters and sensors. [Figure 8] The array of filters and sensors is shown in different diagrams, side views, and top views. [Figure 9] This document outlines the filter configuration for applying multiplexing to the filtering and measurement steps. [Figure 10] This shows the arrangement of multiple filters and sensors within an array for parallel filtering and measurement. [Figure 11] This shows the transparency function for the filters within the filter set. [Figure 12] The transparency functions of the filters within the set are also shown, but we demonstrate that for some subranges, fewer filters can contribute to uniqueness. [Figure 13] The transparency functions for the four filters are shown, but there are some changes here as well. [Figure 14] The transmission function for a single filter is shown, and the transmission is constant over several wavelength subranges. [Figure 15] A matrix with a spectral diagram is shown. [Figure 16] Three typical filter transmission functions with deviation patterns are shown. [Figure 17] The arithmetic function shown in Figure 3 is presented along with its implementation options. [Figure 18] This section describes the setup for a numerical experiment to estimate the degree of uniqueness through simulation. [Figure 19] This demonstrates the decoder function that implements the decoding method. [Figure 20] This shows a pre-trained neural network in the decoder function. [Figure 21] This section describes several aspects of how neural networks can be trained. [Figure 22] A time diagram is shown for a method for identifying changes in a specific spectral distribution by sequentially executing first and second step sequences belonging to an encoding method. [Figure 23] This paper presents filtering techniques with details for hardware and software that have efficiency requirements for filtering. [Figure 24] The transmission functions for four filters with efficiency requirements are shown. [Figure 25] This refers to a typical computer. [Modes for carrying out the invention]

[0048] Introduction Figure 1 shows a simplified overview of a data processing system 100 having a data provider 101, a data link 102, and a data consumer 103.

[0049] Object 110 is a physical object, and this object is ●Light-emitting light sources (for example, artificial light sources such as incandescent lamps, fluorescent lamps, and light-emitting diodes, such as those used in traffic signals, automobile headlights or taillights, and street lighting), ● Light-reflecting surfaces (e.g., the Earth's surface, or the surfaces of human organs), scenes (e.g., landscapes with forests or fields, geological formations such as mountains), ●By being an optical refractive element (for example, a lens, prism, or optical grating), it provides light 210.

[0050] From a more functional standpoint, object 110 is the object of interest.

[0051] The examples in this introduction are not intended to be limiting or exclusionary, and other configurations are possible (for example, a light source illuminating an object).

[0052] The data provider 101 receives light 210 from the physical object 110. In other words, light 210 is incident light to the data provider 101. The physical object 110 does not need to belong to the system 100.

[0053] System components 101, 102, and 103 form a data processing chain for processing LIGHT_DATA. As used herein, the term "LIGHT_DATA" is a general term for data that describes the characteristics of light 210 arriving from a physical object 110.

[0054] The data provider 101 has one or more sensors and has a calculation function for providing LIGHT_DATA. Since LIGHT_DATA transmits the properties of light, LIGHT_DATA also transmits the properties of an object. The optical properties can be, for example, spectral density (i.e., spectral distribution), overall intensity, and polarization.

[0055] More generally, some physical properties of an object may manifest through interaction with photons, and therefore may manifest as optical properties.

[0056] Data provider 101 provides LIGHT_DATA to data consumer 103. Figure 1 further shows a data link 102 that connects data provider 101 to data consumer 103 in a communicative manner. Data link 102 transmits LIGHT_DATA from data provider 101 to data consumer 103. Data link 102 can transmit data over physical distance. Data link 102 can be implemented by computer networks, by temporary or permanent data storage, or a combination thereof. Data link 102 may modify LIGHT_DATA. However, for the sake of simplicity, we will assume that data link 102 transmits LIGHT_DATA without modification.

[0057] In many use case applications, a human user (of the data consumer 103) wants to measure or investigate specific physical properties of a physical object 110 (i.e., the object of interest). As described above, the spectral distribution of light reaching the data provider 101 from the physical object 110 can serve as an indicator of these object properties. The data consumer 103 is a technical device that can derive information about the physical object 110 from LIGHT_DATA. In a simplified example, the data consumer 103 is a device equipped with a display.

[0058] Examples for objects and use cases Examples of use cases for objects of relatively large size include satellite imagery of forest and agricultural areas. Satellites capture photographs or images of such areas. By observing changes in the spectral distribution (spatial and temporal) across the area, it is possible to identify dry areas, determine crop maturity, or even predict fires.

[0059] An example of a use case for relatively small objects is microscopic imaging of patient biopsy samples. The camera captures stitched-together images with high spatial resolution. The health status of the examined tissue can be inferred by observing the spectral characteristics of reflected or transmitted light, or by detecting spectrally separable biochemical markers.

[0060] Use case scenarios can be distinguished not only by different objects, but also in other ways. ●The first factor is the technical propagation delay between capturing light (for example, by taking an image with a camera in the data provider 101) and processing the data to identify the spectral distribution (for example, by the data consumer 103). One aspect of such delay may be a requirement for the system 100 to operate in real time (i.e., to minimize delay). ●The second factor is calculations related to computer (CPU, GPU) processing, data storage (volatile memory and / or permanent storage), and data transparency (bandwidth).

[0061] LIGHT_DATA LIGHT_DATA may have one or more of the following data types: SPECTRAL_DATA (i.e., spectral distribution data), L_DATA (i.e., light intensity data, particularly LUMINANCE), LOCATION_DATA, and TIMEPOINT_DATA.

[0062] In order to enable the data consumer 103 to derive information about object 110, LIGHT_DATA has a quality selected according to a specific use case application.

[0063] data structure Those skilled in the art can process such LIGHT_DATA using appropriate data structures. Therefore, the explanation will remain at a conceptual level, by using vectors and the like.

[0064] However, for explanatory purposes, it is helpful to specify some rules. ●Each vector (notated as [] or {}) has several vector elements. Often, a {} vector has N elements and is also written as {element1, element2, ..., elementn, ..., elementN}, or simply {element1, elementn, elementN}. ● Vector elements are computer-processable numerical values, and those skilled in the art can select appropriate data rules for a computer to process such values. ●As used herein, the term "value precision" refers to the choice of using real numbers, integers, etc. ● For integers, the precision of a value can be given by the number of bits or by the number of distinct possible values. For example, a number represented by a 16-bit integer has a value precision of "16 bits" or "65536". ● Numerical values ​​such as A in vector [A], B in vector {B}, and C in vector {C} can have different precisions. However, unless otherwise specified, all elements of a given vector should have the same precision. ●Since vectors represent different data, the value precision in [A] is "amplitude precision" (see parameter 354), the value precision in {T} is "transmission precision", and so on.

[0065] Details about LIGHT_DATA The SPECTRAL_DATA data represents the specific spectral distribution 310 of light 210. In other words, SPECTRAL_DATA is spectral information and describes the spectrum. The spectrum can be conveniently represented by a curved diagram that shows the intensity across wavelengths as a continuous line.

[0066] As used herein, the term “spectral distribution of light” refers to a situation in which light 210 has different intensities (or “amplitude” A, or “light stream,” or “light power”) at different wavelengths. There are various methods for encoding distributions, etc., and the description and diagrams use vectors such as [A] (as in Figure 1) and {D} (as in Figure 2).

[0067] L_DATA represents the intensity criterion for light 210. Unlike SPECTRAL_DATA, L_DATA can be processed as a single numerical value (real number, integer), but not necessarily as a vector. There are two implementations for obtaining L_DATA: it can be measured (as LUMINANCE) or calculated (from SPECTRAL_DATA).

[0068] LOCATION_DATA represents the relationship of light 210 to a position on object 110 (e.g., a position on the object's surface) and / or a position on a sensor (for sensors in the form of a sensor array, see Figures 8 and 10). LOCATION_DATA can be represented by a vector with two elements for the X and Y coordinates, but the use of Cartesian coordinates is merely for illustrative purposes. Other coordinate systems can be used similarly. When describing an implementation with multiple sensors arranged in an XY array plane such that LOCATION_DATA is essentially linked to a specific sensor providing the data, the description will refer to the X and Y coordinates.

[0069] TIMEPOINT_DATA refers to a specific point in time (e.g., a timestamp having a calendar date and further displaying hours, minutes, and / or other granularities). As used herein, TIMEPOINT_DATA is captured when the time interval in which the sensor captures light 210 ends. This is merely a convention. In many situations, the duration of the time interval can be ignored.

[0070] To provide LIGHT_DATA, one or more sensors in the data provider 101, ● An optical sensor for converting the amount of photons into a readable electrical signal, and ● Equipped with an analog-to-digital converter (ADC) that converts signals into data.

[0071] Ideally, LIGHT_DATA corresponds to the optical properties with the highest precision that the most advanced sensor technology can provide. Those skilled in the art are familiar with photodetectors, photodiodes, phototransistors, photomultiplier tubes, photomultiplier tube cameras, photometers, and ADCs. In many implementations, both functions (acquiring a signal and digitizing it) are implemented in a single device. Two technologies, namely charge-coupled devices (CCDs) and complementary metal-oxide-semiconductor (CMOS) sensors, are given here as representative examples.

[0072] LOCATION_DATA and TIMEPOINT_DATA are typically available as metadata (e.g., associated with SPECTRAL_DATA). Those skilled in the art are familiar with tracking such metadata, so further explanation is unnecessary.

[0073] Encoding, decoding, and parameters Generally, the term "encoding" refers to representing a physical phenomenon using data so that data processing can retrieve the phenomenon. In the context of this explanation, ● The term "encoding" applies to representing a specific spectral distribution 310 of light 210 by SPECTRAL_DATA. ● The term "decoding" applies to re-establishing (or re-recognizing) a specific spectral distribution 390 from SPECTRAL_DATA.

[0074] In a highly simplified theoretical scenario, two specific distributions, 310 and 390, would be identical. However, considering the use-case specific information processing performed by the data consumer 103, this may not be necessary in many use cases.

[0075] There are limitations to both encoding and decoding, and these limitations can be described by parameters. These parameters are called "encoding parameters" because they should be considered for encoding. As an example, the spectral diagram for distribution 310 also shows the following encoding parameters. ●Wavelength range 351, or (λ_min, λ_max), ● Spectral resolution 352, or Δλ, ● Amplitude range 353, and ● Amplitude accuracy 354.

[0076] The coding parameters 351, 352, 353, and 354 define the overall set 315 of spectral distributions, if coding is possible. Figure 1 shows set 315 by extending a 2D diagram of distribution 310 into a 3D diagram with P set members (i.e., P are set concentrations).

[0077] In other words, distribution 310 is a specific distribution of light 210 (reaching data provider 101), and set 315 contains possible distributions defined by coding parameters. Thereafter, set 315 symbolizes each spectral distribution for which coding is possible. The coding results in a specific code (also called a "spectral distribution identifier") that is unique for each member of set 315.

[0078] If distribution 310 is measured, the coding parameters 351, 352, 353, and 354 can also be discussed in consideration of the data obtained by the measurement. The intensity value A (or "amplitude value") can be measured absolutely (in units such as candela / square meter) or relatively (relative to a reference amplitude, for example).

[0079] As used herein, the amplitude vector [A] = [A1, A2, ... Ak, ... AK] is a set of intensity values ​​(i.e., "amplitude values") for distribution 310, where k is the wavelength index. Any other distribution in set 315 is similarly described. K can be thought of as the "wave number".

[0080] Figure 1 shows a specific spectral distribution 310 as a normalized distribution, with the highest intensity at the top line and the lowest intensity at the bottom line. For illustrative purposes only, the figure adds "1" and "0" respectively. The same principle applies to all members of set 315 (where the amplitudes are normalized).

[0081] Regarding the wavelength range 351, note that encoding and decoding are available for light within this range. Alternative notations may be "wavelength band" or "spectral band." The wavelength range 351 extends from the minimum wavelength λ_min to the maximum wavelength λ_max. Exemplary values ​​for lambda are λ_min = 380 nanometers and λ_max = 780 nanometers (visible light). For convenience, wavelength λ is used in this explanation, but this teaching is also applicable to optical frequencies. In index notation, k is 1 at λ_min and K at λ_max.

[0082] For a spectral resolution of 352, Δλ is the minimum wavelength difference for which its encoding (and decoding) is available. For simplicity, this figure shows that a particular wavelength λ is, for example, λk = λ_min + (k-1) *We assume that Δλ is equal across the entire range 351, so that it can be described as Δλ. In other words, individual wavelengths are equidistant from k to k+1. An exemplary resolution is, for example, Δλ = 0.5 nanometers. Such a resolution at 0.5 nm is an example of so-called "high spectral resolution" or "hyperspectral resolution". The explanation sometimes uses K = 801 as an example for wavelengths from 380 to 780 nm. Spectral resolution can also be expressed, for example, relative to the range 351 (i.e., as the ratio Δλ / (λ_max - λ_min)).

[0083] With respect to the amplitude range 353, encoding (and decoding) is available for amplitudes ranging from the minimum amplitude A_min to the maximum amplitude A_max. The figure for distribution 310 is shown normalized to (A_min, A_max) = (0, 1).

[0084] Regarding amplitude precision of 354 or ΔA, this parameter is the value precision described above that applies to amplitude A. In the example, the explanation assumes 16-bit precision.

[0085] It is sometimes convenient to apply the term "vector space." Set 315 (see Figure 1) corresponds to a "vector space" because, as a basis, it contains all vectors (i.e., intensity distributions) defined by parameters 351, 352, 353, and 354. The "space" is K-dimensional.

[0086] Encoder function and decoder function This description details the encoding and decoding processes by describing the encoder function 105 (Method 400, see Figure 3 for implementation) and the decoder function 106 (Method 600, see Figures 19-21). As described below, the functions can be implemented such that the components for the encoder function 105 are located on the data provider 101 and the components for the decoder function 106 are located on the data consumer 103. This is convenient for the sake of explanation, but not mandatory.

[0087] Method 400 (for encoding) is partially computer-implemented, and Method 600 (for decoding) is computer-implemented in all its steps. Those skilled in the art can provide suitable computers depending on the conditions of the use case, and the description will, in some cases, give some examples. Different locations (provider and consumer) are expected to be connected to different physical computers. The description will also use the term “operational function” because the calculations for the encoder function 105 can be distributed even to a finer location granularity.

[0088] limit Here again, the following should be noted, as encoding parameters 351, 352, 353, and 354 describe the restrictions. ●Outside the wavelength range of 351, encoding (decoding) generally does not occur. ●For wavelength differences less than Δλ (i.e., spectral resolution 352), there may be differences in intensity, but such differences cannot be encoded (or decoded). In other words, two similar distributions with differences smaller than Δλ belong to the same set member (of set 315), and encoding cannot distinguish between the two distributions. ● The light may have an intensity above or below the amplitude range of 353, but no encoding (or decoding) is performed for such excess light. ● For amplitude differences with an accuracy of less than 354, encoding (or decoding) will not be performed.

[0089] As used herein, the delta parameter (i.e., resolution 352 and precision 354) is assumed to be constant over the range parameter (i.e., ranges 351 and 353). This is convenient for illustrative purposes but not required. Since some use cases may require nonlinearity, implementations may take nonlinearity into account. Such nonlinearity can be described by further coding parameters.

[0090] The concentration P (of set 315) is relatively high. For example, the amplitude vector [A] has K elements and has an arbitrary distribution from [A]=[0,0,...0] (all K elements are 0) to [A]=[65535,65535,...65535] (all K elements are maximum), and Ak can be expected to be any number between 0 and 65535. This number P will be explained again when we discuss decoding.

[0091] For simplicity, we can assume that set 315 also applies to decryption, and exceptions based on certain use cases will be discussed later.

[0092] Optical parameters and device parameters The coding parameters can be distinguished into optical parameters (wavelength range 351, spectral resolution 352) and instrument parameters (amplitude range 353, 354).

[0093] Other embodiments Optionally, encoding and decoding may also be related to light intensity. Light intensity can be encoded (and decoded) separately from or in conjunction with the spectral distribution.

[0094] L_DATA can also be defined by encoding parameters, but these parameters are simplified to magnitude range and magnitude precision. Wavelength is irrelevant. For simplicity, the explanation assumes that L_DATA has a magnitude range corresponding to parameter 353 (i.e., the amplitude range of SPECTRAL_DATA, such as between 0 and 1) and a magnitude precision corresponding to parameter 354 (e.g., 16-bit precision is the same precision applied to spectral data). This assumption is convenient for implementations that use the same sensor to measure L_DATA and SPECTRAL_DATA (see sensor 130 in the example in Figure 9), and for implementations that use multiple sensors of the same type.

[0095] Figure 1 is simplified in that it shows LIGHT_DATA from one location at one time interval. Those skilled in the art will not need further explanation herein, for example, that LOCATION_DATA and TIMEPOINT_DATA can be tracked by metadata.

[0096] spectroscopic method A data processing system involving encoding and decoding can be implemented using a commercially available spectrometer (or spectrophotometer). Such a spectrometer provides spectral distribution data through individual data points corresponding to individual wavelengths.

[0097] The spectrometer technique can be summarized as encoding a specific spectral distribution by measuring the amplitude A (within the amplitude range 353 with precision 354) for virtually all discrete wavelengths within range 351 with resolution 352. SPECTRAL_DATA corresponds to the amplitude vector [A]. Those skilled in the art are familiar with the terms “spectral bin,” and sometimes “spectral pixel” to represent it.

[0098] Those skilled in the art can implement similar spectroscopic techniques by using multispectral cameras, hyperspectral cameras, etc. A suitable overview of spectroscopy can be found in the book "Hyperspectral Imaging: Techniques for Spectral Detection and Classification" by Chein-I Chang, Springer Science+Business Media New York 2003. ISBN 978-1-4613-4820-7.

[0099] The spectrometer provides not only SPECTRAL_DATA but also LUMINANCE data, as well as one or more of LOCATION_DATA and TIMEPOINT_DATA.

[0100] Filtering methods, uniqueness conditions Figure 2 provides an overview of Figure 1 and details how the data provider 101 implements the filtering method.

[0101] In this filtering method, the data provider 101 is implemented by hardware and software components. ● The hardware component comprises filters 120-1, 120-n, and 120-N, and one or more sensors (see Figure 3) to provide provisional coding ("provisional intensity" value B in vector {B}). The filters conform to so-called uniqueness conditions related to the coding parameters (see parameters 351, 352, 353, and 354 in Figure 1). Filter 120-n filters the received light (i.e., light 210), and one or more sensors measure the intensity of the filtered light over a wavelength range (parameter 351). The measurement involves integration (see Figure 15). Alternative filter conditions are described at the end of this explanation (see Figures 23-24). ● The software component is a function that fine-tunes {B} to the intensity vector {D}={D1,Dn,...DN}, i.e., SPECTRAL_DATA. The function also processes the intensity data and calibration data (see Figure 3).

[0102] From an overall perspective, data provider 101 encodes the (specific) spectral distribution 310 of light 210 (see Figure 1) into spectral distribution identifiers 250 / 270 (or vector {D}).

[0103] As already explained, the spectral distribution 310 is the distribution within the set 315 described above, which includes the possible distributions defined by the coding parameters 351, 352, 353, and 354. The number of elements in vectors {B} and {D} corresponds to the number of filters (but not the number of K distinguishable wavelengths; see parameters 351 and 352). The number of filters is given as N. There are at least N=3 filters.

[0104] Figure 2 symbolically represents the filters with vertical lines. Filter 120-n filters out light 210 to produce filtered light 220-n, which means (i) light 220-1 is filtered out by filter 120-1 having transmission function T1, (ii) light 220-n is filtered out by filter 120-n having transmission function Tn, and (iii) light 220-N is filtered out by filter 120-N having transmission function TN.

[0105] Filter 120-n is selected according to coding parameters 351, 352, 353, and 354. The transmission function Tn is defined according to the parameters spectral resolution 352 and wavelength range 351, and consequently, Tnk is the transmission of filter n at wavelength k.

[0106] The following uniqueness conditions apply. ● For any given wavelength k, the transmission function Tnk can be combined or concatenated for N≧3 filters. This concatenation is unique for any wavelength k. A more detailed explanation is available in Figures 11-14. ●This concatenation is also unique for any expected spectral distribution within set 315 (and it can also be described by at least parameters 351, 352, 353, and 354, depending on the parameters; see Figure 1). ● The transmission function Tnk remains unchanged (at least for the duration of the time intervals during which the N filters receive and filter light). Therefore, the filters can be considered as single-state filters. The examples shown in Figures 9 and 10 are available.

[0107] In another chapter below, it will be described (see FIGS. 11 to 14 and FIG. 18, for example, by applying rules, by applying simulations and other numerical calculations) how the filter set 120 having the filters 120-n can be defined. Briefly speaking, the filter designer uses the optical parameters 351 / 352 to identify which light at which wavelength needs to be encoded with which resolution. When the filter changes the amplitude (of the light to be filtered), the filter designer also takes into account the device parameters 353 and 354. For example, a filter that attenuates light so that a sensor cannot measure the filtered light with a given accuracy (parameter 354) is not appropriate.

[0108] FIG. 2 also introduces that the filter can be a reference filter 125-n within the reference filter set 125.

[0109] Comparison between methods Both methods (the spectrometer and the filter methods) can be used as alternatives and are not necessarily competing. Different usage scenarios set the encoding parameters. The filter method uses a smaller number of data elements (in many scenarios, N<K, etc.), and the filter method uses two hardware components (the filter and the sensor), but it seems possible to apply the filter method to encode a specific spectral distribution 310 according to the same encoding parameters 351, 352, 353, and 354 as the spectrometer method. For example, the filter method encodes data with the same spectral resolution Δλ (or even smaller) as the spectrometer method.

[0110] Since K is larger than N, the filter method can also be considered to be more efficient than the spectrometer method (at least in the sense of reducing the amount of data to be processed, for example, by the data link 102). Reducing the amount of data to be communicated can also support real-time aspects.

[0111] Interrelationship between methods Using spectroscopic techniques, it is possible to measure filter characteristics (such as transmission function), and as a result, theoretically, a computer can convert [A] to {D}. Calibration techniques can use a portion of the [A] data, as described below.

[0112] Spectral distribution [A] ~ It is also possible to simulate this, and by applying the simulation, a filter that meets the uniqueness condition can be provided (or the conditions for an existing filter can be checked), and a decoder function can be implemented.

[0113] Filter implementation Depending on the filter implementation options, the filtering method can be further: ● Filter wheel implementation (In the measurement sequence, filter 120-n of filter set 120 is changed to one sensor, and filter 120-n remains unchanged between the individual measurement details in Figure 9), ● Characterized by a color mosaic implementation (using filters permanently assigned to the sensor so that measurements are performed in parallel; details are shown in Figure 10, where the words refer to colorimetric and mosaic).

[0114] Since both implementation options for the filter have virtually no impact on the implementation of data consumer 103 (which has decoder functionality), the description will describe the implementation details of the decoder functionality towards the end of the description.

[0115] Here again, it should be noted that the filter is a single-state filter in the sense that a particular filter maintains its transmission characteristics (see [T]) unchanged during at least one measurement (see steps 430 and 440-n in Figures 5 and 7).

[0116] Filters and conductors As will be described in detail later, the filter transmits light with transmission that is wavelength-specific. In some implementations, the filter technique further applies an optical transmission element that passes "all wavelengths" by transmission that is substantially wavelength-nonspecific (lambda isotropy). This description supports that distinction by using terms such as "photoconductor" and "neutral filter".

[0117] Decryption Data consumer 103 implements the decoder function from {D} (for an overview, refer to the decoder function 106 in FIG. 19). As a result, data link 102 does not need to transmit K data points (like the spectrometer technique), but only transmits the spectral distribution identifier {D} (250 / 270 in FIG. 3) having N < K data points. Therefore, the filter technique can provide advantages for data links with bandwidth limitations.

[0118] Data consumer 103 decrypts {D} so as to re-establish a specific spectral distribution 310 to a specific spectral distribution 390. Put simply, the decoder function aligns a specific distribution identifier {D} to a specific spectral distribution 390.

[0119] The accuracy of that alignment is such that data consumer 103 can derive information about physical object 110. Ideally, encoding parameters 351, 352, 353, and 354 are also applicable as decryption parameters, but in some use cases, decryption can use parameters that reduce the re-establishment accuracy.

[0120] The decoder function can be implemented, for example, ● Machine learning (refer to the details in FIG. 20), ● Reference tables, database queries, etc., ● Recursive fitting functions, etc., by techniques like these.

[0121] In other words, in the filtering method, a specific spectral distribution is encoded using a set of different transmission modulation filters (mostly non-blocking filters; see Figures 11-14 for details) placed in the optical path from the light source (or object 110) to the sensor. Decoding is performed by a reconstructor (i.e., a decoder function), which can be implemented using machine learning or other methods, to recover the spectral distribution (at least to a degree suitable for consumption).

[0122] encoder Figures 3-6 illustrate in more detail the filtering technique for encoding a specific spectral distribution 310 of light 210 into a spectral distribution identifier 250 / 270 (vector {D}). Figures 3-4 introduce the system components (for the encoder function), and Figure 5 shows the method 400 being implemented when light 210 arrives.

[0123] It is helpful to keep in mind that the filtering technique implies the activity that should be carried out before light 210 arrives. Figure 6 shows method 400. ●Design time (activities to design the reference filter, reference 701) and ● Presented in the context of calibration time (an activity to consider the deviation from the reference filter, see reference 702).

[0124] Similar to Figure 3, the encoder function 105 is implemented by hardware components (left side of the figure) and software components (or "operating functions," right side of the figure). ● The hardware components are a filter set 120 (having filters 120-1, 120-n, and 120-N; see Figure 2) and a sensor 130. The hardware components provide a provisional intensity vector {B}. ●In the first alternative configuration, the software components are mapper 140 and divider 150, and in the second alternative configuration, they are divider 160 and mapper 170.

[0125] The software component processes the provisional intensity vector {B} by using auxiliary data to compensate for the variation constraints. ● Mapper 140 / 170 processes {CAL} to compensate for filter variation constraints. ● Dividers 160 / 150 process L_DATA to compensate for intensity variation constraints (or "luminance constraints").

[0126] Therefore, the combination of software components can be collectively referred to as a variation compensator (compensators 140 / 150 and 160 / 170). The software components provide spectral distribution identifiers 250 / 270.

[0127] Hardware components The explanation proceeds from left to right, and in some cases refers to method steps (see Method 400, Figure 5). The physical object 110 provides light 210 (by radiation, transmission, reflection, etc.). The light 210 has wavelength-specific intensity A within the wavelength range (λ_min, λ_max), as already introduced in Figure 1.

[0128] The filter set 120 comprises N filters 120-1, 120-n, and 120-N, where N is at least N=3. Filter 120-n filters the light 210 individually. Each filter 120-n in set 120 has a filter-specific transmission function Tn (T1, Tn, and TN, or collectively referred to as {[T]}) with wavelength-specific transmission Tnk. As used herein, “transmission” refers to the intensity ratio between the outgoing and incident light, Tnk = A_out_k / A_in_k.

[0129] Alternatively, the transmission function Tnk can be considered a "transfer function" or "wavelength-dependent transparency characteristic." The graphical representation of this function is sometimes called a "conversion efficiency curve."

[0130] N≧3 filters in filter set 120 satisfy the uniqueness condition described above (i.e., the minimum set of conditions). Since the conditions are too complex to be illustrated in a single diagram, Figure 3 simply represents the conditions with two symbols. ●The first symbol has two vertical dashed lines, indicating that for any two wavelengths k and q, {T}k may be different from {T}q. The vertical lines have different heights (at k and q) not only for T1 but also for Tn and TN. Several exceptions apply. (A difference is established if at least one element of the vector (at vector position n) is different in both vectors. This will be further explained using Figures 11-14.) ●The second symbol has a horizontal dashed line and indicates that the transmitted Tk (as T1k, Tnk, TNk) exceeds a specific threshold of 340. The filter does not block all incident light, but allows at least a certain percentage to pass through. Since this figure is symbolic only, the exception applies here as well (see Figures 11-14).

[0131] Filter 120-n provides filtered light 220-n individually (i.e., separately).

[0132] Sensor 130 individually measures the intensity of filtered light 220-n for each filter 120-n and individually obtains a provisional intensity value Bn. Simply put, Bn is the integral of the light intensity (filtered) A over the wavelength range (λ_min, λ_max). Bn is a numerical value (because sensor 130 functions as an ADC). Bn is provisional (meaning it is intermediate and not yet determined to be used as a code or spectral identifier). For all N filters 120-n, the provisional intensity value Bn can be written as a provisional intensity vector {B}.

[0133] Considering the uniqueness condition, vector {B} is unique for all members of set 315 (see Figure 1), but matching by the decoder function to re-establish the spectral distribution (310-390) is not possible. However, the software component on the right side of the steps in the figure here processes {B} as described below. In other words, the software removes the provisional characters to make the vector final.

[0134] constraints Encoding a specific spectral distribution 310 of light 210 involves means to relax the two variation constraints mentioned.

[0135] More specifically, regarding filter variation constraints, filters 120-n may differ from the reference filter (see Figure 2 for reference filter 125) in their transparency function Tn. This can be due to various reasons, such as manufacturing variations. In other words, individual filters 120-n vary in their transparency function (compared to the reference transparency function of reference filter 125-n, see Figure 2).

[0136] For example, different filter sets 120 are designed to be specified for a single reference filter set 125 (see Figure 2), but are manufactured separately. For the same light (i.e., the light received 210), variations will cause the sensor 130 to provide different encodings, for example, {B}={12,23,16} for a particular first filter set 120, or {B}={13,23,16} for a particular second filter set 120.

[0137] In this example, the vector element B1 is different, and this difference could potentially prevent the decoder function from correctly re-establishing the spectral distribution.

[0138] With respect to the light variability constraint ("luminance constraint"), the overall intensity of light 210 may vary (for iterations when performing method 400), but the spectral distribution 310 remains the same (when amplitude offset is ignored).

[0139] For example, {B}={13,23,16} can be measured for relatively "weak" light, while {B}={23,33,26} can be measured for relatively "strong" light. Since both vectors represent the same spectral distribution, encoding using {B} will be different for different light intensities. As a result, the decoder function re-establishes the spectral distribution differently.

[0140] As a result, due to both constraints, the vector {B} still does not provide an encoding (because it does not yet identify the spectral distribution). Therefore, {B} is a provisional vector.

[0141] Software Components This figure shows the calculation functions 140, 150, 160, and 170 that provide spectral distribution identifiers 250 / 270 that encode a specific spectral distribution 310 of light 210. The simplified calculation functions are: ●By using the calibration data {CAL} and the mapping steps 440 / 470, ●The variation constraint is relaxed (i.e., as a variation compensator) by using the intensity criterion value L_DATA and the division step 460 / 450.

[0142] Since the order of step execution is not important, the arithmetic function has two alternative forms: ●In the first alternative configuration using the mapper 140 and the divider 150, the first is mapping, the second is division, and ●In the second alternative configuration using the divider 160 and the mapping device 170, the implementation can be performed by first performing division and second performing mapping.

[0143] Calculation in an encoder (first alternative form) In the first alternative form (steps 440-n, 450-n), the first operation function 140 ("mapper") maps the provisional intensity value Bn to the calibrated intensity value Cn, which is an element of the calibrated intensity vector {C}, according to the given calibration data ({CAL}) (440-n), and the second operation function 150 ("divider") individually divides the calibrated intensity value Cn of the calibrated intensity vector {C} by the intensity reference value (L_DATA) (450-n) to calculate the normalized calibrated intensity value Dn, which is an element of the normalized calibrated intensity vector {D}. The spectral distribution identifier 250 is provided as the normalized calibrated intensity vector {D}.

[0144] Calculation in an encoder (second alternative form) In the second alternative form (steps 460-n, 470-n), the third operation function 160 ("divider") individually divides the provisional intensity value Bn by the intensity reference value (L_DATA) (460-n) to calculate the normalized intensity value Fn, which is an element of the normalized intensity vector {F}, and the fourth operation function 170 ("mapper") maps the normalized intensity value Fn to the calibrated normalized intensity value Dn, which is an element of the calibrated normalized intensity vector {D} (470-n). The spectral distribution identifier 270 is provided as the calibrated normalized intensity vector {D}.

[0145] Simplification Both alternative forms provide spectral distribution identifiers. Either alternative form can be implemented. Since there is virtually no difference (to the decoder) between the "normalized calibrated" intensity vector (first alternative form) and the "calibrated normalized" intensity vector, the description and diagrams use {D} for both (i.e., identifier 250 / 270).

[0146] Implementation of arithmetic functions A person skilled in the art can implement the arithmetic function by selecting the appropriate technology. Directions are available according to the use case scenario, particularly other factors. For use cases with real-time requirements, a person skilled in the art may even consider using a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). For use cases with relatively large amounts of data (images spanning large object areas), real-time requirements may not be as critical, and consequently, the arithmetic function can be better implemented by a CPU / GPU.

[0147] The same reasoning applies to other calculations, such as those for providing {CAL} and L_DATA to the calculation functions 140 / 150 and 160 / 170.

[0148] Acquisition of auxiliary data {CAL} Referring briefly to Figure 6, calibration data {CAL} becomes available (before light is received in step 410) through a process that includes measuring the filter deviations. Such filter deviations for filters 120-n can be obtained by comparing them with reference filters 125-n (see Figure 2).

[0149] Calibration will be explained in more detail below.

[0150] Acquisition of L_DATA by measurement Figure 4 shows some of the hardware components from Figure 3, including additional sensors 131A / 131B for acquiring L_DATA by measuring brightness.

[0151] In other words, in both alternative forms (see Figure 3), the intensity reference value L_DATA can be obtained by measuring the light intensity by sensors 131A / 131B (210, step 431). This corresponds to method step 431 (measuring intensity as LUMINANCE).

[0152] As shown in Figure 3, the received light 210 proceeds to filter 120-n of filter set 120, and the filtered light 220-n proceeds to sensor 130 which provides a provisional intensity vector {B}.

[0153] As shown by the thick line in Figure 4, light 210 proceeds to sensors 131A / 131B that provide L_DATA. There are two options. ●According to the first option (A), light 210 is received directly (i.e., bypassing the filter set 120) and transmitted to sensor 131A. ●According to the second option (B), light 210 is indirectly received via the neutral photoconductor 121B and transmitted to the sensor 131B. The sensor 131B measures the intensity of light 210, correcting for the loss introduced by the neutral photoconductor 121B (step 431).

[0154] Obtain L_DATA through calculation. Returning to Figure 3, a quick look reveals that the intensity values ​​(such as Bn measured by sensor 130) can be processed to obtain L_DATA, for example, by averaging or other methods.

[0155] Since L_DATA is the input to the divider 150 / 160 (see Figure 3), the two alternative forms in Figure 3 offer slightly different options for calculating L_DATA.

[0156] In the first alternative form, the second operation function 150 ("divider") takes the intensity reference value L_DATA, ●By processing the provisional intensity vector {B}, or ● This can be obtained by processing the calibrated intensity vector {C}.

[0157] In the first alternative form, the second calculation function 150 can obtain the intensity reference value (L_DATA) by a calculation selected from the following:

[0158] ●Calculate the intensity reference value (L_DATA) as the sum of provisional intensity values ​​Bn (or as the sum of calibrated intensity values ​​Cn). ●Calculate the intensity reference value L_DATA as the average Σ / N of the provisional intensity value Bn (or as the average Σ / N of the calibrated intensity value Cn), and ●Calculate the intensity reference value L_DATA as the median value of the provisional intensity value Bn (or as the median value of the calibrated intensity value Cn).

[0159] In the second alternative form, the arithmetic function 160 ("divider") can obtain the intensity reference value L_DATA by processing the provisional intensity vector {B} through a calculation selected from the following: calculating the intensity reference value L_DATA as the sum Σ of the provisional intensity values ​​Bn; calculating the intensity reference value L_DATA as the average Σ / N of the provisional intensity values ​​Bn; and calculating the intensity reference value L_DATA as the median of the provisional intensity values ​​Bn.

[0160] Figure 5 shows a flowchart for a method 400 for encoding the spectral distribution of light 210 from a physical object 110 into spectral distribution identifiers 250 / 270. This explanation also refers to Figures 3 and 4.

[0161] In the light receiving step 410, at least one filter 120-n belonging to the filter set 120 receives light 210 from the physical object 110. As already described, the light 210 has a spectral distribution that is expected to be within the set 315.

[0162] In other words, light 210 has a wavelength-specific intensity Ak in a particular spectral distribution 310 within the wavelength range 351 from the minimum wavelength λ_min to the maximum wavelength λ_max. This does not mean that Ak must be known.

[0163] In filtering step 420-n, light 210 is filtered, resulting in filtered light 220-n. Filtering 420 is performed individually for each filter 120-n. Filters 120-n within filter set 120 satisfy the uniqueness condition (see Figures 4, 11-14).

[0164] In measurement step 430, sensor 130 measures the intensity of filtered light 220-n to obtain a provisional intensity value Bn. As indicated by index n, the provisional intensity value is filter-specific. Bn is provisional in the sense that it does not yet represent reality. The decoder function cannot re-recognize the spectrum. In vector notation, sensor 130 provides a provisional intensity vector {B}={B1,Bn,BN}.

[0165] The filtering step 420 and the measurement step 430 are performed individually for each filter. Figure 5 shows two main options. ● Use N filters in steps 420-1, 420-n, and 420-N, and use N sensors in steps 430-1, 430-n, and 430-N, or ● This involves using N filters and one sensor with multiplexing (steps 420-1 and 430-1 result in B1, steps 420-n and 430-n result in Bn, and steps 420-N and 430-N result in BN).

[0166] These two options align with two main implementation options for filters and sensors (e.g., a color mosaic with N sensors, a filter wheel with one sensor).

[0167] The subsequent steps (mapping 440-n / splitting 450-n (first alternative form) and splitting 460-n / 470-n) have already been described with reference to Figure 3. The steps yield {D} useful for spectral distribution identifier 250 / 270.

[0168] The filtering method was explained by introducing the encoder function 105 (shown in Figures 3-4) and the encoding method (Figure 5). Now, we will explain its implementation in more detail.

[0169] It should be noted that steps 410, 420, and 430 of the method are defined as method 401, and steps 440 / 450 and 460 / 470 are collectively referred to as sequence 402 ("Providing a Spectral Distribution Identifier").

[0170] This formal separation into hardware and software steps is useful for illustrating specific use case scenarios (Figure 22) and for explaining the procurement of auxiliary data {CAL} and L_DATA (see Figure 6).

[0171] Method and step sequence in preparation Figure 6 presents Method 400 in the context of design time (activity for designing the reference filter, activity sequence 701) and calibration time (activity in sequence 702 for considering the deviation from the reference filter).

[0172] The figure shows the repetition of method 400 using steps 420, 410-n, 430-n, and sequence 402. Step 431, which measures the intensity, has already been described (used by the option to measure L_DATA).

[0173] Looking at the right side of box 402, as a result of step 431, L_DATA is available by measurement (to the divider arithmetic function). Without referring to a specific step, L_DATA can be obtained by calculation as outlined above.

[0174] Identify the reference filter set during the design phase. Step sequence 701 describes the identification of a set of reference filters 125 having reference filters 125-n that meet the uniqueness criteria (see Figure 2).

[0175] Similar to step 711, parameters 351, 352, 353, and 354 are obtained (i.e., identified according to the use case scenario). In other words, this step refers to receiving the use case specification.

[0176] Similar to step 721, the reference filter set 125-N is identified (i.e., identified), and in principle there are two options. ●For physically existing sets of filters, and for which the transmission function {[T]} is known (e.g., by measurement using a spectrometer), the uniqueness condition is numerically checked, for example, by simulating light for substantially all members in set 315 (concentration P, see Figure 1), applying the simulated light to the filters, and checking whether all {B} (for all P, or for a subset) are unique. If uniqueness is found, the filter set is selected as a candidate to become the reference filter set 125. ●If the set of filters is not yet known, the simulation is performed for all members of set 315 (or for a random subset of set 315), but also for possible variations of the filter's transparency function {[T]}. A suitable function {[T]} identified as satisfying the uniqueness condition provides candidates for the reference filter set 125.

[0177] Similar to step 731 "Manufacturing Verification," it is verified that such filters can actually be reproduced (first option) or manufactured (second option). For example, the transparency function is determined by a specific filter steepness (e.g., Tnk=2 *This can result in relatively large changes in Tnk over relatively narrow wavelength differences, such as Tnq (qk < 10). In some cases, such filters may not actually be manufactured. In such cases, the simulation is repeated with other settings, or an existing set of filters is not considered to serve as a reference.

[0178] In other words, step 731 changes the candidate filter set into the base filter set.

[0179] When the reference filter set 125 is designed (step sequence 701 having steps 711, 721, and 731), the filter set 120 can be manufactured (according to the reference filter set 125).

[0180] It can be expected that the reference filter set 125 and filter set 120 are not physically identical. Since individual filters 120-n will exhibit some minor manufacturing variations (compared to the reference filters 125-n in set 125), calibration data will need to be obtained.

[0181] Looking at the left side of box 402 (the last step of method 400), the calibration data {CAL} is provided by sequence 702 using steps 712, 722, and 732. ●Similar to step 712, filter set 120 is provided (as physical filters manufactured to match the reference filter set 125). ●Similar to step 722, each filter 120-n is compared with the reference filters 125-n of the reference filter set 125. As a result of this comparison, the deviation {DEV} is identified (i.e., as a vector with elements for each of the N filters). This will be explained in detail below by describing Figures 15-17. ●Similar to step 732, the deviation is processed to obtain the calibration data {CAL} which is input into step sequence 402. In simplified form, {CAL} is simply the inversion or negation of {DEV}. For example, if a particular reference filter set 125 yields {B}={12,23,16} and filter set 120 yields {B}={11,24,17}, the calibration data would be {1,-1,-1}. Mappers 140 / 170 provide the correct values ​​(see {C}, {D}), and calibration by adding vector elements is just an example for mapping.

[0182] Step sequence 702 is performed before method 400 is executed, but sequence 702 does not need to be performed every time. In many cases, it is sufficient to obtain {DEV} and {CAL} once and use {CAL} for each call to step 440 / 470 (mapping).

[0183] Step sequence 702 can reach STOP under (at least) the following conditions. The stopping criteria are available during calibration. ●The deviation {DEV} can be relatively large, and as a result, the uniqueness condition cannot be met (for filter set 120). This can be established by testing, such as by applying the test as in step 721. ●{CAL} is available for a specific {DEV}.

[0184] As mentioned above, other stopping criteria are available at design time (see Figures 11-14, and the simulation in Figure 18). The explanation will describe the design and calibration times separately, but this is merely a convenient way of explaining them. It is possible to combine criteria. For example, manufacturing variations can be predicted (before manufacturing), and therefore such variations can be taken into consideration at design time. It is also possible to check the uniqueness condition for expected filter variations.

[0185] Furthermore, the following explanation provides further details, including design rules (design rules to provide uniqueness, see Figures 11-14), simulations, and numerical experiments (Figure 18), which can be similarly applied at design time 702.

[0186] Methods that take implementation options into consideration Figure 7 shows the details of filtering (step 420) and measurement (step 430), considering different implementation options for the filter and sensor. The steps can be presented by the filter index n = 1 to N, given here as {1, n, N}.

[0187] case

[0188]

number

[0189] case

[0190]

number

[0191] case

[0192]

number

[0193] L_DATA can be obtained by measurement, but in such cases the sequence requires at least one additional measurement, i.e.,

[0194]

number

[0195] case

[0196]

number

[0197] case

[0198]

number

[0199] We have distinguished both options in terms of method execution, explanation, and drawings, but we will explain them by looking at the hardware (filters and sensors) in Figures 8 and 9.

[0200] Figure 8 shows the filter and sensor array in different diagrams, i.e., The image above is a side view (X coordinate from left to right) with a Z value indicating the direction of light. Below is a top-down view of the array with respect to the XY plane.

[0201] Light 210 reaches the optical splitter 104, passes through the filter 120-n, and reaches the sensor 130-n.

[0202] There are two implementations (on the left and right sides of the diagram). ●The first implementation has only sensor 120-n (n=1~N) and yields only vector {B}. L_DATA is calculated. This implementation is the case shown in Figure 7.

[0203]

number

[0204]

number

[0205] The diagram above shows multiple rows with N filters for the first implementation, namely filters 120-1, 120-n, and 120-N in the first row (e.g., Y=1) and filters 120-1', 120-n', and 120-N' in the second row (e.g., Y=2). The rows essentially provide LOCATION_DATA.

[0206] The diagram above shows a zigzag arrangement of N filters for the second implementation, where M = 3 filters 121B, 121B', and 121B'' (+ corresponding sensor 131B). This arrangement resembles a chessboard. Having multiple LUMINANCE sensors 131B is convenient for measuring light intensity over a wavelength range more efficiently and accurately than using a single sensor.

[0207] multiplexing Measuring the intensity of filtered light 220 (430-n, see Figure 7) can be performed sequentially in multiple sequences for each filter 120-n using a single sensor.

[0208] Figure 9 shows an overview of the filter configuration for applying the multiplexing of the filtering and measurement steps. In parts (A) and (B), the figure focuses on the number of filters. ● Arrangement (A) has N filters 120-1, 120-n, and 120-N within the array 128-BLOCK. ●Arrangement (B) has N filters 120-1, 120-n, and 120-N, but also has one or more photoconductors 121A / 121B in array 129-BLOCK (conductor 121A is "unfiltered", and conductor 121B is the neutral photoconductor in Figure 4). An alternative term for "neutral photoconductor" is "neutral optical filter".

[0209] Filter 120-N (and conductors 121A / 121B) moves in a direction that is substantially perpendicular to the direction Z of light 210. During the measurement time interval, light 210 passes through one filter and reaches sensors 130 / 131A / 131B (as filtered light 220-n).

[0210] In arrangement (A), sensor 130 individually provides element Bn to {B} at the end of each time interval during which filter 120-n passes light 210. In arrangement (B), sensor 130 provides {B}, but also provides LUMINANCE. Sensor 130 has the function of sensor 130 in FIG. 3 for providing {B}, and also has the function of sensors 131A / 131B in FIG. 4 for providing LUMINANCE.

[0211] One skilled in the art can synchronize the movement of the filter so that the sensor output is aligned with the corresponding filter. Regarding the single-state condition described above, the moving speed of the filter is limited such that during the time interval in which the sensor measures light, the sensor receives light from only one filter.

[0212] In other words, for filtering the received light 210 in the sequence (step 420-n), the separation in progress, for one execution of the filtering step 430-n, can be provided by having filter 120-n of the filter set 120 arranged as a moving filter array 128-BLOCK or 128-WHEEL such that the received light 210 travels through only one filter (120-n) to a single sensor 130.

[0213] A multiple sequence using a single sensor 130 can also be enhanced by measuring the intensity of the received light 210 as LUMINANCE by the single sensor 130. In such an implementation, the moving filter array 129 (see part (B)) has two implementation methods. ●In the first method, the moving filter array 129-BLOCK further includes a gap 121-B that allows the received light 210 to pass directly to a single sensor (see Figure 4 for the function of sensor 131A). The neutral photoconductor is simply an air gap. ●In the second method, the moving filter array 129-BLOCK is equipped with a neutral photoconductor 121B (see Figure 4), which passes the received light 210 to a single sensor (see Figure 4 for the function of sensor 131B).

[0214] Figure 9 also distinguishes two basic movement techniques, moving the filter linearly as 128-BLOCK or 129-BLOCK as in parts (A) and (B), or as 128-WHEEL / 129-WHEEL as in part (C). In the case of the filter wheel technique in part (C), the filter moves around the center of the circle, otherwise the same teachings for 128-BLOCK and 129-BLOCK apply. In both techniques, light 210 reaches the Z direction perpendicular to the direction of movement (the direction of movement is tangential in the wheel implementation).

[0215] Therefore, the moving filter array can be implemented by selecting from the following: ● A filter wheel (128-WHEEL, 129-WHEEL) on which filters (120-n, etc.) are arranged along a circle, and the filter wheel as a whole (128-WHEEL, 129-WHEEL) rotates around the center of that circle, or ● A filter block (128-BLOCK, 129-BLOCK) through which filters (120-n, etc.) are arranged along a line, and the filter block moves in a linear direction (see parts (A) and (B)).

[0216] Multiple filters and sensors within the array Figure 10 shows the arrangement of multiple filters and multiple sensors (many-to-many) in a top view and a side view.

[0217] This arrangement is part of a device 800 adapted to encode a specific spectral distribution 310 of light 210. Light 210 reaches direction Z.

[0218] Device 800 provides an implementation that allows you to select the following design options: ● The intensity reference L_DATA for light 210 is obtained by measurement, not by calculation (although calculation may also be used). ● Although there are no moving parts, there are N filters, N filter-related sensors, and M all-wavelength sensors (see sensors 131A and 131B). ● Execution of Method 400 involves applying the steps in parallel (see the left side of Figure 7). ●Filters and photoconductors can be manufactured by similar manufacturing steps (see further references to thin films in this description).

[0219] Device 800 comprises a filter layer 825 attached to a sensor layer 835. Both the filter layer 825 and the sensor layer 835 are planar layers. The filter layer 825 is adapted to receive light 210 on its surface (see coordinates XY in the top view of the device) and to transfer the received light 210 to the sensor layer 835.

[0220] The filter layer 825 and the sensor layer 835 are divided into separate pixel positions 850-xy, which correspond to sensor 830-xy and filter position 823-xy.

[0221] The sensor 830-xy is adapted to quantify the intensity of transmitted light (see acronym A in Figure 1) by integrating the intensity of the transmitted light (see Figure 15) during the measurement time interval, providing a sensor-specific provisional intensity value representing the transmitted light (220). These intensity values ​​are Bn (in vector {B} and LUMINANCE).

[0222] For a continuous combination of M pixel positions (hereinafter, area 880), when N < M, the following applies. ● The N filter positions 823-xy hold a set of N ≧ 3 filters 823-21, 823-12, 823-32, 823-23 above the corresponding sensors 830-21, 830-12, 830-32, 830-23, such that the corresponding sensors provide the sensor-specific intensity value as the provisional intensity value Bn. The example of FIG. 10 shows N = 10 such positions. ● The M~N filter positions 823-11, 823-31, 823-22, 823-13, 823-33 are either empty (see the "gap" in FIG. 9) or hold a filter with a wavelength-unspecific transmission function (see the conductor 121B in FIG. 4), such that the corresponding sensors 830-11, 830-31, 830-22, 830-13, 830-33 provide the sensor-specific intensity value as the luminance value LUMINANCE (see sensors 131A / 131B in FIG. 4). The example of FIG. 10 shows M = 5 such positions (see the acronym "w" for "white").

[0223] The N filters corresponding to the filters 120-n of the filter set 120 (e.g., 823-21, 823-12, 823-32, 823-23) can conform to the uniqueness conditions already introduced.

[0224] The device 800 further includes a calculation module having calculation functions 840, 850, 860, 870. This module processes the provisional intensity value Bn into a normalized intensity value Dn that is an element of the normalized intensity vector {D}, and this normalized intensity vector {D} becomes the spectral distribution identifier 250 / 270 that encodes the specific spectral distribution 310 of the received light 210.

[0225] The two alternative forms introduced above in FIG. 3 also apply similarly. The first alternative form uses the calculation functions 840 ("mapper") and 850 ("divider") shown to the left of the dashed-dotted line, and the second alternative form uses the calculation functions 860 ("divider") and 870 ("mapper").

[0226] L_DATA is obtained as LUMINANCE (measured by M sensors; also refer to step 431 in FIGS. 4 and 6), and {CAL} is obtained as described above (refer to FIGS. 3 and 6). In that sense, since it is not necessary to calculate L_DATA from {B} as shown in FIG. 6 (refer to the Σ calculation), device 800 provides a simpler design.

[0227] In other words, in the case of the first alternative form, the calculation modules 840 / 850 in device 800 are implemented such that mapper 840 is adapted to map a provisional intensity value Bn to a calibrated intensity value Cn, which is an element of the calibrated intensity vector {C}, using predetermined device-specific calibration data {CAL} (step 440-n in FIG. 5), and divider 860 is adapted to individually divide the calibrated intensity values Cn of the calibrated intensity vector {C} by the luminance value LUMINANCE (step 450-n) to obtain a normalized intensity vector {D}.

[0228] In other words, in the case of the second alternative form, the calculation modules 860 / 870 of device 800 are implemented such that divider 860 is adapted to individually divide the provisional intensity value Bn by the luminance value LUMINANCE (step 460-n) to obtain a normalized intensity vector {F}, and mapper 870 is adapted to map the normalized intensity value Fn to a calibrated normalized intensity value Dn of the normalized intensity vector {D} using predetermined device-specific calibration data {CAL} (470-n).

[0229] In both alternative forms, vector {D} functions as a spectral intensity identifier 250 / 270.

[0230] The calibration is device-specific in the sense that variations in the filter and the sensor are taken into account.

[0231] Thin film technology Those skilled in the field of optical filter technology can implement the layers using various techniques known in semiconductor manufacturing. The sensor layer 835 can be designed similarly to the sensor array in a digital camera, and the filter layer 825 can be conveniently implemented by thin-film technology.

[0232] The following book provides an overview of thin films: Knittl, Z. Optics of thin films. John Wiley, London 1981.

[0233] Regarding the single-state conditions described above, please note that the thin film does not change during the measurement.

[0234] Now, let's continue the explanation by describing further implementation details.

[0235] Detailed uniqueness conditions Figures 11-14 show the transparency functions, thereby introducing design rules that are applied when the reference filter is selected in step sequence 701 (design time), particularly in activity 721 (see Figure 6).

[0236] We assume that parameters 351, 352, 353, and 354 have already been identified at this stage.

[0237] Figure 11 shows the transparency function for N=3 filters 125-n within set 125. (This explanation refers to the reference filter, but this rule also applies to individual filters 120-n).

[0238] As already mentioned, the N filters in filter set 125 satisfy the uniqueness condition. The figure shows the filter set transparency function {[T]} when [T1], [T2], and [T3] are present (each having vector elements from k=1 to K).

[0239] For each wavelength k, the filter-specific transmission function can be concatenated into a linked transmission vector {T}k having N elements. The figure shows such a vector for k on the left, as vector {T1k, T2k, T3k}, or collectively as {T}k. On the right, the figure shows the vector for q, as vector {T1q, T2q, T3q}, or collectively as {T}q.

[0240] The concatenated transmission vector {T}k is unique for any wavelength k. In other words, for any two wavelengths k and q, {T}k is different from {T}q. A difference is established if at least one element of the vector (at vector position n) is different in both vectors. For example, if {T}k is {20, 30, 40} and {T}q is {30, 30, 40}, the filter set has different transmissions in that at least one element (in this case, filter 1) has a different wavelength-specific transmittance.

[0241] In the symbol diagram of Figure 3, the wavelength-specific λ transmittance is greater than or equal to the transmission threshold 340-n for wavelength λk within the wavelength range 351. The provisional intensity value Bn will have a minimum value (greater than zero). (This is a simplification, and the sensor has minimum sensitivity.)

[0242] The threshold is shown by a dashed line at a specific rate T_threshold340. The threshold can be filter-specific. In other words, each filter transmits light across the entire minimum / maximum range 351, but there is virtually no light blocking. Alternatively, there is light attenuation across the entire range, but no blocking (no cutoff).

[0243] Figure 12 also shows the transparency function for N=3 reference filters 125-n within set 125, but shows that for some subranges, fewer than N filters can contribute to uniqueness.

[0244] The second filter indicates a cutoff range (between wavelengths identified by k and q) in which transmission is substantially unavailable, and the second filter blocks light. B2 (in the case of the second filter) is not zero in the case of the second filter (because the sensor integrates light over the entire range). However, for the cutoff range, the spectral distribution within k and q must be encoded only by the first and third filters.

[0245] Figure 13 shows the transmission function for an example with N=4 filters 125-n in set 125, again with some modifications (labeled "Element 1" to "Element 4"). For most of the wavelength range (parameter 351), the transmission Tn is greater than the threshold. The threshold 340 is the transmission at approximately T=0.5 and is represented by the horizontal dashed line. In other words, the minimum transmission in the spectral range 351 is 50%.

[0246] However, some filters may have a cutoff range in which transmission is virtually zero (i.e., the sensor does not detect light after the filter). This figure shows that filter 124-4 (i.e., transmission function T4) has two relatively narrow cutoff subranges, and filters 125-2 and 125-3 each have one cutoff subrange.

[0247] From the perspective of λ_minλ_max, the number of filters N' contributing to uniqueness (in this example) is 4, 3 (first cutoff subrange), 4, 3 (second cutoff subrange), 4, 3 (third cutoff subrange), 4, 3 (third cutoff subrange), and again 4.

[0248] Generally, if the transparency function of a particular filter has a constant value within a specific subrange, it does not contribute to uniqueness. Figure 13 shows such a value as the cutoff value (T=0).

[0249] FIG. 14 shows the transmission function of a single filter 125-n. In some sub-ranges, the transmission is constant over the wavelength (symbolically, the transmission has a flat region). A function having a graph substantially parallel to the horizontal axis can also be described by its derivative that becomes zero (i.e., dT / dλ = 0).

[0250] The figure is simplified, and the "constant" (or zero derivative) can be defined within the allowable band.

[0251] In this example, the transmission shows a first transmission flat region between wavelengths k1 and q1, and a second transmission flat region between wavelengths k2 and q2. The transmission varies at wavelengths outside the flat regions.

[0252] The flat regions can have two extreme values: ● a blocking flat region (see the discussion of FIGS. 12 - 13), ● a full transmission flat region (see the description of "neutral filter").

[0253] To design a filter set, at least the following design rules can be applied. ● The filter set should be avoided in that two filters each have two flat regions in the same sub-range (see FIG. 13. The number N' is N - 2 for such a sub-range). ● There should be variability in filter transmission for at least a subset of the filters (N' < N), but a sub-range with constant T for one or two filters is allowed (simplified uniqueness). ● There should be combinations of T that are equal over the complete range 351 (strict uniqueness).

[0254] Reference filter and individual filters: steps of mapping However, the reference filter 125-n may not be available for every measurement. The requirement that the calculation function must process the calibration data {CAL} has already been explained (using Figure 6, calibration time, step sequence 702).

[0255] Figure 15 shows a matrix with a spectral diagram (for the range 351): ●Having a first row showing the spectral distribution 310 of light reaching the filter, ●It has a second row showing the filter's transmission function, ●A matrix is ​​shown with a third row representing the filtered light measurements (distribution 320) obtained by a sensor that integrates incident light over wavelength.

[0256] The left column of the figure relates to method step "Measurement 430". Light 210 (see Figure 3) has a specific spectral distribution 310 (normalized here between 0 and 1). For simplicity, the figure shows an approximately equal distribution (between λ_min and λ_max).

[0257] Filter 120-n should have a transmission function Tn_individual (similarly normalized). Again, for simplicity of explanation, it should be assumed that Tn_individual increases almost linearly with increasing wavelength. In this example, filter 120-n should block at λ_min (Tn_individual=0) and be semi-transparent at λ_max (Tn_individual=1).

[0258] In this simplified example, the filtered light 220 (see Figure 4) has a spectral distribution 310 that starts at low intensity from λ_min and reaches an intermediate intensity at λ_max (similar to the almost unfiltered distribution 310).

[0259] As already mentioned, sensor 130 (see Figure 3) is equipped with an analog-to-digital converter. The sensor measures all light that reaches it during the measurement time interval, whatever the wavelength lambda may be. In other words, sensor 130 simply identifies the amount of photons that reach it (during the time interval). The area below line 321 corresponds to the numerical output of sensor 130, i.e., the provisional intensity value Bn.

[0260] Bn corresponds to the integral of the filtered light (intensity) between λ_min and λ_max.

[0261] If the values ​​for distribution 310 are known (e.g., measured by spectrometer techniques), they can be given as vectors [A] (k=1~K). If the transmission function Tn is also known, the spectral distribution for filtered light 220 is given by Tn for k=1~K. * It can be calculated by [A]. * [A] can be integrated, and the calculation will result in Bn.

[0262] However, filters and sensors are not computers that can accurately calculate Bn. The aforementioned variability constraints apply.

[0263] The central column of the figure is explained below, taking into account the method step mapping 440 / 470 in Figure 5. In this case as well, the distribution 310 should remain unchanged (as shown in the left column).

[0264] Tn_reference is the so-called standard transparency function. In this simplified example, the manufacturer defines Tn as increasing linearly from 0 to 1, from λ_min to λ_max, as symbolized by the dashed graph. Of course, Tn_reference can be given for each k (k=1 to K). Tn_reference can be considered a nominal curve or reference curve.

[0265] There is a slight offset between Tn_individual and Tn_reference, meaning that from λ_min to λ_q, filter 120-n transmits more light than expected, and from λ_q to λ_min, transmits less light than expected.

[0266] As a result, Bn measured by sensor 130 (with filter 120-n) does not necessarily correspond to the value that can be expected from a reference instrument (such as reference filter 125-n). The figure illustrates this by the area between line 321 (Tn_individual) and line 322 (Tn_reference). This area must be considered in terms of positive and negative values.

[0267] Since Bn is acquired by a filter (and sensor) that does not necessarily correspond to the reference filter (and reference sensor), adjustment may be required.

[0268] As shown by step 722 in Figure 6, filter 120-n is compared with reference filter 125-n to obtain a deviation {DEV}, and from the measured deviation, calibration data {CAL} can be calculated as in step 732 (Figure 6). For each specific measurement (step 430 in Figures 5-6), adjustment is made by using {CAL} in mapping step 440 / 470.

[0269] Light intensity L_DATA, division step As shown on the right side of Figure 15, the received light has a specific spectral distribution 310' (normalized between 0 and 1), but the overall intensity of the light is different (compared to 310' in the left and middle columns). In this example, the light is brighter.

[0270] Therefore, distribution 310' represents an offset relative to distribution 310, and for all wavelengths λ, the intensity of the received light is higher by the same amount (it may also be lower).

[0271] Filter 120-n maintains its transmission function (this function is independent of light intensity). As a result, sensor 130 acquires Bn over a larger area (below line 321', which is different from the area below line 321).

[0272] In other words, there are two offsets to consider. ●The first offset is the difference between distribution 310' and distribution 310, and for all k, [A] of distribution 310' is [A] * It is a coefficient. ●The second offset is the difference in the measured value Bn.

[0273] The offset of distribution 310' changes with each measurement (it cannot be predicted at design time (701 in Figure 6) and at calibration time (702 in Figure 6)), and is therefore determined in step 431 by determining L_DATA (see method 400 in Figure 6). L_DATA (measured or calculated as LUMINANCE) is the input to the divider function 150 / 160 (see Figure 3).

[0274] Deviation patterns Figure 16 shows the filter transmission function Tn with three typical deviation patterns. For convenience and to save space on the figure, Tn is shown from Tn=0.6 to Tn=1. The figure shows three curves (i.e., transmission functions), but this is not N=3.

[0275] The curves are shown for a reference filter (dashed line, Tn_reference) and an individual filter (Tn_individual, solid line). In other words, a particular filter 120-n has an expected (or "reference") transmission function Tn, while a particular filter 120-n has an individual transmission function Tn (see Tn_individual in Figure 15). T_individual can be measured (at least partially) so that deviations can be identified.

[0276] In calibration sequence 702 (see Figure 6), step 722 (comparing filter 120-n to reference 125-n and obtaining the deviation {DEV}) does not need to be performed for each wavelength. It is possible to obtain {CAL} in step 732 by identifying the deviation type and applying that type. ● Deviation type (1) refers to amplitude deviation. This indicates amplitude mismatch with lower-than-normal transmission from λ_min to λ_turn, amplitude mismatch with the same transmission at λ_turn, amplitude mismatch with higher transmission from λ_turn to λ_max, or vice versa. ● Deviation type (2) represents a repeating pattern deviation. It is similar to type (1) but has more turning points. ● Deviation type (3) refers to offset deviation. This is characterized by lambda shift. For any wavelength λ within the range between λ_min and λ_max, T_reference(λ) = T_individual(λ + delta λ).

[0277] Delta λ is not the same as Δλ mentioned above.

[0278] example For example, the standard filter set provides {B}_regular={0.9676,0.8356,0.9176}, and the individual filter set 120 provides {B}_individual={0.93760.8566,0.9526}.

[0279] Filter 120-1 provides B1 with ΔB1 = 0.030 (i.e., it measures more than the rated filter), filter 120-2 provides B2 with ΔB2 = -0.021 (less than rated), and filter 120-3 provides B3 with ΔB3 = -0.035 (also less than rated).

[0280] The filter's deviation type can be determined by sample-checking some of its values, and (if necessary) extrapolating to other values. Furthermore, for types (1) and (2), deviation characteristics such as λ_turn can be determined, and for type (3), delta λ can be determined.

[0281] Once the type is identified, {CAL} will be, for example, as follows: ● Identify {CAL} from the reference table. It is easy to obtain {CAL} by applying one or more of the following: λ_turn, sample-checked deviations in λ with a specific interval |λ_turn-λ|, and interval-specific corrections.

[0282] Application of calibration data in mapping Figure 17 shows the arithmetic function 140 / 170 ("Mapper," see Figure 3) along with its implementation options. Distinction between the first and second alternative forms is not necessary here.

[0283] The mapper receives {B}, provides {C} (or {F} and {D}), and also uses {CAL}. As already explained, {CAL} becomes available at calibration time 702 (steps 722 and 732 in Figure 6).

[0284] In the first implementation, mapping is performed by a pre-trained neural network 140 / 170-NETWORK. The network weights are the result of training, including {CAL} and / or deviation data {DEV} (see Figure 6).

[0285] In the second implementation, mapping is performed using tables 140 / 160-LOOKUP. This implementation is useful when the patterns or types described above apply (see Figure 16).

[0286] In the third implementation, both neural networks and reference tables are combined.

[0287] In the fourth implementation, the mapping is performed by simply applying arithmetic in 140 / 170-ARITHMETIC. For step 721, an example was given of adding {CAL}={1, -1, -1} to {B}.

[0288] Reconsideration of Uniqueness The design of the reference filter 125-n (refer to the design time) was introduced, the details of using the individual filters 120-n (refer to the mapping steps deviating from the reference filter 125-n), and different light conditions (considering L_DATA in the division step) were briefly mentioned. Therefore, here, the explanation reconsiders the uniqueness condition.

[0289] Such conditions can be verified by simulation. The explanation will focus on the reference filter, but the results also apply to filters with deviations.

[0290] Simulation Figure 18 shows the setup of a numerical experiment for estimating the degree of uniqueness by simulation. This experiment showed that the implementation options conform to the uniqueness condition. Filter variations (refer to the middle column of Figure 15) or luminance variations (refer to the right column of Figure 15) are ignored because they are irrelevant (or only slightly related) to the uniqueness condition.

[0291] The simulation computer is obtaining S distributions [A] ~ from 1 to distribution [A] ~ S distributions [A] belonging to the set 315 up to S ~ s. In this example, the S distributions are random samples of the elements within the set 315.

[0292] ~ represents the simulation and is a single ~ indicating that a specific vector [A] (or other database vector) has been simulated.

[0293] The simulation computer applies the filter set transmission matrix {[T]} to each distribution [A] ~ The value of 's' was processed, and an intermediate value was obtained. [A]_filtered_by_filter_1 ~ 1=[A1 * T11,...,Ak * T1k,...,AK ~ T1k] ~ 1 ... [A]_filtered_by_filter_2 ~ 1=[A1 * T21,...,Ak * T2k,...,AK ~ T2k] ~ 1 ... [A]_filtered_by_filter_n ~ s=[A1 * Tn1,...,Ak * TnK,...,AK ~ TinK] ~ s ... [A]_filtered_by_filter_2 ~ S=[A1 * TN1,...,Ak * TNK,...,AK ~ TNK] ~ It is S.

[0294] N * Each of the S lines [A]_filtered corresponds to line 321 in Figure 321 (left column). Figure 16 shows the simulated filter 120-1. ~ From 120-4 ~ The simulation of filtering is symbolized by showing (i.e., N=4). In other words, the simulation computer shows N in Figure 15. * We are simulating curve 321 in book S.

[0295] Next, the simulation computer calculates the intermediate value (simulated sensor 130~ The simulated intensity values ​​{B} for s=1 to s=S (by this method) ~ s={B1,B2,B3,B4} ~ The integral was performed with respect to s. Since the reference filter has reference transparency, the value is not "provisional" (therefore, {CAL} and mapping are not required).

[0296] For S=32768 and N=4, the exemplary values ​​are: {B} ~ 1 = {0.9543, 0.8765, 0.8932, 0.9701}, {B} ~ 2 = {0.9531, 0.7503, 0.7854, 0.8802}.

[0297] Next, the computer generates vector {B} ~ The numerical distance between any possible pair of s (i.e., S * The distances for (S-1) items were calculated.

[0298] For numerical distances, it is convenient to apply either the Hamming distance or the (simplified) Euclidean distance.

[0299] By definition, the Hamming distance / H / between a pair of vectors is 1.0 if all vector elements are different, 0.75 if one vector element is equal, 0.5 if two elements are equal, 0.25 if three elements are equal, or 0.0 if all vector elements are equal. Here, we performed a simulation using the example N=4.

[0300] By definition, the Euclidean distance / D / between (simplified) vector pairs is calculated as the sum of the differences between the vector elements. For example, the differences (positive as quantities) are {0.0012, 0.1263, 0.1078, 0.0899}, and the sum / D / = 0.3253.

[0301] In the next step, a histogram was generated. The number of pairs was PAIR = 5.368 × 10⁻¹⁰.8 (That is, approximately 500 million pairs). The number of occurrences for / H / distance can be obtained, for example, as follows: If 0 occurrences occur in the pair, / H / = 0.00. In the case of 0 occurrences in the pair, similarly, / H / =0.25. Out of the pairs, approximately 300 occurrences (6 in 1 million) resulted in / H / = 0.50. Out of the pairs, 850,000 occurrences (0.15%) resulted in / H / = 0.75. The occurrence rate was almost at PAIR (99.85%), with / H / = 1.00.

[0302] All S * A further histogram for / D / is created, normalized for the highest of the (S-1) distances / D / , showing a Gaussian peak at approximately 0.2 and (normalized) at only 0.1 percent of occurrences.

[0303]

number

[0304] Such or similar simulations are performed at design time 702 (see Figure 6), and for example, a reference filter set 125 can be selected in step 721.

[0305] evaluation The uniqueness condition requires proof that something is missing (i.e., an equal pair of {T}k and {T}q), so the applicant conducted numerical experiments using multiple simulations (such as filter transmission characteristics) and concluded that the possibility of having two equal pairs is negligible for actual implementations.

[0306] Transition to decryption Now, the explanation moves from encoding to decoding. As mentioned above, the components for the decoder function 106 can be located within the data consumer 103.

[0307] Figure 19 shows a decoder function 106 that implements the decoding method 600. Simplified decoding can be implemented as a computer implementation method 600, which includes the following: (Step 610) to receive a specific spectral distribution identifier {D} generated by performing encoding method 400, ●Map this specific spectral distribution identifier {D} to a specific distribution [E] (step 620).

[0308] The intensity vector {D} is the distribution identifier 250 / 270 for distribution 310, and a particular distribution [E] identifies distribution 390 (see Figure 1). The explanation uses [E] (i.e., an amplitude vector with K amplitudes), but other identifiers such as names, strings, or numbers in a table that are pre-assigned to a particular distribution 390 can also be used. There is no need to recalculate the amplitude in [E].

[0309] Since spectral distribution 310 belongs to set 315 (see Figure 1), an ideal decoder will re-establish the same set member and thus identify one of the P distributions 390. (The concentration P of set 315 is the concentration of possible matches 390-1, 390-p, and 390-P.)

[0310] In many situations, [E] belongs to a set of known spectral distributions, such as previously encoded distributions. In other words, the (encoded) source data is replaced by the target data, or, from the perspective of the use case, information about the original spectral distribution is re-established.

[0311] In the example in Figure 19, the vector {D}={12 23 16} yields distribution 390-1 (the same as 310 in Figure 1).

[0312] option Mapping has various implementation options (i.e., mapping options), including the following: ●(Reference Table) A computer can access data representing a given relationship between one or more {D} and one or more [E]. This relationship can be one-to-one (e.g., one {D} to one [E]), one-to-many, or many-to-one. This relationship data can be provided by a database or similar (i.e., a reference table, library). ●(Recursive Fit) The computer can simulate the operation in the provider (such as the filtering and measurement steps in Method 400), and as a result, the computer can estimate the approximate distribution [A] of the received light. As described above (see the explanation of design time 702 in Figure 6 and the numerical experiment in Figure 18), the simulation can yield multiple code vectors such as {B} or {D}. Since each of the code vectors is unique among its multiple, each particular vector is a specific [A] that is simulated. ~ It can be associated with that. ●(Neural Network) A computer can use a pre-trained neural network. The network can classify a given {D} into a predefined distribution [E]. An embodiment of such a network is illustrated with Figure 20.

[0313] In principle, the decoder function 106 can implement any option individually or in combination. The selection of options depends on constraints that have several aspects.

[0314] In the first aspect, the explanation continues by discussing constraints and examining numbers (and indices). ●As explained using Figure 1, there are P different distributions [A]. ●There are P different distributions [E].

[0315] In the use case scenario, the set concentration P does not need to be saved. The set concentration P relates to the use of computational resources, memory consumption, processing time, etc. In some cases, the decoder may introduce some degree of rounding.

[0316] Figure 20 shows the pre-trained neural network 106-NETWORK in the decoder function 106 (see Figure 19). Those skilled in the art can modify the implementation, but the following generally applies to such decoders. ● The decoder network has a first layer (circular symbol to the left of the node) that receives spectral distribution identifiers 250 / 270, i.e., vector {D}. As already explained, {D} has N vector elements (corresponding to the number of filters). Therefore, the first layer has at least N nodes. ● The decoder network has one or more intermediate layers connected to the first layer. ● The decoder network has an output layer with P nodes (or fewer nodes to reduce accuracy). The diagram shows different distributions, and the network acts as a classifier, which receives {D} and classifies it into a specific distribution 390-p.

[0317] Note that the network is part of the chain described above (from provider 101 to consumer 103, see Figure 1). The number of P nodes in the output is determined by the coding parameters 351, 352, 353, and 354.

[0318] Network training Figure 21 shows several ways in which the neural network 106-NETWORK can be trained. ~ This indicates that the component will be simulated.

[0319] The (random) spectrum generator generates multiple simulated spectral distributions [A] ~This provides the following: Instead of a random spectrum, it is also possible to simulate set 315 (see Figure 1 with concentration P). [A] ~ It can also be considered as a predetermined programmable spectral distribution, [A] ~ This becomes use-case specific according to encoding parameters 351, 352, 353, and 354 (see Figure 1).

[0320] Filter 125-n ~ and sensor 130 ~ However, it is simulated (for N, as in the actual implementation). Using the above calculation (multiply the amplitude value by the transmission value and then integrate; see Figure 15), the vector {B} ~ and {D} ~ However, this is calculated. In many situations, no fluctuations (filter fluctuations, light intensity fluctuations) occur (the simulated filter is the reference filter), so {B} ~ is {D} ~ It is the same as, and therefore, the calculation functions 140 / 150 / 160 / 170 do not need to be simulated.

[0321] {D} ~ Next, proceed to the first layer of network 106-NETWORK, [A] ~ This proceeds to the final layer of network 106-NETWORK (as ground truth).

[0322] Alternatively, network 106-NETWORK can be trained with non-simulated data such as {B} and {D}.

[0323] Simulation and network training methods While the basic concepts of implementing decoder 106 using a neural network have been explained, the explanation will now describe some further aspects, as it involves using simulations for both filter design (see design time in Figure 6) and network training (see Figure 20).

[0324] The simulation can approximate ideal encoders and decoders, and the constraint considering the maximum number (such as concentration P) is merely a constraint on the number of calculations. However, this is not an implementation constraint, as the computer performing the simulation is usually not the same computer involved in the encoding or decoding.

[0325] (All source distribution simulations). The simulation computer performs [A] for virtually all P members in set 315. ~ This can be provided. Encoding parameters can be considered when required by the use case.

[0326] (Random distribution simulation). The simulation computer randomly generates [A] ~ This can provide a set of random distributions (i.e., fewer than P simulated spectra).

[0327] (Noise distribution simulation). The simulation computer can process [A] (e.g., obtained by measurements by data provider 101) and add random fluctuations (i.e., "noise"). This technique can make the network more robust to small fluctuations.

[0328] (Measurement-based filter simulation). The transmission function [{T}] of a specific filter set 120 can be measured, but it can also be simulated.

[0329] (Measurement-based filter simulation). The transmission function [{T}] of a particular filter set 120 can be simulated by sampling measured transmission data, in the sense that a hybrid exists between measurement and simulation.

[0330] Implementation of calibration for decoders It appears possible to train the neural network 106-NETWORK separately to process the spectral distribution identifiers 250 / 270 resulting from different individual filters. In other words, the mapping steps 440 / 470 can be skipped, as can the decoder function 106.

[0331] Identifying changes Figure 22 shows a time diagram of Method 500 for identifying changes in a particular spectral distribution by sequentially performing the first step sequence 401-α and the second step sequence 401-β of Method 400.

[0332] In some use cases, it may suffice to detect changes in a distribution rather than encoding it. In other words, the focus can be on investigating the relationships between distributions.

[0333] Method 401 is a sub-method of Method 400 and provides a provisional intensity value Bn in vector {B} (see Figure 5). Several constraints (such as filter variations and intensity variations) can be ignored to identify changes in the distribution over time. A simplified measurement device can still detect changes, even if it is not calibrated. Furthermore, devices specialized for measuring optical spectra are not highly sensitive to different light intensities. They can still detect changes.

[0334] This assumption applies to many situations in which coding parameters 351, 352, 353, and 354 are defined, but it may not apply to changes that proceed beyond the parameters.

[0335] Therefore, the computer functions for compensating for fluctuations (see 140 / 150 / 160 / 170 in Figure 3) do not necessarily need to be applied. They can be used at will.

[0336] Method 500 is a method for identifying a change in a specific spectral distribution of light 210 received from a physical object 110 (from distribution 310-α to distribution 310-β) (see Figure 1). As described above, the received light 210 has a wavelength-specific intensity [A] within the wavelength range 351 from the minimum wavelength λ_min to the maximum wavelength λ_max. The intensity [A] can vary with respect to a certain wavelength k.

[0337] The first step sequence 401-α begins at the first time point tα. As already described, the sequence begins 420-n by individually filtering the received light 210 using each filter by a filter set 120 having at least N=3 filters 120-1, 120-n, and 120-N, and continues 430-n by measuring the intensity of the filtered light 220 and obtaining a first vector {B}α, e.g., {B}α={13,23,16}, which has filter-specific intensity values ​​Bnα. (The values ​​do not need to be called "provisional" values.)

[0338] Filter set 120 satisfies the uniqueness condition described above.

[0339] The second step sequence 401-β begins at a second time point tβ, which is after the completion of the first step sequence 401-α (here, tα_end). By again filtering the light 210 received by the same set of filters 120-1, 120-n, and 120-N individually (420-n) and again measuring the intensity of the filtered light 220 (430-n), the second sequence yields a second vector {B}β, e.g., {B}β={66,32,61}, which has filter-specific intensity values ​​Bnβ.

[0340] The waiting time between tα_end and tβ depends on the use case.

[0341] Method 500 continues by comparing a first vector {B}α and a second vector {B}β according to a predetermined comparison rule, and by identifying a change when both vectors are distinguishable from each other.

[0342] Those skilled in the art know how to compare vectors, and the Hamming distance and Euclidean distance described above are merely examples of appropriate tools. This rule should be use-case specific. Use-case specific acceptable rules can be applied at will. For example, a shift from {13,23,16} to {14,23,15} crosses a non-zero distance but may not be counted as a “change”.

[0343] If a change in light intensity is expected (between tα and tβ), such a change may lead to misidentification. As described above, method 400 takes L_DATA into account in the step following sequence 401.

[0344] Therefore, it is possible to apply an arithmetic function that has at least a divider function (see Figure 5, division 450 / 460). When applied to Method 500, this means dividing the elements of the first vector {B}α individually by the first intensity criterion value L_DATA_α and dividing the elements of the second vector {B}β by the second intensity criterion value L_DATA_β to obtain the normalized first and second vectors. A comparison can then be performed using the normalized first and second vectors. Options for obtaining L_DATA are illustrated with respect to Figure 4 by calculation or measurement (step 431).

[0345] Consideration Conventional methods use subrange-specific filters. For example, a digital camera uses three subrange filters (i.e., RGB filters), and the corresponding spectral distribution can be described as a vector with only three elements. Such methods are optimized for data consumers with color displays or color printers. Human observers (viewing the displayed or printed image) will be satisfied, and the spectral accuracy will be sufficient for the purpose.

[0346] However, changes in the spectral distribution convey information of potential interest. Using filters that transmit light across the entire range 351 seems advantageous because each filter transmits light across the entire range. Variations in light, and even relatively small variations within subranges, will alter the transmission (and therefore the numerical value Bn in the sensor output).

[0347] The following example helps illustrate this. System 100 can support medical professionals examining patients. It may be necessary to distinguish specific areas of the body; some areas of skin may differ from others, and internal organs may exhibit subtle differences in color. A camera with only RGB may not be able to provide LIGHT_DATA that represents such subtle differences (in the spectral distribution). This is an example of insufficient spectral resolution. Medical professionals may not want to use spectrometers, etc. (which may be too large or even dangerous to the patient). However, they can use image acquisition devices that employ alternative techniques.

[0348] Using filters has the added advantage of not requiring a complete hardware change. A camera with RGB filters has roughly the same physical dimensions as a camera with filter set 120, and the additional components (operating functions 140 / 150 / 160 / 170, and decoders, etc.) do not substantially increase the computational effort (and do not add size to the system).

[0349] Fabry-Perot (FB) filter Filter arrays can be constructed by implementing filters using Fabry-Perot filters. For example, hyperspectral cameras (marketed as "xiSpec") are commercially available from XIMEA GmbH (48155 Muenster, Germany). In these cameras, the hyperspectral filters are implemented as Fabry-Perot interference filters. These filters are added at the wafer level on top of the sensor's pixel structure.

[0350] FP filters have relatively high transmission (approximately 15-30%) in a relatively narrow bandwidth, but relatively low transmission in other bandwidths. The average transmission across the minimum to maximum range rarely exceeds 50%.

[0351] In contrast, using the filters described above (i.e., filters 120-n in Figures 2-3, the filter at layer 825 in Figure 10, and others) can exhibit a minimum transmission of 50% for any wavelength (within the range 351) (see Figure 14, for example, for threshold 340). Having an average of 50%, or having higher transmission (as described above), does not conflict with the occasional requirement for relatively low transmission at some specific single wavelength.

[0352] Since the sensor integrates over a range of 351 (see Figure 15 for details), more available light contributes to the provisional intensity value Bn. As a result, the filtering method described above introduces less noise, and therefore the accuracy of Bn is less affected by noise.

[0353] By making relatively more light available (compared to FP filters), the sensor collects light more efficiently. This can be advantageous, for example, because it allows for shorter measurement time intervals (i.e., the time it takes to perform the measurement steps). In applications where light needs to be encoded for consecutive images (i.e., within a frame), this can lead to higher frame rates.

[0354] Alternative filter conditions As mentioned above, filters meet the uniqueness requirement, but are constrained by the fact that all filters inherently absorb light. Light (see 210 in Figure 3) may only be available at relatively low intensity. Simplified, lower light intensities require higher filter transmission, and higher intensities allow for lower filter transmission.

[0355] Filters with relatively high transmittance (even filters that are semi-transparent over relatively large lambdas) may lack uniqueness. This constraint can be overcome by considering further conditions, namely efficiency conditions.

[0356] Figure 23 shows a filtering method with hardware and software details that have efficiency requirements for the filter.

[0357] Figure 23 is based on Figure 3. It is also shown that sensor 130 (in Figure 3) can be implemented by multiple physically separate sensors 130-1, 130-n, and 130-N within sensor set 130-SET. Such arrangements have been described in detail above for device 800 (see Figures 8 and 10) and for method steps performed in parallel (see step 430-n measurement).

[0358] However, the constraints associated with filter efficiency are the same whether sensor 130 is implemented as a single physical sensor (see the filter wheel method in Figure 9) or using multiple physical sensors.

[0359] In Figure 3, the threshold 340 (or "T_threshold") symbolizes when the transmitted Tk (as T1k, Tnk, TNk) exceeds a certain threshold 340 (with some exceptions, see Figures 11-14). However, Figure 23 shows an alternative form with a different threshold 350 (see T_av_th in Figure 23). When the filter fits T_av_th, the vector {B} can still be obtained for relatively bright intensities.

[0360] Figure 24 shows the transparency functions T1, T2, T3, and T4 for four filters (N=4) with a threshold condition T_av_th, 350 (see Figure 3). The number N=4 was chosen simply for explanatory purposes, and the condition applies similarly to N=3 and N>4.

[0361] Since the sensor 130 measures the intensity of filtered light 220 (see Figures 3 and 23) over the wavelength range 351 (see Figure 1) (see step 430-n in Figure 6) to obtain a provisional intensity value Bn (see Figures 3 and 23), light absorption within the filter affects the detection efficiency (of the sensor).

[0362] As described above, in some use case scenarios, the received light 210 may have a relatively low brightness (or intensity), and the filtered light 220-n will have an even lower brightness. The sensitivity of the sensor 130 may not be able to adapt to such insufficient light conditions. Therefore, the filter should absorb as little light as possible. Here, we continue to discuss useful efficiency conditions for the filter with reference to Figure 6 for design time 701, which involves selecting a (reference) filter in step 721.

[0363] As already explained, each filter 120-n has its own specific transmission function Tn. The transmission function Tn is defined according to the parameters spectral resolution 352 and wavelength range 351, and as a result, Tnk is the transmission of filter n at wavelength k.

[0364] As shown in Figure 24, it is possible to estimate the filter-specific (intrinsic to n) average transmission function T_av(n). To reach this average, there are at least the following options: ●The wavelengths λ_min and λ_max (see parameter 351 in Figure 1) can be normalized to λ_min_norm=0 and λ_max_norm=1. The integral of Tn(λ) (over minimum / maximum) is the filter-specific average transmission T_av(n). (For example, for a perfectly semi-transparent filter with T=1 for all λ, the integral is:

[0365]

number

[0366]

number

[0367] Both calculation options are applicable, and the second option may be more suitable for computer-based activities during design and calibration (see items 701 and 702 in Figure 6).

[0368] In the example in Figure 24, the average transmission rate of the first filter, T_av(1), is approximately 90%, and the transmission rate of the second filter, T_av(2), is approximately 80%, and so on.

[0369] The average T_av is calculated by summing T_av(n) for n=1 to N and dividing by N. T_av can be called the "N-filter average" or "all-filter average".

[0370] In the example in Figure 24, the average T_av is approximately 80%.

[0371] The filter selection criteria are defined by the threshold mean T_av_th (see item 350 in Figure 3). The threshold mean T_av_th is 60%.

[0372] Filters can have a cutoff range (between specific wavelengths) in which transmission is substantially zero (see Figure 13 for example). A filter set having some filters with cutoff ranges can still meet efficiency requirements.

[0373] The integral of Tn(λ) (over the minimum / maximum) has a first non-zero component (from λ_min to the cutoff range, or between the cutoff ranges) and a second non-zero component (from after the cutoff range to λ_max, or between the cutoff ranges). The filter-specific average transmittance T_av(n) may be below or above the threshold T_av_th(350). This is not a problem. The threshold is averaged over N filters and not applied to individual filters. In other words, the cutoff range in some filters can be compensated for by other filters.

[0374] For a specific subrange (of range 351), a particular filter may block light (e.g., Tn=0) or be transparent (Tn=1). In the case of blocking, transmission is zero, but non-blocking filters still contribute to uniqueness. Efficiency conditions can still be met.

[0375] While the efficiency requirements for the filter influence the filter selection, all other explanations above remain applicable (for example, there are no changes to the general principles and methods 400 in Figure 13, nor to the simulation, any decoding function, etc.).

[0376] (Outline for implementations with efficiency requirements) A method for encoding a specific spectral distribution of light into a spectral distribution identifier is disclosed. Light has wavelength-specific intensity in a specific spectral distribution within a wavelength range from the minimum wavelength to the maximum wavelength, which has wavelength resolution.

[0377] The step of receiving light is followed by filtering the received light. The light is filtered individually using each filter in a filter set of at least N=3 filters. Each filter in the filter set has a filter-specific transmission function that describes the wavelength-specific transmission of the filter.

[0378] The filter set satisfies the following conditions (uniqueness and efficiency conditions): <1> In a combined transmission function for a filter set, the transmission vector, which is the connection of transmissions, is unique for any wavelength and unique for each spectral distribution of light within the wavelength range. <2> The filter-specific transmission function satisfies the efficiency condition that the average of the filter-specific average transmissions from the minimum wavelength to the maximum wavelength exceeds a 60% threshold.

[0379] This method continues by individually measuring the intensity of the filtered light across the wavelength range for each filter to obtain provisional intensity values. These values ​​are elements of the provisional intensity vector.

[0380] This method is computer-implemented in at least several steps. By computing an operational function, the computer provides a spectral distribution identifier that encodes a specific spectral distribution of the received light. This operational function absorbs light variations using an intensity reference value and absorbs filter variations using predetermined calibration data.

[0381] Providing spectral distribution identifiers is performed by an operational function implemented according to either the first or second alternative form.

[0382] The alternative forms simply distinguish absorption by their temporal sequence. However, since it is inappropriate to cover both alternative forms in a single claim, the alternative forms are presented in two separate claims.

[0383] In the first alternative form, the first calculation function maps provisional intensity values ​​to calibrated intensity values, which are elements of a calibrated intensity vector, according to predetermined calibration data. The second calculation function calculates normalized calibrated intensity values, which are elements of a normalized calibrated intensity vector, by individually dividing the calibrated intensity values ​​of the calibrated intensity vector by the intensity reference value. Thus, the spectral distribution identifier is provided as a normalized calibrated intensity vector.

[0384] In the second alternative form, the third calculation function calculates normalized intensity values, which are elements of a normalized intensity vector, by individually dividing the provisional intensity values ​​by the intensity reference values. The fourth calculation function maps the normalized intensity values ​​to calibrated normalized intensity values, which are elements of a calibrated normalized intensity vector, according to predetermined calibration data. Thus, the spectral distribution identifier is provided as a calibrated normalized intensity vector.

[0385] Optionally, filtering of the received light is performed by a set of filters whose conditions are checked for suitability through simulation, based on coding parameters that define each spectral distribution.

[0386] Optionally, the intensity reference value is obtained by measuring the intensity of the light received by the sensor. In the first method, the received light proceeds directly to the sensor. In the second method, the received light proceeds indirectly to the sensor via a neutral photoconductor, and the sensor measures the intensity of the received light, correcting for the loss introduced by the neutral photoconductor.

[0387] Optionally, filtering the received light with a set of filters is performed by a combination of multiple filters and sensors positioned in a plane perpendicular to the direction of the received light. Measuring the intensity of the received light is performed by directing a portion of the received light directly to at least one additional sensor, thereby bypassing the filtering of the filter set.

[0388] In both alternative configurations, the intensity reference value is obtained by measuring the intensity of the received light, and filtering the received light with a set of filters, and measuring the intensity of the filtered light, is performed in an arrangement where the filters of the filter set are combined with corresponding sensors. These filter-sensor combinations are positioned in a plane perpendicular to the direction of the received light. Measuring the intensity of the received light and obtaining the intensity reference value is performed by a conductor-sensor combination positioned in the same plane. The filter-sensor combinations and the conductor-sensor combinations form a mosaic pattern in the plane.

[0389] Optionally, prior to the mapping step, a step is taken to determine calibration data by one of the following: (i) measuring at least partially the individual transmission functions of the filters to determine the filter-specific deviation type relative to the reference filter, which is amplitude deviation, repeating pattern deviation, or wavelength offset deviation, and deriving calibration data from a deviation type-specific reference table; (ii) training a neural network with simulated data of a predefined programmable spectral distribution and simulated filtering by a simulated intensity vector and a simulated reference filter as ground truth, wherein the weights in the network serve as calibration data for performing the mapping step, which in the first alternative form transfers provisional intensity values ​​to a neural network that outputs calibrated intensity values, or in the second alternative form transfers normalized intensity values ​​to an auxiliary neural network that outputs calibrated normalized intensity values; and (iii) training a neural network with data obtained by measurement (i.e., data from a calibrated instrument).

[0390] Furthermore, a computer implementation method for decoding spectral distribution identifiers (obtained by performing an encoding method) is disclosed. The decoding method includes the steps of: receiving a spectral distribution identifier; and mapping the spectral distribution identifier to a specific spectral distribution. The mapping is performed by one of the following: (i) accessing a library representing a predefined relationship between a specific spectral distribution identifier and a specific spectral distribution; (ii) simulating the encoding of several known input distributions into corresponding spectral distribution identifiers, such that the corresponding simulated distribution identifiers identify a specific spectral distribution as a specific simulated input distribution to fit the received spectral distribution identifier; and (iii) processing the received spectral distribution identifier with a pre-trained neural network that classifies the received spectral distribution identifier into one of the predefined distributions.

[0391] A computer program product, when loaded into the computer's memory and executed by at least one of the computer's processors, causes the computer to perform steps of a computer-implemented encoding and decoding method.

[0392] When efficiency conditions are applied to the device (see FIGS. 8-9), the device can be adapted to encode a specific spectral distribution of light. The device can include a filter layer attached to the sensor layer. The filter layer and the sensor layer can be planar layers. The filter layer can be adapted to receive light on its surface and transmit the received light to the sensor layer. The filter layer and the sensor layer can be divided into separate pixel positions corresponding to filter positions and corresponding to the sensor. The sensor can be adapted to quantify the intensity of the transmitted light by integrating the intensity of the transmitted light during a measurement time interval and provide a sensor-specific provisional intensity value representing the transferred light. For a continuous combination of M pixel positions, referred to hereinafter as areas, when N<M, the following can be applied. The N filter positions hold a filter set of N≧3 filters above the corresponding sensor such that the corresponding sensor provides a sensor-specific intensity value as a provisional intensity value. The M-N filter positions are either empty filters or holding filters having a wavelength-nonspecific transmission function, and thus the corresponding sensors provide a sensor-specific intensity value as a luminance value. Each of the N filters in the filter set has a filter-specific transmission function that describes the wavelength-specific transmission of the filter.

[0393] The filter set can meet both conditions (1) and (2), namely uniqueness and efficiency.

[0394] The device further includes a calculation module that processes the provisional intensity value into a normalized intensity value that is an element of a normalized intensity vector corresponding to a spectral distribution identifier encoding the specific spectral distribution of the received light.

[0395] Optionally, this calculation module is implemented such that a mapper is adapted to map the provisional intensity value to a calibrated intensity value that is an element of a calibrated intensity vector using predetermined device-specific calibration data, and a divider is adapted to individually divide the calibrated intensity values of the calibrated intensity vector by the luminance value.

[0396] A computer program product, when loaded into the computer's memory and executed by at least one of the computer's processors, causes the computer to perform the step of computing a function by providing a spectral distribution identifier. The program product is implemented within the device's computing module.

[0397] A computer implementation method for decoding spectral distribution identifiers is disclosed. Identifiers are obtained in advance by performing a coding method. The coding method includes the following steps: receiving spectral distribution identifiers and mapping spectral distribution identifiers to specific spectral distributions. The mapping is performed by one of the following: (i) accessing a library representing a predefined relationship between specific spectral distribution identifiers and specific spectral distributions; (ii) simulating the coding of multiple known input distributions to multiple corresponding spectral distribution identifiers, such that the corresponding simulated distribution identifiers identify specific spectral distributions as specific simulated input distributions to fit the received spectral distribution identifiers; and (iii) processing the received spectral distribution identifiers by a pre-trained neural network that classifies the received spectral distribution identifiers into predefined distributions.

[0398] General-purpose computer Figure 25 shows examples of general-purpose computer devices that can be used with the techniques described herein. Figure 25 shows examples of general-purpose computer devices 900 and general-purpose mobile computer devices 950 that can be used with the techniques described herein. Computing device 900 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. General-purpose computer device 900 may correspond to computers shown by other figures. Computing device 950 is also intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, driver assistance systems, or in-vehicle computers and other similar computing devices. For example, computing device 950 may be used by a user as a front-end to interact with computing device 900. The components shown herein, their connections and relationships, and their functions are illustrative and do not limit the implementation of the invention described and / or claimed herein.

[0399] The computing device 900 includes a processor 902, memory 904, a storage device 906, a high-speed interface 908 connected to memory 904 and a high-speed expansion port 910, and a low-speed bus 914 and a low-speed interface 912 connected to storage device 906. Each of components 902, 904, 906, 908, 910, and 912 may be interconnected using various buses and mounted on a common motherboard or in other ways as needed. The processor 902 can process instructions to be executed within the computing device 900, including instructions stored in memory 904 or on storage device 906 to display graphical information for a GUI on an external input / output device such as a display 916 coupled to the high-speed interface 908. In other implementations, multiple processors and / or multiple buses may be used, along with multiple memories and multiple types of memory as needed. Also, multiple computing devices 900 may be connected, each device providing some of the necessary operations (e.g., as a server bank, a group of blade servers, or a multiprocessor system).

[0400] Memory 904 stores information within the computing device 900. In one implementation, memory 904 is one or more volatile memory units. In another implementation, memory 904 is one or more non-volatile memory units. Memory 904 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0401] The storage device 906 can provide large-capacity storage to the computing device 900. In one implementation, the storage device 906 may be or include a computer-readable medium such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory or other similar solid-state memory device, or an array of devices including devices in a storage area network or other configuration. The computer program product can be tangibly embodied on the information carrier. The computer program product may also include instructions that, when executed, cause the computer to perform one or more methods, such as those described above. The information carrier is a computer-readable or machine-readable medium such as memory 904, the storage device 906, or memory on the processor 902.

[0402] The high-speed controller 908 manages bandwidth-intensive operations for the computing device 900, while the low-speed controller 912 manages lower bandwidth-intensive operations. Such function assignments are merely illustrative. In one implementation, the high-speed controller 908 is coupled to memory 904, a display 916 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 910 that can accept various expansion cards (not shown). In this implementation, the low-speed controller 912 is coupled to the storage device 906 and the low-speed expansion port 914. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet), may be coupled to one or more input / output devices such as a keyboard, pointing device, scanner, or networking devices such as a switch or router, for example, via a network adapter.

[0403] The computing device 900 may be implemented in several different forms, as shown in the figure. For example, it may be implemented as a standard server 920, or multiple times in a group of such servers. It may also be implemented as part of a rack server system 924. Furthermore, it may be implemented in a personal computer such as a laptop computer 922. Alternatively, components from the computing device 900 may be combined with other components in a mobile device (not shown), such as device 950. Each such device may contain one or more computing devices 900, 950, and the entire system may consist of multiple computing devices 900, 950 communicating with each other.

[0404] The computing device 950 includes, among other components, input / output devices such as a processor 952, memory 964, and display 954, a communication interface 966, and a transceiver 968. Device 950 may also include storage devices such as a microdrive or other devices to provide additional storage. Each of the components 950, 952, 964, 954, 966, and 968 are interconnected using various buses, and some of the components may be mounted on a common motherboard or in other ways as needed.

[0405] The processor 952 can execute instructions within the computing device 950, including instructions stored in memory 964. The processor may be implemented as a chipset of chips, including several separate analog and digital processors. The processor may also coordinate other components of the device 950, such as the user interface, applications run by the device 950, and control of wireless communication by the device 950.

[0406] The processor 952 may communicate with the user via a control interface 958 and a display interface 956 coupled to the display 954. The display 954 may be, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, or other suitable display technology. The display interface 956 may include suitable circuitry for driving the display 954 to present visual and other information to the user. The control interface 958 may receive commands from the user and convert them for transmission to the processor 952. In addition, an external interface 962 may be provided in a manner that communicates with the processor 952 in order to enable short-range communication of device 950 with other devices. The external interface 962 may be provided by wired communication in some implementations, by wireless communication in other implementations, or multiple interfaces may be used.

[0407] Memory 964 stores information within the computing device 950. Memory 964 can be implemented as one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Extended memory 984 may also be provided and connected to device 950 via an extended interface 982, which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such extended memory 984 may provide additional storage space for device 950, or it may also store applications or other information for device 950. In particular, extended memory 984 may include instructions for executing or supplementing the processes described above, and may also include secure information. Therefore, for example, extended memory 984 may function as a security module for device 950 and may be programmed with instructions that enable the secure use of device 950. Furthermore, secure applications may be provided via a SIMM card, along with additional information, such as placing identification information on the SIMM card in a hack-proof manner.

[0408] The memory may include, for example, flash memory and / or NVRAM memory, as described later. In one implementation, the computer program product is tangibly embodied in an information carrier. When executed, the computer program product includes instructions that perform one or more of the methods described above. The information carrier is a computer-readable or machine-readable medium such as memory 964, extended memory 984, or memory on the processor 952, which may be received, for example, via a transceiver 968 or an external interface 962.

[0409] Device 950 may, if necessary, communicate wirelessly via a communication interface 966, which may include digital signal processing circuitry. The communication interface 966 may provide communication under various modes or protocols, including, among others, GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. Such communication may be conducted, for example, via a radio frequency transceiver 968. In addition, short-range communication may be conducted using Bluetooth, WiFi, or other such transceivers (not shown). Furthermore, a GPS (Global Positioning System) receiver module 980 may provide device 950 with additional navigation and location-related radio data, which may be used as appropriate by applications running on device 950.

[0410] Device 950 may also communicate audibly using an audio codec 960 that can receive information spoken by the user and convert it into usable digital information. The audio codec 960 may also generate audible sounds for the user, for example, through a speaker in the handset of device 950. Such sounds may include sounds from voice calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications running on device 950.

[0411] The computing device 950 may be implemented in several different forms, as shown in the figure. For example, it may be implemented as a mobile phone 980. It may also be implemented as part of a smartphone 982, a personal digital assistant, or other similar mobile device.

[0412] Various implementations of the systems and techniques described herein can be realized in digital electronic circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to them.

[0413] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and / or device (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including machine-readable medium that receives machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0414] To provide user interaction, the systems and techniques described herein can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (e.g., a mouse or trackball) on which the user can provide input to the computer. User interaction can be provided using other types of devices. For example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and the input from the user can be received in any form, including acoustic, voice, or tactile input.

[0415] The systems and techniques described herein can be implemented in computing devices that include backend components (e.g., as data servers), middleware components (e.g., application servers), or frontend components (e.g., client computers having a graphical user interface or web browser that allows users to interact with implementations of the systems and techniques described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., communication networks). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the Internet.

[0416] Computing devices can include clients and servers. Clients and servers are generally geographically separated from each other and typically interact via a communication network. The relationship between a client and a server arises from computer programs running on each computer that have a client-server relationship with each other.

[0417] Several embodiments have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the present invention.

[0418] In addition, the logical flow shown in the figure does not require a specific order or sequence to achieve the desired result. Furthermore, other steps may be provided, or steps may be excluded from the described flow, or other components may be added to or removed from the described system. Therefore, other embodiments are within the scope of the following claims.

[0419] Reference symbol 100 Data Processing Systems 110 Physical objects 101 Data Providers 102 Data Link 103 Data Consumer 104 Optical Splitter 105 Encoder function 106 Decoder function 120 filter set 120-n specific filter 121A, 121B Photoconductors 125 Reference Filter Set 125-n Specific Criteria Filter 128, 129 Moving filter array (BLOCK, WHEEL) 130 sensors 131A, 131B sensors 130-SET Sensor Set 140 / 170 Mappa 150 / 160 divider 210 light 220-n filtered light 230-n, Bn provisional intensity values 250 / 270 spectral distribution identifiers 310 Specific spectral distribution (of received light) 315 Overall set of spectral distributions 320 Filtered light distribution 321, 322 line 340, 350 thresholds 351 wavelength range 352 spectral resolution 353 Amplitude range 354 Amplitude accuracy 390 Re-established specific spectral distribution 400 Method (encoding) 401 Step sequence applied in Method 500 402 Provide spectral distribution identifiers 410 Receiving light 420-n Filtering the received light. 430-n: Measuring the intensity of filtered light. 431. Measuring the intensity of unfiltered light. Mapping 440-n / 470-n Perform the division 450-n / 470-n. 500 Methods for identifying changes in spectral distribution Methods for decrypting 600 610 Receiving an identifier 620 Mapping 701 Design-time activity sequence 702 Activity Sequence During Calibration Receive a use case specification with 711 encoding parameters. Provides 712 filter sets 721 Select a reference filter set 722 Compare the filter to the reference filter. Verify the manufacturing of 731. 732 Obtain calibration data 800 devices 820 Filter 823 Filter position 825 filter layers 835 Sensor layer 830 Sensor 850 pixel position 840 / 870 Mappa 850 / 860 Divider 880 Area 900, 9xx general-purpose computers and components [] Notation for representing a vector that usually has K elements {} Notation used to indicate a vector (or matrix) that typically has N elements. / / Notation for showing multiple vectors ~~ Notation for showing multiple vectors when simulated ~ Showing the simulated data A, Ak filter, wavelength-specific light intensity n, N: Filter index for identifying individual filters, number of filters. k, q are indices used to identify specific wavelengths. K is the number of individual wavelengths. L_DATA Strength Reference Value P: Concentration of the set for all distributions Concentration of the S subset Tnk transparent Transmitted average specific to the T_av(n) filter T_av Transmittance average across all N filters t time point {DEV} Filter-specific deviation {CAL} Filter-specific calibration data λ wavelength Δλ spectral resolution α, β Continuous step sequence 401 and time point in method 500

Claims

1. A method (400) for encoding a specific spectral distribution (310) of light (210) into a spectral distribution identifier (250), wherein the light (210) has wavelength-specific intensity (Ak, [A]) in the specific spectral distribution (310) within a wavelength range (351) from a minimum wavelength (λ_min) to a maximum wavelength (λ_max) having a wavelength resolution (352, Δλ), and the method (400) Receiving the aforementioned light (210) (410), The received light (210) is filtered individually (420-n) using each filter (120-1, 120-n, 120-N) of the filter set (120) (120-1, 120-n, 120-N), wherein the filter set (120) has at least N=3 filters (120-1, 120-n, 120-N), The aforementioned filter set (120) has the following conditions: (1) Each filter (120-n) in the filter set (120) has a filter-specific transmission function (Tn) that describes the wavelength-specific (k) transmission (Tnk) of the filter (120-n), (2) In the combined transmission function ({T}) for the filter set (120), the transmission vector ({T}k), which is the concatenation ({T}k) of the transmissions (TnK), is unique for any wavelength (k) and unique for each spectral distribution (315) of light (215) within the wavelength range (351). (3) Filtering that conforms to the condition that each filter (120-n) in the filter set (120) is a single-state filter, For each of the N filters (120-n), the intensity of the filtered light (220) over the wavelength range (351) is measured individually (430-n) to obtain provisional intensity values ​​(Bn), which are the N elements of the provisional intensity vector ({B}, 230-n). The present invention provides (402) a spectral distribution identifier (250) having N elements, which encodes the specific spectral distribution (310) of the received light (210) using an operation function (140, 150) which absorbs light fluctuations using an intensity reference value (L_DATA) and absorbs filter fluctuations using predetermined calibration data ({CAL}), Method (400) wherein providing the spectral distribution identifier (250) (402) is performed by an operation function (440, 450), the first operation function (140) maps the provisional intensity value (Bn) to the calibrated intensity value (Cn), which is the element of the calibrated intensity vector ({C}), according to the predetermined calibration data ({CAL}) (440-n), the second operation function (150) divides the calibrated intensity value (Cn) of the calibrated intensity vector ({C}) individually by the intensity reference value (L_DATA) (450-n), and calculates the normalized calibrated intensity value (Dn), which is the element of the normalized calibrated intensity vector ({D}), such that the spectral distribution identifier (250) is provided as a normalized calibrated intensity vector ({D}).

2. A method (400) for encoding a specific spectral distribution (310) of light (210) into a spectral distribution identifier (270), wherein the light (210) has wavelength-specific intensity (Ak, [A]) in the specific spectral distribution (310) within a wavelength range (351) from a minimum wavelength (λ_min) to a maximum wavelength (λ_max) having a wavelength resolution (352, Δλ), and the method (400) is Receiving the aforementioned light (210) (410), The received light (210) is filtered individually (420-n) using each filter (120-1, 120-n, 120-N) of the filter set (120) (120-1, 120-n, 120-N), wherein the filter set (120) has at least N=3 filters (120-1, 120-n, 120-N), The aforementioned filter set (120) has the following conditions: (1) Each filter (120-n) in the filter set (120) has a filter-specific transmission function (Tn) that describes the wavelength-specific (k) transmission (Tnk) of the filter (120-n), (2) In the combined transmission function ({T}) for the filter set (120), the transmission vector ({T}k), which is the concatenation ({T}k) of the transmissions (TnK), is unique for any wavelength (k) and unique for each spectral distribution (315) of light (215) within the wavelength range (351). (3) Each filter (120-n) in the filter set (120) is a single-state filter, and the filtering (420-n) conforms to this condition. For each of the N filters (120-n), the intensity of the filtered light (220) over the wavelength range (351) is measured individually (430-n) to obtain provisional intensity values ​​(Bn), which are the N elements of the provisional intensity vector ({B}, 230-n). The calculation function (160, 170) absorbs light fluctuations using an intensity reference value (L_DATA) and absorbs filter fluctuations using predetermined calibration data ({CAL}), and provides (402) a spectral distribution identifier (270) having N elements that encodes the specific spectral distribution (310) of the received light (210), wherein the provision of the spectral distribution identifier (270) (402) is performed by the calculation function (460, 470), and the first calculation function (160) uses the provisional intensity value (Bn) Method (400), which calculates the normalized intensity value (Fn), which is an element of the normalized intensity vector ({F}), by dividing by the intensity reference value (L_DATA) individually (460-n), and the second calculation function (170) maps the normalized intensity value (Fn) to the calibrated normalized intensity value (Dn), which is an element of the calibrated normalized intensity vector ({D}), according to the predetermined calibration data ({CAL}) (470-n).

3. The method according to claim 1 or 2 (400), wherein filtering (420-n) the received light (210) is performed by the filter set (120) in such a way that the conditions are checked for suitability by simulation based on coding parameters (351, 352, 353, 354) that define each spectral distribution (315).

4. The method according to claim 1 (400), wherein in step (402), the second calculation function (150) obtains the intensity reference value (L_DATA) by processing the provisional intensity vector ({B}) or by processing the calibrated intensity vector ({C}).

5. In the aforementioned step (402), the second arithmetic function (150) The aforementioned strength reference value (L_DATA) is calculated as the sum (Σ) of the provisional strength values ​​(Bn), or as the sum (Σ) of the calibrated strength values ​​(Cn), The aforementioned strength reference value (L_DATA) is calculated as the average (Σ / N) of the provisional strength value (Bn), or as the average (Σ / N) of the calibrated strength value (Cn), The method according to claim 1 or 4 (400), wherein the strength reference value (L_DATA) is obtained by a calculation selected from: calculating the strength reference value (L_DATA) as the median of the provisional strength values ​​(Bn) or as the median of the calibrated strength values ​​(Cn).

6. The first arithmetic function (160) is, The aforementioned strength standard value (L_DATA) is calculated as the sum (Σ) of the aforementioned provisional strength values ​​(Bn), The aforementioned strength reference value (L_DATA) is calculated as the average (Σ / N) of the aforementioned provisional strength value (Bn), The method according to claim 2 (400), wherein the strength reference value (L_DATA) is obtained by a calculation selected from the calculations of the median of the provisional strength values ​​(Bn).

7. The aforementioned intensity reference value (L_DATA) is obtained by measuring the intensity of the received light (210) using a sensor (131A / 131B) (431). In the first method, the received light (210) proceeds directly to the sensor (131A), The second method (400) according to any one of claims 1 to 6, wherein the received light (210) indirectly travels to the sensor (131B) via a neutral photoconductor (121B), and the sensor (131B) measures the intensity of the received light (210) by correcting the loss introduced by the neutral photoconductor (121B) (431).

8. The method according to any one of claims 1 to 7 (400), wherein the received light (210) is filtered (420-n) by the set (120) of filters (120-1, 120-n, 120-N) by a combination (823-12 / 830-12, 823-14 / 830-14) of a plurality of filters (880) arranged in a plane (X, Y) perpendicular to the direction (Z) of the received light (210).

9. The method according to any one of claims 1 to 8 (400), wherein in the step of filtering the received light (420-n), the wavelength-specific transmission (Tnk) exceeds at least 50% of the transmission threshold (340-n) for all wavelengths (k) within the wavelength range (351).

10. The aforementioned intensity reference value (L_DATA) is obtained by measuring the intensity of the received light (431), The process of filtering the received light (210) by the set of filters (120-1, 120-n, 120-N) (420-n) and measuring the intensity of the filtered light (210) (430-n) is carried out in an arrangement where the filter (120-n) of the filter set (120) is combined with the corresponding sensor, and these filter-sensor combinations (823-12 / 830-12, 823-14 / 830-14) are arranged in a plane perpendicular to the direction of the received light (210). The intensity of the received light (210) is measured (431) to obtain the intensity reference value (L_DATA), which is performed by a combination of a conductor and a sensor arranged in the same plane, and the combination of the filter and the sensor, and the combination of the conductor and the sensor, form a mosaic pattern in the plane. Before the mapping (440-n, 470-n) step, (ii) simulate the intensity vector ({B}) using simulation data for a predefined programmable spectral distribution and as ground truth. ～ ), and the simulated reference filter (125 ～ The method according to claim 1 or 2 (400), wherein calibration data ({CAL}) is determined by training a neural network (140 / 170-NETWORK) using simulated filtering by (702), and the weights in the network function as calibration data ({CAL}) for performing the mapping (440-n, 470-n) steps by, in an alternative form of claim 1, transferring the provisional intensity value (Bn) to the neural network (140-NETWORK) that outputs a calibrated intensity value (Cn), or, in an alternative form of claim 2, transferring the normalized intensity value (Cn) to a preliminary neural network (170-NETWORK) that outputs a calibrated normalized intensity value (Dn).

11. A computer implementation method (600) for decoding spectral distribution identifiers (250, 270) obtained by carrying out the method described in any one of claims 1 to 10, wherein the decoding method (600) comprises the following steps: (610) A step of receiving the spectral distribution identifier (250 / 270, {D}), The step of mapping the spectral distribution identifier (250 / 270, {D}) to a specific spectral distribution [E] (620) is included, Mapping (620) Accessing a library that represents a predefined relationship between a specific spectral distribution identifier (250 / 270, / {D} / ) and a specific spectral distribution ( / [E] / ), Multiple known input distributions (310, ～ [A] ～ ) corresponds to multiple spectral distribution identifiers (250 / 270, ～ {D} ～ The simulation is performed to encode the above into the corresponding simulated distribution identifier ({D} ～ The specific spectral distribution ([E]) is configured to match the received spectral distribution identifier ({D}) to the specific simulated input distribution ([A]) ～ ) to identify as, A computer implementation method (600) is carried out by either processing the received spectral distribution identifier ({D}) with a pre-trained neural network (106-NETWORK) that classifies it into one of a predefined distribution ([E]), or by any other means.

12. A device (800) adapted to encode a specific spectral distribution (310) of light (210), wherein the device (800) comprises a filter layer (825) attached to a sensor layer (835), and the filter layer (825) and the sensor layer (835) are planar layers. The filter layer (825) is configured to receive light (210) on its surface (XY) and to transmit the received light (210) to the sensor layer (835). The filter layer (825) and the sensor layer (835) are divided into separate pixel positions (850-xy) that correspond to the filter position (823-xy) and the sensor (830-xy), The sensor (830-xy) is adapted to quantify the intensity of the transmitted light by integrating the intensity (A) of the transmitted light during a measurement time interval, thereby providing a sensor-specific provisional intensity value (Bn, LUMINANCE) representing the transferred light (220). Below, for a sequence of M pixel positions referred to as area (880), if N < M, N filter positions (823-xy) hold a filter set (823-21, 823-12, 823-32, 823-23) of N≧3 filters above the corresponding sensors (830-21, 823-12, 823-32, 823-23), and as a result, the corresponding sensors provide the sensor-specific intensity values ​​as provisional intensity values ​​(Bn). The following is applied: M-N filter positions (823-11, 823-31, 823-22, 823-13, 823-33) are either empty filters or hold filters with wavelength-independent transmission functions, and as a result, the corresponding sensors (830-11, 830-31, 830-22, 830-13, 830-33) provide sensor-specific intensity values ​​as luminance values ​​(LUMINANCE). Each of the N filters (823-21, 823-12, 823-32, 823-23) in the filter set has a filter intrinsic transmission function (Tn) that describes the wavelength-specific (k) transmission (Tnk) of the filter (823-21, 823-12, 823-32, 823-23), and in the combined transmission function ({T}) for the filter set, the concatenation ({T}k) of the transmission (TnK), which is the transmission vector ({T}k), is unique for any wavelength (k) and unique for each spectral distribution (315) of light (215) within the wavelength range (351). The device (800) further comprises a computing module (840, 850, 860, 870) that processes the provisional intensity value (Bn) with respect to a normalized intensity value (Dn), which is an element of a normalized intensity vector ({D}) corresponding to a spectral distribution identifier (250, 270) that encodes the specific spectral distribution (310) of the received light (210).

13. The device (800) according to claim 12, wherein the calculation modules (840, 850) are implemented such that the mapper (840) is adapted to map the provisional intensity value (Bn) to the calibrated intensity value (Cn), which is the element of the calibrated intensity vector ({C}), using predetermined (732) and device-specific calibration data ({CAL}) (440-n), and the divider (850) is adapted to individually divide the calibrated intensity value (Cn) of the calibrated intensity vector ({C}) by the luminance value (LUMINANCE) (450-n).

14. The device (800) according to claim 12, wherein the calculation modules (860, 870) are implemented such that a divider (860) is adapted to obtain an intermediate normalized intensity vector ({F}) by individually dividing the provisional intensity value (Bn) by the luminance value (LUMINANCE) (460-n), and a mapper (870) is adapted to map the intermediate normalized intensity value (Fn) to a calibrated normalized intensity value (Dn) of the normalized intensity vector ({D}) using predetermined, device-specific calibration data ({CAL}) (470-n).

15. The device (800) according to any one of claims 12 to 14, wherein the filter layer (825) and the sensor layer (835) are arranged in a plane (X, Y) perpendicular to the direction (Z) of the received light (210).

16. The device (800) according to any one of claims 12 to 15, wherein the wavelength-specific transmission (Tnk) exceeds at least 50% of the transmission threshold (340-n) for all wavelengths (k) within the wavelength range (351).

17. A computer program product that, when loaded into the memory of a computer and executed by at least one processor of the computer, causes the computer to provide (402) the spectral distribution identifiers (250, 270) according to any one of claims 1 to 10, and to perform the arithmetic functions (440-n, 450-n, 460-n, 470-n, 840, 850, 860, 870).

18. A computer program product that, when loaded into the memory of a computer and executed by at least one processor of the computer, causes the computer to perform the steps of the computer implementation method (600) for decoding the spectral distribution identifiers (250, 270) according to claim 11.

Images