Secure processing of spectroscopic data

Abstract

The invention is in the field of secure processing of spectroscopic data. The invention relates to method for processing spectroscopic data on a mobile computing device comprising: a. receiving spectroscopic data from a spectrometer module of the mobile computing device, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment on the mobile computing device and c. outputting the chemometric data.

Description

[0001] Secure Processing of Spectroscopic Data

[0002] The invention is in the field of secure processing of spectroscopic data. The invention relates to method for processing spectroscopic data, a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method and portable computing device.

[0003] Background

[0004] Spectrometer modules which can be integrated into portable devices like smartphones or tablets will become commercially available soon. In order to provide a meaningful information to a user, a chemometric model is required to transform the raw data obtained from the spectrometer module into compositional data, i.e. an information related to the composition of the measured object. The model may contain sensitive information, for example medical parameters of a user. A portable device can be typically accessed by internet. In addition, software is frequently loaded onto the portable device. Consequently, there is a serious threat of loss of sensitive data or manipulation.

[0005] WO 2022 / 223788 A1 discloses a method for spectroscopic monitoring of liquids. The method may be run on a virtual machine above the physical hardware. However, no details about any measures against data loss or manipulation are disclosed.

[0006] US 2023 / 0103518 A1 discloses a system to generate a trusted execution environment including multiple accelerators. It is mentioned that the system may be used for functional near-infrared spectroscopy. However, no details on how to implement the system on a mobile computing device for spectroscopy is given.

[0007] Summary

[0008] The objective of the present invention was to provide a method to securely process spectroscopic data, i.e. make it less likely that sensitive data can be accessed by unauthorized persons. The method was aimed to be flexible, so it does not compromise usability of a spectrometer device.

[0009] In one aspect the invention relates to a method for processing spectroscopic data on a mobile computing device comprising: a. receiving spectroscopic data from a spectrometer module of the mobile computing device, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment on the mobile computing device and c. outputting the chemometric data. In another aspect the invention relates to a method for processing spectroscopic data comprising: a. receiving spectroscopic data, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment and c. outputting the chemometric data.

[0010] In another aspect the invention relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a. receiving spectroscopic data, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment and c. outputting the chemometric data.

[0011] In another aspect the invention relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a. receiving spectroscopic data from a spectrometer module of the mobile computing device, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment on the mobile computing device and c outputting the chemometric data.

[0012] A mobile computing device comprising: a a spectrometer module to generate spectroscopic data, b a processor configured to generate chemometric data by executing a chemometric model using the spectroscopic data as input and outputting the chemometric data, wherein the chemometric model is executed in a secure environment, and c. an output to output the chemometric data.

[0013] In another aspect the invention relates to a portable computing device comprising: a. an input configured to receive spectroscopic data, b. a processor configured to generating chemometric data by executing a chemometric model using the spectroscopic data as input and outputting the compositional data, wherein the chemometric model is executed in a secure environment, and c. an output to output the chemometric data. In another aspect the invention relates to a use of the chemometric data obtained by the mobile computing device or the process according to any of the preceding claims for providing the composition of an object or a recommendation for acting on the object based on its composition.

[0014] By executing the chemometric model in a secure environment it is made sure that sensitive data are shielded against unauthorized access. The chemometric model itself may contain sensitive data, for example medical data of a person it is trained for. It is therefore critically important that no unauthorized person has access to such chemometric model data. Further, execution in a secure environment ensures that the chemometric model does not get compromised, for example by changes to its parameters by an unauthorized person. In this way, the output chemometric data becomes more reliable and trustworthy. It can be made sure that the chemometric model has not undergone any changes by unauthorized persons. At the same time, the device does not need to be shielded against any access, hence it is still possible to connect the device with the internet, so updates to apps or remote access remain possible. Ensuring security in a secure environment can achieve a high security level at comparatively low extra computing time in comparison to pure software-based security solutions.

[0015] The term “spectroscopic data” may refer to data comprising intensity values, wherein each intensity value is indicative of the intensity of the electromagnetic radiation in at least a part of its spectral range. The spectroscopic data may hence contain a set of value pairs, wherein a pair contains a value indicative of the spectral range, i.e. the wavelength or the wavelength range of the electromagnetic radiation, and a value indicative of the corresponding intensity of the electromagnetic radiation. Hence, the spectroscopic data may comprise a spectrum of electromagnetic radiation Spectroscopic data may be obtained from a measurement of an object with a spectrometer The spectroscopic data may contain the direct output values of the sensors of a spectrometer, sometimes referred to raw data, or it may contain adjusted values, for example corrected with calibration coefficients. Spectroscopic data may further comprise data associated with the measurement device, for example a device ID or device settings, or with the measurement environment, for example the time of measurement or temperature of the measurements.

[0016] The term “electromagnetic radiation” may refer to radio waves, microwaves, infrared light, visible light, ultraviolet light, X-rays, or gamma rays. Infrared light may be near-infrared with a wavelength of 760 nm to 1 .4 pm, shortwavelength infrared with a wavelength of 1.4 to 3 pm, mid-wavelength infrared with a wavelength of 3 to 8 pm, long- wavelength infrared with a wavelength of 8 to 15 pm or far infrared with a wavelength of 15 to 1000 pm. For example, spectroscopic data may comprise intensity values for electromagnetic radiation of wavelength 1 to 3 m.

[0017] The spectroscopic data may be received directly or indirectly from a spectrometer. Directly may refer to receiving the spectroscopic data via a data link, for example a cable or a wireless communication interface. Indirectly may refer to receiving the spectroscopic data from a storage device where the spectroscopic data from a spectrometer has been stored to. The spectrometer may be a device separate to the device on which the spectroscopic data is processed. The spectrometer may be part of the device on which the spectroscopic data is processed, for example as spectrometer module built into a portable device such as a smartphone, a tablet or a smartwatch. The following sections refer to the spectrometer module, but the disclosure is equally applicable to any spectrometer.

[0018] The spectrometer module may comprise a radiation source. The term "radiation source” may refer to a device that generates and / or emits electromagnetic radiation, for example electromagnetic radiation in the wavelength range described above. The radiation source may be configured for illuminating an object. The radiation source may be or comprise a thermal radiator or a semiconductor-based radiation source. The at least one semiconductor-based radiation source may be a light emitting diode (LED) or a laser, in particular a laser diode. The thermal radiator may be an incandescent lamp or a thermal infrared emitter.

[0019] The spectrometer module may be configured to generate spectroscopic data. The spectroscopic data may hence be received from the spectrometer module. The spectrometer module may contain a photosensitive detector. The term “photosensitive detector” may refer to a device which generates an electrical signal in response to receiving electromagnetic irradiation. The photosensitive detector may contain one or more than one photosensitive region, wherein each photosensitive region may generate a separate electrical signal indicative for the intensity of the electromagnetic irradiation impinging onto the photosensitive region.

[0020] The photosensitive detector may be an inorganic camera element, such as an inorganic camera chip, a CCD chip or a CMOS chip. The photosensitive detector may comprise a photoconductive material, in particular an inorganic photoconductive material, especially selected from lead sulfide (PbS), lead selenide (PbSe), germanium (Ge), indium gallium arsenide (InGaAs, including but not limited to ext. InGaAs, i.e. InGaAs which exhibits a spectral response up to 2.6 pm), indium antimonide (InSb), or mercury cadmium telluride (HgCdTe or MCT).

[0021] The photosensitive detector may contain multiple photosensitive regions, for example an array of photosensitive regions, wherein electromagnetic radiation of different wavelength impinges on the photosensitive regions. For this reason, the spectrometer module may contain a dispersive element, such as a prism or a grating. The spectrometer module may contain one or more than one optical filters, for example a linear variable filter or multiple filters transmit different wavelengths of the electromagnetic radiation for different photosensitive regions. The dispersive element or the filter may be arranged such that electromagnetic radiation of different wavelength impinges on the multiple photosensitive regions. The photosensitive detector may contain multiple photosensitive regions, wherein the photosensitive regions are covered by filters, wherein the filters transmit electromagnetic radiation of different wavelengths.

[0022] The photosensitive detector may contain only one photosensitive region. Light of different wavelength may impinge on this one photosensitive region at different points in time. For this reason, the spectrometer module may contain an interferometer, for example a Michelson interferometer. The interferometer may be arranged such that it receives the incoming electromagnetic radiation and directs the electromagnetic radiation having passed the interferometer to the photosensitive region. The interferometer may also be placed between the radiation source of the spectrometer module and the object to be measured, so the wavelength of the electromagnetic radiation received by the spectrometer module varies with time.

[0023] The electric signal from the detector may be analog or digital. If it is analog, it may be converted into a digital signal. Therefore, the spectrometer module may contain an analog-to-digital converter (ADC). The spectrometer module may contain a controller. The controller may contain ADC functionality. The controller may provide the spectroscopic data. The spectroscopic data may contain some or all of the electric signals. The spectroscopic data may further contain the wavelength or wavelength range of the electromagnetic radiation associated with the electric signal. The information about the wavelength or wavelength range may be obtained by identifying which photosensitive region the electric signal originates from and a stored value indicating a wavelength or wavelength region for this photosensitive region, for example by providing the filter information of the filter on the particular photosensitive region.

[0024] The spectrometer module may be operationally coupled to a processor, for example the CPU of a portable device which the spectrometer module is integrated into. The processor may be configured to execute code to apply calibration coefficients. Calibration coefficients may relate to values usable for correcting the output of the spectrometer module to compensate for fabrication imperfections or drifts over time. Calibration coefficients may need a change from time to time to adjust to changes like said drift. In order to do so, the spectrometer module needs to execute a calibration Calibration can be performed in various ways.

[0025] Calibration may involve a reference target which is measured instead of a sample. A reference target may have a highly reproducible reflectance behavior. The reference target may have a high reflectivity within the wavelength range of the spectrometer module, sometimes also referred to a white reference target. Other reference targets may be applied, for example reference targets with a characteristic reflectance behavior which is different for different wavelengths. The reference target may be a separate piece which needs to be placed in front of the spectrometer module. Alternatively, the spectrometer module may comprise a reference target. In this case, the spectrometer module may contain optics capable of directing radiation from the radiation source to the reference target. The optics may further be capable of directing radiation reflected by the reference target to the photosensitive detector.

[0026] Calibration may be performed by directing radiation from the radiation source directly onto the photosensitive detector. The spectrometer module may hence comprise optics capable of directing radiation from the radiation source to the photosensitive detector, for example a mirror or a lens. In case of a mirror, it is possible to use the characteristic reflectance of the mirror for calibration. The spectrometer module may contain a window which partially transmits the radiation from the radiation source to the sample and partially reflects the radiation from the radiation source to the photosensitive detector. The window may comprise or consist essentially of or consist of an optical glass, for example silica glass, phosphate glass, aluminosilicate glass, or silicon. Many of such materials exhibit Fresnel reflection characteristics, i.e. a window containing said materials may reflect radiation at certain wavelengths in well-defined ratios which are often independent of the temperature. During sample measurement, such reflected radiation may be subtracted from the measurement or it may be blocked by optics, for example a shutter or an aperture.

[0027] Calibration involving a reference target or directing radiation from the radiation source directly onto the photosensitive detector yields a detector signal which can be correlated to the known intensity of the incoming radiation. This correlation may be used to generate calibration coefficients. The calibration coefficients may be used to correct the detector signals.

[0028] Calibration may be performed by avoiding any incoming radiation onto the photosensitive detector, sometimes also called dark calibration. Avoiding incoming radiation may involve removing any sample from the spectrometer module, switching off the radiation source or blocking any incoming light, for example by an aperture or a shutter.

[0029] Alternatively, the spectrometer module may contain one or more than one photosensitive detectors which never receive any incoming radiation, for example by covering them with an absorbing or reflecting material. The signal of a photosensitive detector which temporarily or permanently does not receive any incoming radiation may be used as background signal. The background signal may be subtracted from the signal upon measurement. The background signal may also serve to determine calibration coefficients. The calibration coefficients may be used to correct the detector signals.

[0030] A calibration may be triggered by the application. A measurement request submitted by the application to the interface may contain a request to calibrate prior to the measurement, for example as a flag or parameter to the request function call. The interface may provide a separate function call to the application for triggering a calibration request. The request for a calibration may comprise which kind of calibration is requested, for example a white or a dark calibration as described above.

[0031] The spectrometer module may determine if a calibration is required, for example prior to a measurement upon receiving a measurement request. In this case, the processor is configured to determine if calibration of the spectrometer module is required and if so trigger a calibration.

[0032] The term “chemometric model” may refer to a model which uses spectroscopic data as input and outputs chemometric data. Hence, the chemometric model may translate spectroscopic data of an object, for example optical spectra, into corresponding chemometric data. The chemometric data obtained from the chemometric model may be associated with one or more than one physical or chemical characteristics, for example at least two or at least three physical or chemical characteristics. A chemometric model which outputs more than one physical or chemical characteristic may be referred to as multilabel chemometric model. A multilabel chemometric model may contain several partial chemometric models, wherein each partial chemometric model may be configured to translate a spectrum to one physical or chemical characteristic. The multilabel chemometric model may be configured to merge the chemometric data obtained from the at least two partial chemometric models, for example to output one vector comprising values for each physical or chemical characteristic.

[0033] A chemometric model may comprise a pre-processing method and a machine learning model to obtain the chemometric data. A chemometric model may comprise a pre-processing method, a feature selection filter and a machine learning model. If the chemometric model comprises two or more partial chemometric models, each partial chemometric model may comprise a separate pre-processing method, a feature selection filter and a machine learning model. Alternatively, the partial models may use the same pre-processing method or feature selection filter.

[0034] A chemometric model may comprise a pre-processing method. The term “pre-processing” may refer to a method to reduce or eliminate interferences from a spectrum such as stray light, noise or baseline drift to enhance the subsequent machine learning. Hence, the pre-processing method may be applied before the machine learning method. Pre-processing may include one or more of baseline correction, scatter correction, smoothing, scaling, aggregation.

[0035] A chemometric model may comprise a machine learning method. The term “machine learning method” may refer to a model which translates spectroscopic into corresponding chemometric data. The machine learning method hence may use spectroscopic data as input and derive chemometric data therefrom. The machine learning method may be considered as an integral part of the chemometric model. Machine learning methods may be supervised, semisupervised or unsupervised Machine learning methods may include multivariate calibration, classification, pattern recognition, clustering, ensemble methods, neural nets and deep learning, or multivariate curve resolution.

[0036] A chemometric model may comprises a feature selection filter. The term “feature selection filter" may refer to a method to select those parts of the spectrum with a correlation to the chemometric data. A feature selection filter may facilitate the machine learning method of the chemometric model and thus avoid overfitting and reduce the number of required training datasets. A feature selection filter may use a spectroscopic data input, remove all unselected parts and output a spectrum with only the selected parts left. Hence, the output of the feature selection filter may be a spectrum in form of a vector of lower dimensionality than the input vector. The output of the feature selection filter can be used as input for the machine learning method. Hence, the feature selection filter may be applied before the machine learning method. The input of the feature selection filter may be the received spectroscopic data or it may be the pre-processed spectrum, preferably the pre-processed spectrum. Hence, the feature selection filter may be applied after the pre-processing method.

[0037] A chemometric model may be a trained chemometric model. Training may comprise adjusting parameters of the chemometric model such that the output of the chemometric model most closely fits to historic chemometric data associated with the spectroscopic data. Often, training comprises minimizing a loss or cost function, for example a least mean square value of chemometric model output to provided chemometric data. The complete set of received spectra and associated chemometric data may be used for training or parts thereof. A trained chemometric model may be validated chemometric model. Validation may be done with parts of the historic chemometric data associated with the spectroscopic data not used for training. Alternatively, cross-validation may be applied, for example K-fold cross-validation, leave-one-out cross-validation, stratified cross-validation.

[0038] The term “chemometric data” may refer to data associated with a physical or chemical characteristics of an object. Chemometric data may hence contain parameters representing physical or chemical characteristic of the object. Physical characteristics may comprise thermal characteristics, for example the temperature, the thermal conductivity or the specific heat capacity; the macroscopic or microscopic state of matter, for example the degree of crystallinity; optical characteristics, for example refractive index, optical conductivity or absorption coefficients; electrical characteristics, for example electrical conductivity or dielectric constant. Chemical characteristics of an object typically refer to the chemical composition of the object comprising the material composition, for example the type and the concentration of certain chemical compounds such as the water content; the molecular structure of an object, for example the presence of functional groups like carbonyl groups or hydrogen bonds; the molecular weight, for example the degree of polymerization of a polymer; the state of matter, for example the degree of crystallinity or the aggregate state, for example in a disperse phase.

[0039] Chemical characteristics may further comprise biomarkers if the measured object is a living organism, in particular a human. Biomarkers may be chemical compounds found in the body which are indicative for a certain condition of the body like a disease, the fitness level or an intoxication condition. Examples for biomarkers are blood components, for example the blood glucose level, the oxygen level in blood or the blood alcohol level; skin components like moisture level or lipid content; metabolites like uric acid or degradation products of drugs in urine; protein biomarkers like cancer proteins or proteins of a virus. The chemometric model may been optimized for a group of people or it may be optimized for one person, i.e. it has been trained with training data from one particular person. Alternatively, the chemometric model has been trained with training data from multiple persons, but has been retained with training data from one particular person. In particular in the later case, the chemometric model may contain sensitive medical information and hence needs to be shielded against unauthorized access.

[0040] Chemometric data may comprise one physical or chemical characteristic of the object or more than one, for example the concentration of two chemical compounds in a sample. A physical or chemical characteristic can be variable, it can assume any value in a certain range, for example a concentration, or it can be categorial, for example the type of fiber in a fabric.

[0041] A chemometric model may be configured to determine fitness and health information from a spectrum of the skin of a person, for example the hydration level of the skin or the blood glucose or lactose content of the skin. The result may be used to make recommendations to the user, for example to drink water according to the hydration level of the skin or adjust the training program according to the lactose level in the blood.

[0042] A chemometric model may be configured to analyze agricultural products, food or feed from a spectrum of an agricultural product, for example a fruit, vegetable, meat, dairy product; food, for example bread, sauces, sausages, sweets; feed like fodder or forage. For example the chemometric model may determine a relevant content, for example the sugar or protein content. The result may be used to make recommendations, for example the expected date for harvesting to a farmer or a suitable recipe for a cook.

[0043] A chemometric model may be configured to determine information related to recycling from a spectrum of an object to be recycled, for example a plastic object like a bottle or a metal object like a can. For example, it may be determined which material the object is made of, for example polyethyleneterephthalate (PET) for a bottle. The result may be used to make recommendations, for example the most suitable way for recycling the object or the most convenient place put the object so it can be recycled.

[0044] A chemometric model may be configured to determine product quality in a production process. Determining product quality may mean determining parameters of a specification, for example the concentration of an ingredient. Product quality may refer to the quality of the input material, for example as received from a supplier, of an intermediate, i.e. a material which as undergone some processing steps and will undergo further processing steps, and an output material, for example before it is packaged and shipped to a customer.

[0045] The chemometric model may be executed in a secure environment. The term “secure environment" may refer to an execution environment which is protected against unauthorized access and thus ensures confidentiality, integrity, and authenticity of sensitive data. The secure environment may involve implementing specific hardware, firmware, or software mechanisms that establish a trusted environment within the hardware, in particular processor and memory. For the secure environment, one or more than one security measure may be utilized such as encryption, authentication, access controls, secure memory management, secure bootstrapping, secure storage, or secure communication protocols. The secure environment may be backed by hardware-based access control, for example to memory sections or peripheral devices, in particular a spectrometer module. The secure environment may also comprise a secure execution environment or a trusted execution environment, where critical processes or applications can run in isolated and protected environments, shielding them from potential threats originating from other parts of the system or external entities. The processor may be configured to execute processes in a secure mode. The processor may be communicatively coupled to a security-aware debug infrastructure, for example by a bus system. Such bus system may be separate from the general purpose bus system. The security-aware debug infrastructure may enable control over access to debug processes or system components in the secure environment without impairing debug visibility of processes or components in a regular environment. The chemometric model may be stored in a secure memory section to which only processes running in the secure environment have access. The chemometric model may be encrypted. The encryption key may be stored in a memory section to which hardware-based access control is implemented. Upon execution, the chemometric model, intermediate results or states may be kept in memory spaces only accessible to threads running in the secure environment. The encrypted chemometric model may be loaded in the regular environment, transferred to secure environment in which it is decrypted with a key stored in secure memory and executed. The processor may have a register indicating that the execution is in the secure environment, for example a secure flag, to exclude any access from threads or processes in a non-secure or regular environment. Processors implementing a secure environment are commercially available, for example Secure Enclave on Apple A7 processors or TrustZone on processors with ARM architecture.

[0046] The processor may be configured to switch from a regular, i.e. non-secure, environment to a secure environment. Switching may comprise flushing registers and caches, in particular when switching from secure to regular environment. Alternatively, a processor may comprise multiple cores, wherein some cores execute in the regular environment and the remaining cores execute in the secure environment. The regular environment and the secure environment may be separate, e.g. separate memory space including cache memory may be used for each mode. Memory space of processes in the secure environment may be invisible or inaccessible to processes in the regular environment. The processor may be configured to provide a communication interface for processes in the regular environment and processes in the secure environment. For example, a process in the regular environment may provide spectroscopic data to a process in the secure environment via a communication interface. A process in the secure environment may execute a chemometric model with the spectroscopic data as input and thus generate chemometric data A process in the secure environment may provide chemometric data via a communication interface to a process in the regular environment. Hence, a process in the regular environment may trigger the execution of a chemometric model in the secure environment and receive the generated chemometric data. In this way, the process in the regular environment has no access to any details of the chemometric model.

[0047] The spectrometer module may be integrated into the device comprising a processor executing the chemometric model in the secure environment. The spectrometer module may be registered with the processor as a trusted hardware component. In this way, it may be made sure that spectroscopic data originates from the spectrometer module and is uncompromised. The spectroscopic data may be received from the spectrometer module in the secure environment, any spectroscopic data received from any process in a non-secure environment may be rejected. The spectrometer module may only be visible to processes running in the secure environment. Hence, the processor may receive the spectroscopic data from the registered spectrometer module and execute the chemometric model with the received spectroscopic data in the secure environment. A process in the regular environment may trigger a measurement, for example by calling a function or setting a request flag. A process in the secure environment may react to such trigger by causing the spectrometer module to execute a measurement. The process in the secure environment may receive the spectroscopic data from the spectrometer module. The process in the secure environment may execute the chemometric model with the received spectroscopic data as input and thereby generate chemometric data. The process in the secure environment may provide the generated chemometric data to a process in the regular environment via a communication interface. In this way, both the integrity of the spectroscopic data as well as the chemometric model can be made sure.

[0048] The chemometric data generated by the chemometric model may be output. The term “outputting” may refer to writing the chemometric data to a non-transitory data storage medium, for example into a file or database, display it on a user interface, for example a screen, or both. It is also possible to output the chemometric data through an interface to a cloud system for storage and / or further processing.

[0049] The method may be implemented on a computer system comprising an application, an interface to the application and an operating system. The application may trigger the execution of the chemometric model in the regular environment. The chemometric data may be output by providing it to the application in the regular environment. The application may trigger the execution of the chemometric model through the interface. The chemometric data may be output by providing it to the application through the interface.

[0050] The term “application” may refer to a piece of executable program code which carries out one or more specific tasks which are not related to the operation of the device. An application may contain a user interface, for example a graphical user interface (GUI), to receive input from a user and display information to the user. An application usually makes use of the hardware via programming interfaces, for example a programming interface to the operating system An application may be preinstalled on the device by the manufacturer of the device or it may be provided by other software providers, for example in an app store, so the user can download and install the application In the context of the present invention, the application may make use of the spectrometer, i.e. it may process information retrieved from the spectrometer.

[0051] The term “interface" may refer to a shared boundary across which the spectrometer, for example via its firmware or the operating system of the device in which the spectrometer is built into, can exchange information with the application. The interface may be or include an application programming interface (API) configured to receive function calls, messages or other types of invocations which when executed enable information exchange between the application and the spectrometer. In addition, the API may provide the calling program code of the application the ability to use data types or classes defined in the API and implemented in the called program code. The interface can include further functions, for example hardware-specific tools which can communicate with the spectrometer hardware and / or the portable device. Hence, the interface may be, may be part of or may include a software development kit (SDK).

[0052] The interface may be configured to receive a request for performing a spectroscopic measurement. The interface may receive a request for performing a spectroscopic measurement from the application. The interface may be configured to receive measurement requests from more than one application. The interface may be configured to receive more than one measurement request concurrently. Such request may be a function call provided by the interface. Alternatively, a request may be an entry into a request list, for example a queuing file, which is provided by the interface and continuously read by the interface to react to the request. A request may further contain parameters to influence the measurement, for example the sample time of the measurement or a number of how often a measurement shall be repeated.

[0053] The request may be forwarded to the spectrometer module to trigger a measurement. The request may be directly forwarded from the interface to the firmware of the spectrometer module. The request may be forwarded from the interface to an operating system which may forward the request to the firmware of the spectrometer module. The operating system may contain a hardware abstraction layer (HAL) and a kernel. The HAL may be executed in user space of the processor, i.e. with limited memory access, for example the user space in virtual memory, and the kernel may be executed in kernel mode of the processor, i.e. without limited memory access, for example in kernel space in virtual memory. Hence, the request may be forwarded from the interface to an HAL of the operating system which may forward it to the kernel of the operating system which may forward it to the firmware of the spectrometer module. Each instance may modify the request in a way that it is compatible to the next instance.

[0054] The HAL may receive standardized, i.e. hardware-independent, function calls or messages and converts them to hardware-specific ones. The HAL may be a passthrough HAL or a binderized HAL. The HAL may contain a spectroscopy HAL which is dedicated to provide a software interface to spectroscopy hardware, such as an interface to the spectroscopy hardware driver. The spectroscopy HAL may provide functions and hardware accesses which are specific to spectroscopy hardware. One example is invoking the calibration function of a spectrometer module.

[0055] The kernel of the operating system may contain a driver for the spectrometer module. The kernel may receive the request from the HAL and may generate hardware-specific code to trigger the spectrometer module to execute a measurement. The kernel may forward the request to the firmware of the spectrometer module. The firmware may control the actual functionality of the spectroscopy hardware, for example the analogue to digital conversion of the sensor signals.

[0056] The firmware may further provide metadata and / or calibration coefficients. Metadata may relate to sensor data, for example the temperature measured at or close to the spectrometer module or movement data measured during the spectroscopy measurement, such as by a gyroscope. Calibration coefficients may relate to values usable for correcting the output of the spectrometer module to compensate for fabrication imperfections or drifts over time. Calibration coefficients may originate from a test just after production of the spectrometer module, such as an end-of- line test, or from a calibration which has been performed in between two spectroscopy measurements. The interface to the application may be configured to provide metadata and / or calibration coefficients from the spectrometer module to the application. The metadata and / or calibration coefficients may be forwarded from the firmware to the interface, for example via the kernel and / or a HAL.

[0057] Chemometric data may be obtained from the spectroscopic data by executing a chemometric model. The chemometric model may be part of the HAL. The chemometric data may be passed to the interface to the application. The interface to the application may provide the chemometric data to the application. The interface may provide the data as return value for the measurement request function call. Alternatively, the interface may provide the data by storing it in a dedicated place in memory from which the application can read the data. The interface may return a pointer to the place in memory to the application.

[0058] In another aspect, the invention relates to a mobile computing device such as a portable device or a device integrated into a vehicle. The term “mobile computing device” may refer to a device which is designed, for example in terms of weight and dimensions, to be regularly moved to difference locations, for example by being carried by a human or by being moved by the vehicle it is integrated into.

[0059] The term “portable device” may refer to a device having dimensions and weight which allow a human to carry the portable device without major physical effort. The portable device may be a smartphone, a tablet or a wearable. The wearable may be a wristwatch, a wristband, a hearing aid, a ring, a belt, a necklace, an ankle band, a thigh band, or a forearm band. The portable device may comprise a spectrometer module. The spectrometer module may be fully integrated into the portable device, i.e. all parts of the spectrometer are contained in the portable device Parts of the spectrometer module, for example the processor, may be shared with other functionalities in the portable device.

[0060] Alternatively, one or more than one components of the spectrometer may be physically placed outside the portable device and may be communicatively coupled to the portable device. The portable device may contain a communication interface, for example a telecommunication interface or a Bluetooth interface. The communication interface may enable data flow between components of the spectrometer outside the portable device and the portable device. The portable device may be communicatively connected to a cloud computing system. The cloud computing system may execute parts or all of the data processing of the spectrometer, for example derive composition data from the spectroscopic data. The portable device may send via the communication interface spectroscopic data to a cloud system, the cloud system derives composition data from the spectroscopic data, for example by using a chemometric model, and send the composition data back to the portable device.

[0061] The spectrometer module may be part of a separate spectrometer device, for example a hand-held measurement head. The spectrometer device may be communicatively coupled to the portable device, for example by a Bluetooth interface. The portable device may send a measurement request from the application to the spectrometer device via a communication interface. The spectrometer device may send spectroscopic data to the portable device by a communication interface. The mobile computing device may be integrated into a vehicle, for example a, a motorcycle, a bus, a truck, a train or an airplane. The vehicle may comprise a device comprising both a processor and a spectrometer module. The vehicle may comprise a spectrometer module communicatively coupled with a processor of the vehicle, for example the board computer. The board computer may be configured to execute other functions of the vehicle as well as executing a chemometric model. In the context of the invention, all parts of the vehicle involved in executing the method may be regarded as a mobile computing device.

[0062] The mobile computing device comprises a processor. The processor may be a logic circuitry configured for performing basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-processor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an L1 and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor’s elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs) and / or one or more tensor processing unit (TPU) and / or one or more chip, such as a dedicated machine learning optimized chip, or the like The processor specifically may be configured, such as by software programming, for performing one or more evaluation operations.

[0063] The mobile computing device comprises an output or output interface. The output or output interface may be configured to output the chemometric data. The output interface may be an interface to a screen, an interface to a memory or an interface to a different process, for example an inter process communication interface provided by an operating system of the mobile device.

[0064] The present invention further relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to the present invention. The term "computer-readable data medium” may refer to any suitable data storage device or computer readable memory on which is stored one or more sets of instructions (for example software) embodying any one or more of the methodologies or functions described herein. The instructions may also reside, completely or at least partially, within the main memory and / or within the processor during execution thereof by the computer, main memory, and processing device, which may constitute computer-readable storage media. The instructions may further be transmitted or received over a network via a network interface device. Computer-readable data medium include hard drives, for example on a server, USB storage device, CD, DVD or Blue-ray discs. The computer program may contain all functionalities and data required for execution of the method according to the present invention or it may provide interfaces to have parts of the method processed on remote systems, for example on a cloud system.

[0065] Brief Description of the Figures

[0066] Figure 1 illustrates the acquisition of chemometric data and spectroscopic data from an object.

[0067] Figure 2 illustrates an example for a portable computing device with an integrated spectrometer module.

[0068] Figure 3 illustrates an embodiment of the present invention.

[0069] Figure 4 illustrates another embodiment of the present invention.

[0070] Figure 5 illustrates exemplary components of a processor configured to switch between regular environment and secure environment.

[0071] Figure 6 illustrates an exemplary embedding of the process into a software architecture of a portable computation device.

[0072] Description of Embodiments

[0073] Figure 1 illustrates the acquisition of chemometric data and spectroscopic data from an object. A spectrometer 110 may illuminate the object 103, for example a nutrient, a pharmaceutical or a body part, with a light beam 102. The spectrometer may contain a radiation source 110, for example an incandescent lamp or an LED, and illumination optics. The light beam 102 may contain light in the infrared wavelength range, for example 780 nm to 3000 nm. The light beam may hit the object and be reflected back to the spectrometer. This measurement mode is often referred to as reflectance mode. Alternatively, the light beam may penetrate the object and be directed back to the spectrometer, for example with mirrors. This measurement mode is often referred to as transmission mode. The spectrometer 110 may comprise a detector 112 comprising light collection optics to capture the light beam 102 travelling from the object 103 to the spectrometer 101. The detector 112 may comprise an optical element which separates different wavelengths of the light beam 102 travelling from the object 103 to the spectrometer 110 in space or in time. The detector 112 may contain one or multiple optical sensors which generate an electrical signal depending on the intensity of light impinging on the optical sensor. The optical sensor may output an analog signal which may be converted into a digital signal by an analog to digital converter (ADC) 113. The intensity data may be associated with the wavelength data, for example by a processor or microcontroller in the spectrometer. The spectrometer 110 may hence output a spectroscopic data 104, for example as vector of wavelength values and an associated vector of intensity values. The object 103 may be analyzed in a laboratory 105 to obtain chemometric data 106, for example the content of a compound in the object 103 as determined by gas chromatography. The spectroscopic data 104 and the chemometric data 106 may be associated, for example by labelling both with an object identifier. The spectroscopic data 104 and the chemometric data 106 may be used to train a chemometric model.

[0074] Figure 2 illustrates an example for a portable computing device 200 with an integrated spectrometer module 210. The portable device 200 may be a smartphone, a tablet or a wearable such as a smartwatch. The spectrometer module 210 may contain a radiation source 211, for example an incandescent lamp or an LED. The radiation source 211 may produce electromagnetic radiation in the desired range, for example in the near infrared range. The radiation source 211 may further contain optics to direct the electromagnetic radiation from the light source to the object, for example lenses, mirrors and / or apertures. The radiation source 211 may be communicatively coupled to a controller 213. The controller 213 may supply electric power, e.g. from the battery of the portable computing device 200, and switch the radiation source 211 on and off when required.

[0075] The spectrometer module 210 may further contain a detector 212. The detector 212 may generate electric signals in response to electromagnetic irradiation impinging on a sensor. The detector 212 may contain an array of photosensitive regions. Each photosensitive region may be covered by a filter such that electromagnetic radiation of a dedicated wavelength or wavelength range impinges on a photosensitive region. The photosensitive region may be sensitive in the wavelength region of interest, for example in the near infrared region. The photosensitive region may contain a photoconductor, for example PbS or PbSe. The photosensitive region may generate an electric current which is indicative of the intensity of the electromagnetic radiation impinging on the photosensitive region The detector 212 may contain optics to collect a maximum of incoming electromagnetic radiation. The optics may include mirrors, lenses and / or apertures.

[0076] The detector 212 may be communicatively coupled to the controller 213. The controller 213 may collect the signal or the signals from the detector 212 and forward these to the processor 202. The controller 213 may convert the signal or signals from analog to digital, hence the controller 213 may act as analog to digital converter (ADC). This may, for example, be accomplished by integrating the electric current obtained from each photosensitive region and providing a value of the result in a digital form. By combining these values with the origin of the photosensitive region each of which measures the electromagnetic radiation at a particular wavelength or wavelength region, the controller 213 may gather spectroscopic data and forward these to the processor 202.

[0077] The processor 202 may execute the code for the method described above. The processor 203 may obtain the code from memory 201 . The processor 202 may be configured to execute code in a secure environment or in a regular environment. The processor 202 may execute the chemometric model in the secure environment to generate chemometric data. The portable device 200 may further contain a display 204 for collecting user input and display measurement results, for example via a graphical user interface (GUI) Such GUI may be part of an application using the chemometric data and / or the spectroscopic data. The GUI may also be part of the interface of the spectrometer, for example it may be contained in an API provided by the spectrometer which the application can use and / or adjust for its purposes.

[0078] The processor may be operationally coupled with a communication interface 203. The communication interface 203 may be an interface to a communication device capable of transmitting data to and from a telecommunication network and thus enable an internet connection.

[0079] Figure 3 illustrates an embodiment of the present invention. Two processes may be executed on a processor, one in regular environment 310 and one in secure environment 320. The process in regular environment 310 may provide spectroscopic data 311, for example from memory or from a spectrometer or a spectrometer module. The spectroscopic data 311 may be passed via an interface 301 to a process in secure environment 320. The interface 301 may be an inter process communication interface provided by the operating system. The interface 301 may be implemented to secure controlled data flow, i.e. shield the chemometric model from processes executed in the regular environment. The process in secure environment 320 may receive the spectroscopic data 311 and use it as input for a chemometric model 321 . The chemometric model 321 may be stored in a secure memory location to which only processes executed in secure environment 320 have access to. The chemometric model 321 may be encrypted. When executing the chemometric model 321 in secure environment 320, chemometric data 322 are generated. The chemometric data 322 may be passed to a process in regular environment 310 via an interface 301 . The chemometric data 322 may be output on an output interface 312 in regular environment 310, for example on a graphical user interface.

[0080] Figure 4 illustrates another embodiment of the present invention. Two processes may be executed on a processor, one in regular environment 410 and one in secure environment 420 The process in regular environment 410 may submit a measurement request 411 through an interface 401 to the process in secure environment 420. A measurement request 411 may be a function call or setting a flag in a register designated for triggering a measurement. The measurement request 411 may trigger the spectrometer module 421 to execute a measurement and thus provide spectroscopic data 422. The spectrometer module 421 may only be visible to processes in secure environment 420, hence a process in secure environment may need to react to the measurement request 411 and cause the spectrometer module 421 to execute a measurement. The spectroscopy data 422 may be processed by a process in secure environment 420 by using it as input for a chemometric model 423 which generates chemometric data 424. The chemometric data may be passed via interface 401 to a process in regular environment 410 which may output it on an output interface 412, for example a graphical user interface.

[0081] Figure 5 illustrates exemplary components of a processor configured to switch between regular environment and secure environment. The processor may comprise a secure configuration register 511 in which a non-secure (NS) flag may be set to 1 indicating that the processor operates in regular environment 501 or 0 indicating that the processor operates in secure environment 502. The processor may comprise a CPU core 520 comprising a CPU 521 as well as i-cache 522 and D-cache 523. Both i-cache 522 and D-cache 523 may comprise separate memory space in regular environment 501 and secure environment 502. The different memory spaces may have separate memory addressing so they can only be accessed by a process running in the respective mode. The same may apply to L2 cache 531 and RAM 532. Separate address spaces may be realized on the hardware level or by virtual memory implemented in the operating system. The processor may comprise an interrupt controller 533 triggering a switch to a different process and potentially a switch between regular environment 501 and secure environment 502. The processor may comprise controllers for peripheral devices in case the processor is a system on a chip, or the processor may comprise interfaces to peripheral devices, for example a system bus. Accessibility may be restricted to a certain mode. In this example, USB 534 and network 535 may only be accessible from regular environment 501 while spectrometer module 536, graphics 537 and security controller 538 are only accessible from secure environment 502.

[0082] Figure 6 illustrates an exemplary embedding of the process into a software architecture of a portable computation device. An application 610 may send a measurement request 611 to an interface 620. The interface 620 may be an API containing function calls. The request 611 may be a function call provided by the API provided to trigger a measurement of the spectrometer, for example spectrometer.startMeasurementQ. The request 611 may in addition contain parameters, for example measurement variables like the sample time.

[0083] The measurement request 611 may be forwarded to a hardware abstraction layer, HAL 630 which may forward it to a kernel 640. The kernel may forward the measurement request 611 to the spectrometer module 650, for example by using the appropriate spectrometer module driver built into the kernel 640 The spectrometer module 650 may receive the measurement request 611 via its firmware 651. The firmware 651 may trigger the spectroscopy hardware 652 to execute a measurement. Once the measurement completes, the firmware 652 may collect the spectroscopic data 655 and provide it to the kernel 640. The firmware 651 may accomplish that by writing to registers of the spectrometer module 650 which can be read by the spectrometer module driver built into the kernel 640.

[0084] The kernel 640 may pass the spectroscopic data 655 to the HAL 630. The HAL 630 may contain code for a chemometric model for obtaining chemometric data 635 from the spectroscopic data 655. This code may be executed in secure environment of a processor. HAL 630 may pass the chemometric data 635 to the interface 620 which makes it available to the application 610 having sent the measurement request 611.

[0085] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims. Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are per-formed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment / data processing.

[0086] As used herein .determining" also includes .initiating or causing to determine", "generating" also includes .initiating and / or causing to generate" and “providing” also includes “initiating or causing to determine, generate, select, send and / or receive”. “Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0087] In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation. In the claims as well as in the description the word “comprising" or “including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included.

[0088] Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and / or a software module interface. Providing may include communication of data or sub-mission of data to the interface, in particular display to a user or use of the data by the receiving node, entity or interface.

[0089] Various units, circuits, entities, nodes or other computing components may be described as “con-figured to” perform a task or tasks. Configured to shall recite structure meaning “having circuitry that" performs the task or tasks on operation. The units, circuits, entities, nodes or other computing components can be configured to perform the task even when the unit / circuit / component is not operating. The units, circuits, entities, nodes or other computing components that form the structure corresponding to “configured to” may include hardware circuits and / or memory storing program instructions executable to implement the operation. The units, circuits, entities, nodes or other computing components may be described as performing a task or tasks, for convenience in the description. Such descriptions shall be interpreted as including the phrase “configured to.” Any recitation of “configured to” is expressly intended not to invoke 35 U.S.C. § 112(f) interpretation. In general, the methods, apparatuses, systems, computer elements, nodes or other computing components described herein may include memory, software components and hardware components. The memory can include volatile memory such as static or dynamic random-access memory and / or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. The hardware components may include any combination of combinatorial logic circuitry, clocked storage devices such as flops, registers, latches, etc., finite state machines, memory such as static random-access memory or embedded dynamic random-access memory, custom designed circuitry, programmable logic arrays, etc.

[0090] Any disclosure and embodiments described herein relate to the methods, the systems, apparatuses, devices, chemi- cals, materials, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa. All terms and definitions used herein are understood broadly and have their general meaning.

Claims

Claims1 . A method for processing spectroscopic data on a mobile computing device comprising: a. receiving spectroscopic data from a spectrometer module of the mobile computing device, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment on the mobile computing device and c. outputting the chemometric data.

2. The method according to claim 1, wherein the secure environment is backed by hardware-based access control.

3. The method according to claim 1 or 2, wherein the chemometric model is stored in a secure memory section to which only processes running in the secure environment have access.

4. The method according to any of the claims 1 to 3, wherein the spectroscopic data is received from a spectrometer module, wherein the spectrometer module is only visible to processes running in the secure environment.5 The method according to any of the claims 1 to 4, wherein the chemometric data comprises a biomarker.6 The method according to any of the claims 1 to 5, wherein the chemometric model is optimized for one person.7 The method according to any of the claims 1 to 6, wherein an application executed in a regular environment triggers the execution of the chemometric model.

8. The method according to any of the claims 1 to 7, wherein chemometric model is part of the hardware abstraction layer of an operating system of a computer system.

9. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a. receiving spectroscopic data from a spectrometer module of the mobile computing device, b. generating chemometric data by executing a chemometric model using the spectroscopic data as input, wherein the chemometric model is executed in a secure environment on the mobile computing device andc. outputting the chemometric data.

10. A mobile computing device comprising: a. a spectrometer module to generate spectroscopic data, b. a processor configured to generating chemometric data by executing a chemometric model using the spectroscopic data as input and outputting the chemometric data, wherein the chemometric model is executed in a secure environment, and c. an output to output the chemometric data.

11. The mobile computing device according to claim 10, wherein the mobile computing device is a smartphone, a tablet or a wearable.

12. The mobile computing device according to claim 10, wherein the mobile computing device is a device mounted on a vehicle.

13. The mobile computing device according to any of the claims 10 to 12, wherein the chemometric model is stored in a secure memory section to which only processes running in the secure environment have access.

14. The mobile computing device according to any of the claims 10 to 13, wherein the spectrometer module is only visible or accessible to processes running in the secure environment.

15. The mobile computing device according to any of the claims 10 to 14, wherein the spectrometer module comprises a radiation source that generates and / or emits electromagnetic radiation with a wavelength of 1 to 3 pm.

Images