Spatially resolved NIR spectrometer

Abstract

A method is proposed for obtaining at least one item of object information (110) relating to at least one object (112) by spectroscopic measurements, the method comprising the following steps: i. acquiring spectroscopic data (114) using at least one spectrometer device (116) within at least one spatial measurement range (118) of the spectrometer device (116); ii. acquiring image data of a scene (124) within a field of view (126) of the imaging device (120), using at least one imaging device (120), specifically a camera (122), the scene (124) including at least a portion of the object 112 and at least a portion of a spatial measurement range (118) of the spectrometer device (116); and iii. evaluating the spectroscopic data (114) of step i. and at least one item of image information (128) derived from the image data of step ii. to obtain at least one item of object information (110) relating to at least one object (112). Includes.

Description

[Technical field]

[0001] The present invention relates to a method for obtaining at least one item of object information for at least one object by spectroscopic measurements. Furthermore, the present invention relates to a system for obtaining at least one item of object information for at least one object by spectroscopic measurements, as well as a computer program and a computer readable storage medium comprising instructions for carrying out the method. The method and device can be used in particular for obtaining chemical information of an object, in particular information about its chemical composition, and can be used in particular for the analysis of heterogeneous objects. [Background technology]

[0002] Spectroscopy is widely used in research, industrial and consumer applications, enabling multiple applications such as optical analysis and / or quality control. Its applications can be found, for example, in food production and quality control, agriculture, pharmaceuticals, medical applications, life sciences and many other fields. Various methods are available, such as photometry, absorption, fluorescence and Raman spectrometry, allowing the qualitative and / or quantitative analysis of a sample. These methods usually involve obtaining spectroscopic data of an object, also called a sample, by using at least one spectrometer device, in particular comprising at least one wavelength-selecting element and at least one detector device.

[0003] US2019 / 0033210A1 discloses a system and method for qualifying plant material. The system for qualifying plant material includes an inspection zone, a support stage configured to support the plant material in the inspection zone, at least one camera configured to acquire at least one image of the plant material in the inspection zone, at least one processor configured to receive and analyze the camera image and identify an area of ​​interest that includes a specific plant structure having an active ingredient, and at least one spectrometer configured to acquire a spectroscopic measurement of the plant material in the inspection zone. The at least one processor is further configured to facilitate a spectroscopic measurement of the specific plant structure identified in the camera image and to enable outputting an indication of a quality measurement of the plant material based on the spectroscopic measurement of the specific plant structure identified in the camera image.

[0004] US2018 / 172510A1 describes a system for analyzing food in a kitchen appliance for one or more of identifying the food, determining nutritional information of the food, and / or monitoring the food's preparation state. The system includes a spectrometer device integrated with the kitchen appliance, such as an oven, or remote from the kitchen appliance. The system may include a compound parabolic concentrator or focusing lens coupled to a spectrometer module and an illumination module of the appliance. The system may include respective compound parabolic concentrators or focusing lenses coupled to the spectrometer module and illumination module, respectively, for analyzing the food at close range.

[0005] US2016 / 150213A1 provides a method and system for using one or more sensors configured to capture two-dimensional and / or three-dimensional image data of one or more objects. In particular, the method and system combine one or more digital sensors with visible and near-infrared illumination to capture spectral image data in the visible and non-visible ranges of one or more objects. The captured spectral image data can be used to separate and identify the one or more objects. Furthermore, the three-dimensional image data can be used to determine the volume of each of the one or more objects. The identification data and the volume data for the one or more objects can be used individually or in combination to obtain characteristics related to the objects. The method and system provide a user with the ability to capture images of one or more objects and obtain associated characteristics or information for each of the one or more objects.

[0006] US2019 / 026586A1 discloses a portable complete analysis solution that integrates computer vision, spectrometry, and artificial intelligence to provide self-adaptive real-time information and recommendations for objects of interest. The solution has three main components: (1) a mobile device with a camera that captures images of the object and then performs fast computer vision analysis for feature and key element extraction, (2) a portable wireless spectrometer that acquires spectral information of the object's area of ​​interest and then transmits the data (from all built-in sensors) to the mobile device and the cloud, and (3) a sophisticated cloud-based artificial intelligence model that encodes features from the image and chemical information from the spectral analysis to decode the object of interest. This complete solution enables fast, accurate, and real-time analysis, allowing users to obtain clear information about the object of interest as well as personalized recommendations based on that information.

[0007] Spectroscopic methods, such as near infrared (NIR) spectroscopy, and chemometric methods are particularly applicable to obtain the chemical composition of samples, particularly heterogeneous samples, whose chemical composition is highly dependent on the precise location within the sample. Examples of heterogeneous samples include food products, such as fruits and / or vegetables.

[0008] Inhomogeneous samples can present unique challenges for spectroscopic measurements. These measurements may involve measuring multiple spots of the inhomogeneous sample to obtain the average chemical composition of the sample. Alternatively, the sample may be moved, e.g., rotated, during integration of the individual measurements. While both approaches collect more global sample parameters, they typically result in reduced accuracy of the measurements, as information about spatial variations in chemical composition may be lost in the averaging process.

[0009] Thus, despite recent advances in the field of spectroscopic sample analysis, particularly with respect to determining chemical sample composition, challenges remain. In particular, it would be desirable to have means and methods that can obtain accurate spectroscopic data for an object that takes into account local variations and heterogeneity in the object. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] US2019 / 0033210A1 [Patent Document 2] US2018 / 172510A1 [Patent Document 3] US2016 / 150213A1 [Patent Document 4] US2019 / 026586A1 Summary of the Invention [Problem to be solved by the invention]

[0011] It is therefore desirable to provide means and methods that address the above-mentioned technical challenges in the field of spectroscopic sample analysis, in particular, that allow for obtaining accurate spectroscopic data of an object, taking into account possible local variations and inhomogeneities of the object. [Means for solving the problem]

[0012] This problem is addressed by a method for acquiring at least one item of object information for at least one object by spectroscopic measurements, a system for acquiring at least one item of object information for at least one object by spectroscopic measurements, a computer program and a computer-readable storage medium having the features of the independent claims. Advantageous embodiments, which can be realized in separate aspects or in any combination, are set out in the dependent claims as well as in the specification as a whole.

[0013] The terms "having", "comprises" or "including" as used herein, or any grammatical variants thereof, are used non-exclusively. Thus, these terms refer to both the situation where, apart from the features introduced by these terms, no further features are present in the entity described in this context, and the situation where one or more further features are present. As an example, the expressions "A has B", "A comprises B" and "A includes B" refer to both the situation where, apart from B, no other elements are present in A (i.e., A consists solely and exclusively of B), and the situation where, apart from B, one or more further elements are present in the entity A, such as element C, elements C and D, or further elements.

[0014] Furthermore, it should be noted that the terms "at least one," "one or more," or similar language indicating that a feature or element may be present more than one time, are typically used only once when introducing each feature or element. In most cases, when referring to each feature or element, the language "at least one" or "one or more" will not be repeated, despite the fact that each feature or element may be present more than one time.

[0015] Furthermore, as used herein, the terms "preferably", "more preferably", "particularly", "more particularly", "particularly", "more particularly" or similar terms are used in connection with any feature without limiting the possibility of alternatives. Thus, features introduced by these terms are optional features and are not intended to limit the scope of the claims in any way. The invention can be practiced with alternative features, as will be recognized by those skilled in the art. Similarly, features introduced by "in one embodiment of the invention" or similar expressions are intended to be optional features, without any limitation regarding alternative embodiments of the invention, without any limitation regarding the scope of the invention, and without any limitation regarding the possibility of combining the feature so introduced with other optional or non-optional features of the invention.

[0016] In a first aspect of the invention, a method for obtaining at least one item of object information for at least one object by spectroscopic measurements is disclosed. The method comprises the following method steps, which are specifically performed in a given order. However, different orders are possible. The method may further comprise additional method steps not listed. Furthermore, one or more or all of the method steps may be performed only once or repeatedly.

[0017] The method comprises the steps of: i. acquiring spectroscopic data, in particular spectroscopic data of at least one object, using at least one spectrometer device and within at least one spatial measurement range of said spectrometer device; ii. using at least one imaging device, specifically a camera, to acquire image data of a scene within a field of view of the imaging device, the scene including at least a portion of the object and at least a portion of a spatial measurement range of the spectrometer device; and iii. evaluating the spectroscopic data of step i. and at least one item of image information derived from the image data of step ii. to obtain at least one item of object information relating to the at least one object. Includes.

[0018] The term "spectrometry" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, obtaining spectroscopic data on at least one object. The spectroscopic data is specifically obtained using at least one spectrometer device. As part of the spectroscopic measurement, the object is illuminated with electromagnetic radiation in the infrared spectral range, specifically in the near infrared spectral range. In particular, the electromagnetic radiation may be in the wavelength range of 760 nm to 1000 μm, specifically in the wavelength range of 760 nm to 15 μm, more specifically in the wavelength range of 1 μm to 5 μm, more specifically in the wavelength range of 1 μm to 3 μm. Electromagnetic radiation is also referred to as light, and the two terms are used interchangeably herein. The spectroscopic measurement may further include receiving the incident light after interaction with the object, and generating at least one corresponding signal that may form part of the spectroscopic data. The spectroscopic data may include information about at least one optical or optically measurable property of the object, determined for one or more different wavelengths as a function of wavelength. More specifically, the spectroscopic data may relate to at least one property characterizing at least one of the transmission, absorption, reflection and emission of the object. The at least one optical property is determined for one or more wavelengths. The spectroscopic data may be in the form of a signal intensity, determined in particular as a function of the wavelength of the spectrum or a division thereof, for example a wavelength interval, which signal intensity is preferably provided as an electrical signal that can be used for further evaluation. Thus, the spectroscopic data is generated as part of the spectroscopic measurement.

[0019] The term "acquiring spectroscopic data" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, any process of capturing, recording, and / or storing spectroscopic data with a spectrometer device by, for example, measuring at least one of transmission, absorption, reflection, and emission of an object as a function of wavelength for one or more different wavelengths.

[0020] The term "spectrometer device" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to a special or customized meaning. The term specifically refers to, but is not limited to, an instrument configured to obtain spectroscopic data of at least one object in at least one spatial measurement range. The spectrometer device used in step i. may specifically be a near-infrared spectrometer device. Thus, the spectrometer device may specifically be configured to detect electromagnetic radiation in the near-infrared range. The spectrometer device may be configured to perform at least one spectroscopic measurement on the object. The spectrometer device may specifically comprise at least one detector device comprising at least one optical element and a number of photosensitive elements. The at least one optical element may specifically be configured to separate the incident light, specifically electromagnetic radiation in the near-infrared range, into a spectrum of constituent wavelength components. Each photosensitive element may be configured to receive at least a portion of one of the constituent wavelength components and to generate a respective detector signal in response to illumination of the respective photosensitive element by at least a portion of the respective constituent wavelength components. The detector signals, in particular the signal intensities, together with the corresponding wavelengths form part of the spectroscopic data. The spectrometer device may be or may comprise a dispersive spectrometer device, for example as described above, for analyzing the radiation of an object illuminated with broadband illumination. However, additionally or alternatively, further configurations and / or arrangements of the spectrometer device are feasible. By way of example, the object may be illuminated with light of a limited number of different wavelengths and the spectrometer device may comprise a broadband detector. In particular, the spectrometer device may be a Fourier transform spectrometer, in particular a Fourier transform infrared spectrometer. Thus, a narrowband light source, for example at least one light emitting diode (LED) and / or at least one laser, may be used to illuminate the object. In particular, the spectrometer device may be configured to determine the spectrum by measuring and processing an interferogram, in particular by applying at least one Fourier transform to the measured interferogram. The spectrometer device may in particular be embodied as a portable spectrometer device.In particular, the spectrometer device may be part of a mobile device such as a laptop, a tablet, or in particular a mobile phone, e.g., a smartphone. Additionally or alternatively, the mobile device may be or comprise a smart watch and / or a wearable computer, also called a wearable, e.g., a body mounted computer. Further mobile devices are also possible. The spectrometer device may be integrated into and / or attachable to the mobile device.

[0021] The term "spatial measurement range" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to a special or customized meaning. The term specifically refers to, but is not limited to, a spatially limited section that is spectroscopically examined by a spectrometer device. As an example, the spatial measurement range is defined as a solid angle or a three-dimensional angular segment in space, and an object placed within the solid angle or angular segment is analyzed by the spectrometer device. As an example, the solid angle or angular segment is defined by the geometrical and / or optical properties of the spectrometer device. Thus, the spatial measurement range may be the field of view of the spectrometer device within which the spectroscopic measurement is performed. Thus, an object or a part of an object placed within the spatial measurement range is accessible for spectroscopic analysis by the spectrometer device. In particular, the spectrometer device may be configured to obtain spectroscopic data based on incident light from within the spatial measurement range. The spatial measurement range may in particular be a three-dimensional space section, such as a cone-shaped space section, the light content of which is received and analyzed by the spectrometer device. The spectroscopic data acquired by the spectrometer device may comprise information related to at least one object located within a spatial measurement range of the spectrometer device. In particular, for spectroscopically analyzing the object, the spectrometer device may be positioned proximate to the object such that the spatial measurement range at least partially includes the object, for example at a distance in the range of 0 mm to 100 mm from the object, in particular in the range of 0 mm to 15 mm.

[0022] The term "imaging device" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, any device configured to record or capture image data and / or capture 2D or 3D spatial information regarding at least one object and / or scene. The imaging device may be or may comprise at least one camera having one or more imaging sensors, specifically one or more CCD or CMOS imaging sensors, for acquiring image data. The camera may specifically comprise at least one camera chip, for example at least one CCD chip and / or at least one CMOS chip, configured to record images. The camera may comprise a one-dimensional or two-dimensional array of imaging sensors, for example pixels, arranged on the camera chip. As an example, the camera may comprise at least 100 pixels in at least one dimension, for example at least 100 pixels in each dimension. As an example, the camera may comprise an array of imaging sensors comprising at least 100 imaging sensors in each dimension, specifically at least 300 imaging sensors in each dimension. For example, the camera may be a color camera comprising color pixels, each color pixel comprising at least three color sub-pixels sensitive to different colors. For example, the camera may comprise black and white and / or color pixels. Color and black and white pixels may be combined inside the camera. The camera may be a camera of a mobile device. The invention is particularly applicable to cameras typically used in mobile devices such as laptops, tablets or in particular mobile phones, for example smartphones. Thus, in particular, the camera may be part of a mobile device comprising, besides at least one camera, one or more data processing devices, such as one or more processors. The mobile device may in particular have at least one function different from the spectroscopic function, for example a mobile communication function, such as a mobile phone function. However, other cameras are also feasible. As outlined above, the spectrometer device may be part of a mobile device.In particular, both the camera and the spectrometer device may be part of a mobile device, in particular a smartphone. The camera may comprise, besides at least one camera chip or imaging chip, one or more further optical elements, such as one or more lenses. As an example, the camera may be a fixed focus camera with at least one lens fixedly adjusted relative to the camera. Alternatively, however, the camera may comprise one or more variable lenses that can be adjusted automatically or manually. Alternatively or additionally, the imaging device may be or comprise at least one LIDAR-based imaging device. This LIDAR stands for Light Detection and Ranging or Light Imaging, Detection and Ranging. The LIDAR-based imaging device may comprise at least one laser source, for example at least one tunable laser diode, for illuminating the object or at least a part of the object. The LIDAR-based imaging device may further comprise at least one localization unit configured to determine at least one distance of the illuminated part of the object from the imaging device and / or from at least one further point or position in space. The localization unit may in particular comprise at least one sensor element, e.g. a photodiode, configured to detect at least one laser beam emitted by the laser light source and reflected by the object. The determination of the distance, and thus the generation of image data, e.g. as described in more detail below, may comprise processing the light beam reflected by the object and / or at least one reference light beam and / or a corresponding signal detected by the at least one sensor element.

[0023] The term "image data" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, spatially resolved one-dimensional, two-dimensional, or even three-dimensional optical information. The image data may include a plurality of electronic readings from an imaging device, for example from the pixels of an imaging sensor, for example a camera chip, and / or from the sensor elements of a LIDAR-based imaging device. In particular, the image data may include a plurality of numerical values ​​corresponding to the electronic readings from the imaging device. The electronic readings, in particular the numerical values, relate to at least one optical characteristic of at least one object in the field of view of the imaging device. Thus, the image data may include at least one array of information values, such as grayscale values ​​and / or color information values. Alternatively or additionally, the information values ​​included in the image data may include distance values, each indicating a distance between a portion of the object and at least one reference point, such as an imaging device, in particular a LIDAR-based imaging device.

[0024] The term "acquiring image data" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art and is not limited to any special or customized meaning. The term specifically refers, but is not limited to, any process of capturing or recording image data by an imaging device, specifically a camera, for example in the form of an electronic reading generated by an imaging sensor in response to illumination.

[0025] The term "field of view" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically, but not exclusively, refers to a spatially limited section, the contents of which are imaged by an imaging device. Specifically, image data generated by an imaging device may include spatially resolved optical information related to objects located within the field of view of the imaging device. The field of view may specifically be a three-dimensional spatial section accessible to the imaging device. Specifically, the scene that the field of view includes is imaged by the imaging device.

[0026] The term "scene" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to a special or customized meaning. The term specifically refers to, but is not limited to, the optical content of the field of view of the imaging device. Specifically, the scene may include one or more objects, such as the objects mentioned with respect to step i. above, and at least one object in the scene may be imaged by the imaging device. Thus, the scene may specifically include a plurality of objects having a particular arrangement, and the objects and their arrangement may be imaged by the imaging device, thereby generating at least one image. As part of method step ii., image data of the scene in the field of view of the imaging device is acquired, the scene including at least a portion of the object and at least a portion of the spatial measurement range of the spectrometer device. The object of step i. may be at least partially visible in the image data of step ii. Thus, the field of view of the imaging device and the spatial measurement range of the spectrometer device may at least partially overlap. The spatial relationship between the field of view of the imaging device and the spatial measurement range of the spectrometer device may be known and may be used, for example, in step iii., e.g., an offset between the field of view of the imaging device and the spatial measurement range of the spectrometer device and / or at least one angle between the field of view of the imaging device and the spatial measurement range of the spectrometer device. Thus, an arrangement and / or an object in the field of view of the imaging device may be located within the spatial measurement range of the spectrometer device or vice versa. In particular, at least one object or at least a part thereof may thus be located both in the field of view of the imaging device and in the spatial measurement range of the spectrometer device. Thus, at least one object or at least a part thereof is not only spectroscopically examined by the spectrometer device but also at least partially imaged by the imaging device.

[0027] The term "image information" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, any item of information derived from image data acquired by an imaging device. Specifically, step iii. of the method may further include deriving at least one item of image information from the image data of step ii. The item of image information may be or may comprise at least one item of spatial information relating to a spatial measurement range of the spectrometer device, for example information relating to a position or location of the spatial measurement range within a scene imaged by the imaging device. Additionally or alternatively, the item of image information may be or may comprise at least one item of identification information relating to at least one object, specifically identification information relating to at least one of the following: object type, specifically identification information relating to at least one of the following: object type, object boundary within a scene, object size, object orientation, location, tilt. A wide variety of items of image information may be derived from image data and are outlined in more detail below.

[0028] The term "object" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to any item, such as a living or inanimate object, that is accessible to be imaged by an imaging device, as well as to be spectroscopically examined by a spectrometer device, but is not limited to this. In particular, the object may be a heterogeneous object, for example an object whose chemical composition may vary within the object, for example depending on the position. However, other objects are also feasible, in particular homogeneous objects whose chemical composition varies little or not at all. The object may specifically be or comprise a food product, such as a fruit or vegetable, or a body part, such as skin.

[0029] The term "item of object information" as used herein is a broad term and is given its ordinary and customary meaning to a person skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to any item of information relating to at least one property of an object, such as at least one of chemical, physical and biological properties, such as the material and / or composition of the object, but is not limited thereto. The item of object information is specifically determined taking into account at least one item of image data of the object, in particular image information derived from the image data of method step ii, as well as spectroscopic data of the object. The item of object information specifically relates to a property that can vary within the object, which property may be characteristic of a particular location or spatial range within the object. However, the property may also show no or only slight variation throughout the object. The item of object information describes the property in a qualitative and / or quantitative way, for example by one or more numerical values. In particular, the item of object information may include chemical information of the object, in particular the chemical composition. An item of object information may include not only information about the property, but also spatial information about the particular location or spatial area within the object where the property was measured.

[0030] The term "obtain at least one item of object information" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to any process of determining at least one item of object information, but is not limited to it. In particular, to determine the item of object information, the spectroscopic data of step i. and at least one item of image information derived from the image data of step ii. are taken into account. In method step iii., the spectroscopic data of step i. and the at least one item of image information derived from the image data of step ii. are evaluated to obtain at least one item of object information relating to at least one object. In particular, the evaluated spectroscopic data and the evaluated item of image information are combined or connected, for example in a predetermined manner and / or according to a predetermined algorithm, to obtain at least one item of object information. Examples are given below.

[0031] The terms "evaluation of data" and "evaluation of information" as used herein in "evaluation of spectroscopic data" and "evaluation of image information" are broad terms and should be given their ordinary and customary meaning to those skilled in the art and should not be limited to any special or customized meaning. The terms specifically refer to any process of analyzing the respective information of the data, for example, but not limited to, by applying at least one analysis step, for example, an analysis step including at least one analysis algorithm applied to the data and / or information. Specifically, the data or information is processed and / or interpreted and / or evaluated as part of the analysis step, for example, by comparing the data or information, or at least a subset thereof, to at least one predefined value or by identifying at least one global or local maximum or minimum. As an example, the evaluation of the spectroscopic data may include analyzing the spectroscopic data to determine at least one peak in the spectroscopic data that reflects a global or local maximum of transmission, absorption, reflection and / or emission of the object. The evaluation of the spectroscopic data may further include identifying at least one corresponding wavelength. The evaluation of the spectroscopic data may further comprise determining a chemical composition of the object, for example by comparing the identified peaks with at least one predefined peak or at least one predefined set of peaks. The evaluation of the spectroscopic data is in particular carried out using at least one spectroscopic evaluation algorithm. The result of the evaluation of the spectroscopic data is also referred to as spectroscopic object information. By way of example, the evaluation of the item of image information may comprise analysing the item of image information, for example using at least one identification algorithm, in particular at least one object recognition algorithm, as outlined in more detail below.

[0032] Method step iii. further comprises deriving at least one item of image information from the image data of step ii. The term "deriving image information from image data" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art and is not limited to any special or customized meaning. This expression specifically, but without limitation, refers to determining at least one item of image information based on image data acquired by an imaging device in step ii. In particular, the at least one item of image information is at least one of the following: - at least one image derived from the image data of step ii.; - at least one item of spatial information relating to a spatial measurement range in the scene, in particular an indication of the spatial measurement range in the image for which the spectroscopic data was acquired; - at least one item of identification information for at least one object, in particular identification information for at least one of the following: type of object, boundary of the object in the scene, size of the object, location of the object, color of the object, texture of the object, shape of the object, contrast of the object, volume of the object, area of ​​interest of the object; - at least one item of positioning information relating to at least one object, in particular an indication of the positioning of the spectrometer device with respect to the at least one object, - at least one item of directional information, in particular an indication of the direction between the spectrometer device and at least one object; - at least one item of similarity information about the object, in particular similarity information about at least one shared characteristic shared between different regions of the object; may include:

[0033] The term "image" as used herein is a broad term and is given its ordinary and customary meaning to a person skilled in the art, and is not limited to a special or customized meaning. The term specifically refers to, but is not limited to, any representation, such as a one-dimensional, two-dimensional or three-dimensional representation, of at least one optically detectable property of a sample. In particular, the image may include a graphical representation of a scene in the field of view of an imaging device. The image may specifically be displayed, for example, on a display device, such as a screen of a mobile device (e.g. a mobile device that may be equipped with an imaging device). The image may specifically include and / or be derived from the image data or a part thereof referred to in step ii. The image may specifically represent at least one visual property of the sample.

[0034] In particular, the at least one item of image information may comprise at least one image derived from the image data of step ii., steps i. and ii. may be performed iteratively, and the at least one item of object information in step iii. may comprise a combination of spectroscopic object information derived from the iteration of step i. and at least one item of spatial information relating to a spatial measurement range in the scene derived from the iteration of step ii. The method may further comprise displaying at least one of the spatial information and the spectroscopic object information in the image. Thus, by way of example, the image may comprise information relating to the acquisition location of the spectroscopic data and / or the results of the evaluation of the spectroscopic data, e.g. compositional information derived from the spectroscopic data. In this way, the image visually shows the scene or a part thereof and information derived from the spectroscopic data acquired in step i., optionally together with positional information relating to the acquisition location of the information. In this way, the image comprises an overlap of at least one object visible in the scene with one or more locations at which one or more spectroscopic measurements have been performed, and may optionally comprise the results of the spectroscopic measurements and / or one or more items of information derived from the spectroscopic measurements.

[0035] Between possible repetitions of steps i. and ii., the scene, the field of view, and / or the object may be modified. Thus, by way of example, the scene may change and / or at least one of the spectrometer device, the imaging device, and the device including both the spectrometer device and the imaging device described above, e.g., a mobile device, may move.

[0036] In particular, the method generates at least one image of a scene having at least two items of spectroscopic object information and corresponding spatial information for each item of spectroscopic object information relating to a spatial measurement range within the image. Furthermore, the image derived from the image data of step ii. may be an image derived from the image data of the iterations of step ii., and in particular may be at least one of a combination image and a selected image of the images derived from the image data of the iterations of step ii.

[0037] As an example, as mentioned above, the imaging device and / or the spectrometer device may move during, in particular during, any repetition of steps i. and ii. In particular, in a first execution of step ii., image data of a first scene is acquired at a first distance, and for each repetition of step ii., the imaging device and / or the spectrometer device moves closer to the object so that the imaged scene is a subsection of said first scene. In particular, image data corresponding to a wide image is acquired in the first execution of step ii. The wide image may completely or almost completely include the object. For further repetitions of step ii., the distance of the imaging device and / or the spectrometer device to the object may be reduced to at least one second distance, which second distance allows acquisition of spectroscopic data of the object by executing step i. The second distance may be in the range of 0 mm to 100 mm, in particular 0 mm to 15 mm. The image derived from the image data acquired at the second distance shows a subsection of the image derived from the image data acquired in the first execution of step ii. The method may further include tracking the movement of the imaging device, e.g., from a first distance to at least one second distance, using the imaging device and the motion tracking software. In particular, a spatial relationship may be inferred between image data and / or spectroscopic data acquired at at least one second distance and an image acquired at the first distance. An item of object information may connect spectroscopic object information, e.g., a chemical composition determined using the spectroscopic data, with an item of spatial information identifying a location of an object in the image, for which the spectroscopic object information is valid. The location of the object is identified in the image by at least one graphical representation, e.g., an arrow pointing to the location, or by a circle, square, or other type of representation surrounding or marking the location. One or more such locations may be marked in the image to indicate the corresponding spectroscopic object information.

[0038] As a further example, the imaging device and / or spectrometer device may move across the object, e.g., along a scan path, at a fixed distance and / or a variable distance while performing one or more repetitions of steps i. and ii. By performing step iii., at least one item of object information is obtained, which may include a plurality of items of chemical information corresponding to a plurality of locations along the scan path. Again, e.g., in a first performance of step ii., image data of the object is acquired, and the scan path is included in an image derived from the image data. In particular, the scan path and / or the spectroscopic object information, in particular the chemical information, may be shown in the image. This allows for the retrieval of chemical information along the scan path.

[0039] As a further example, the item of image information may include at least one item of identification information relating to at least one object, in particular identification information relating to at least one of the following: type of object, boundary of the object in the scene, size of the object, location of the object, color of the object, texture of the object, shape of the object, contrast of the object, volume of the object, area of ​​interest of the object. The item of identification information may be derived in particular by using at least one identification algorithm, for example an image recognition algorithm and / or a trained model configured to recognize or identify the object, for example by using an artificial intelligence, for example an artificial neural network. In particular, the at least one item of image information may include at least one item of identification information relating to the at least one object, the method comprising applying at least one identification algorithm to the at least one item of image information in order to derive at least one item of identification information from the at least one item of image information. The identification algorithm may in particular include at least one object recognition algorithm for determining the type of the at least one object. For example, the object recognition algorithm can identify the type of object, e.g. the category or type of object where the object is an apple, orange or other type of fruit or vegetable, a human body part, e.g. a hand or a face. Further types of objects are also possible, in particular further types of food objects. In particular, step iii. comprises applying at least one spectroscopic evaluation algorithm to the spectroscopic data of step i., the spectroscopic evaluation algorithm being selected according to the item of identification information, in particular according to the at least one type of object.

[0040] The method may in particular comprise providing a plurality of spectroscopic evaluation algorithms for different items of identification information, in particular for different types of objects. Thus, depending on the type of object determined by the identification algorithm, a corresponding spectroscopic evaluation algorithm is selected and information determined from the image data is subsequently used for the evaluation of the spectroscopic data. As an example, the item of image information may comprise item identification information identifying that the object for which the spectroscopic data was acquired is an apple. In this way, the spectroscopic data is evaluated using a spectroscopic evaluation algorithm optimized for the evaluation of apples. Using an application-specific spectroscopic evaluation algorithm can improve the accuracy of the evaluation result (e.g. the chemical composition of the object) and / or speed up the evaluation process.

[0041] Additionally or alternatively, the item of image information may include identifying information regarding the size of the object. Thus, the image information may include identifying information regarding both the type and size of the object. Different items of identifying information may be combined to generate added value. As an example, an object may be identified as an apple and the size of the apple may be derived from the image data. Based on these items of information, an estimated mass of the apple may be determined. This information may be combined with the chemical composition determined by evaluating the spectroscopic data to obtain an item of object information to infer at least one item of nutritional information, such as nutritional value per portion.

[0042] The item of image information may include identifying information, for example identifying information relating to at least one region of interest of the object. The region of interest is so identified, for example by an image recognition algorithm and / or a trained model. By way of example, the region of interest may be or may include an irregularity and / or an unexpected feature. Further regions of interest are possible. The region of interest may for example be a mole on an extension of human skin, such as a hand or a leg. The method provides for providing the user with at least one item of guidance information, for example on a display of the mobile device, indicating the region of interest. The item of guidance information prompts the user in particular to perform step i) of the method on the region of interest. Using the application-specific spectroscopic evaluation algorithm, specific information, for example medical information and / or medical guidance, for example cancer diagnosis information on a mole, may be provided to the user regarding the region of interest.

[0043] The item of image information may include at least one item of similar information about the object, in particular similar information about at least one shared characteristic shared between different regions of the object. In particular, the item of image information may include information about different regions of the object that share at least one common characteristic. The characteristic may be a quality identified in the image data, in particular an image. The shared characteristic may for example be a common color shared between different regions of the object, while further regions of the object exhibit a different color. A shared characteristic identified in the image data, e.g. similar image information, may imply shared and / or similar spectroscopic data, e.g. similar spectral information. The method may include predicting spectroscopic data and / or at least spectroscopically derivable characteristics for regions of the object in which at least one characteristic of the image data is similar to each other. The method may further include checking and / or refining the prediction, for example by guiding the user to obtain spectroscopic data for further regions with the shared characteristic. By way of example, the object may be an apple that includes regions of different colors. As part of the method, regions sharing the color red are identified as items of similar information. The spectroscopic data acquired for one of the regions may indicate a particular sugar content, e.g. a sugar content that exceeds the sugar content of a further region of a different colour, e.g. green. As part of the method, the sugar content of the further red region may be predicted. Additionally, the user may be guided to acquire spectroscopic data for further red regions to check and / or refine the prediction and / or possible further predictions.

[0044] Step iii) of the method may further comprise taking into account information of at least one further sensor when obtaining the at least one item of object information. The further sensor information may for example comprise gyroscope information and / or GPS information. The further sensor, in particular the gyroscope, may be part of the mobile device. Additionally or alternatively, the further sensor information may be provided by the mobile device, such as for example GPS information. The further sensor information may for example be taken into account by checking, verifying or evaluating the item of image information.

[0045] The method may be at least partially computer-implemented, specifically step iii. The computer-implemented steps and / or aspects of the present invention are particularly implemented using a computer or a computer network. As an example, step iii. of the method is fully or partially computer-implemented. Thus, the evaluation of the spectroscopic data is particularly performed using at least one spectroscopic evaluation algorithm. The evaluation of the item of image information may include analyzing the item of image information, for example using at least one identification algorithm. The evaluated spectroscopic data and the evaluated item of image information are combined or connected, for example in a predetermined manner and / or according to a predetermined algorithm, to obtain at least one item of object information. The at least one spectroscopic evaluation algorithm may particularly include at least one trained model. The term "trained model" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to a special or customized meaning. The term particularly refers to a mathematical model trained on at least one training data set using one or more of, but not limited to, machine learning, deep learning, neural networks, or other forms of artificial intelligence.

[0046] The method may further comprise providing at least one item of object information relating to the at least one object, in particular optically providing at least one item of object information relating to the at least one object via a display device, in particular the item of object information may be displayed, for example, on a screen of a display device, for example a mobile device (e.g. a mobile device which may comprise an imaging device and / or a spectrometer device).

[0047] In a further aspect of the present invention, a system for obtaining at least one item of object information relating to at least one object by spectroscopic measurement is disclosed, the system comprising: I. at least one spectrometer device configured to acquire spectroscopic data within at least one spatial measurement range of the spectrometer device; II. At least one imaging device, specifically a camera, configured to acquire image data of a scene within a field of view of the imaging device, the scene including at least a portion of an object and at least a portion of a spatial measurement range of the spectrometer device; and III. At least one evaluation unit, configured to evaluate spectroscopic data acquired by the spectrometer device and at least one item of image information derived from image data acquired by the imaging device to obtain at least one item of object information relating to the at least one object. Includes.

[0048] The system for obtaining at least one item of object information is in particular used for carrying out the method for obtaining at least one item of object information according to the invention according to any one of the embodiments described above and / or according to any one of the embodiments further described below. Thus, with regard to terms and definitions, reference is made to the description of the method for obtaining at least one item of object information given above.

[0049] The term "system" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, a set or assembly of interacting components that may interact to perform at least one common function. At least two components may be treated independently or may be coupled or connectable.

[0050] The term "evaluation unit" as used herein is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term specifically refers to, but is not limited to, any functional element configured to analyze and / or process data. The evaluation unit is specifically configured to analyze items of spectroscopic data and / or image data, specifically image information. The evaluation unit specifically processes and / or interprets and / or evaluates the data and / or information as part of the analysis process. The evaluation unit includes, in particular, at least one processor. The processor is specifically configured to perform one or more evaluation operations on the data and / or information, such as by software programming. The term "processor", as generally used herein, also referred to as "processing unit", is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term specifically refers to, but is not limited to, any logic circuitry configured to perform the basic operations of a computer or system, and / or generally, a device configured to perform calculations or logical operations. In particular, the processing unit may be configured to process basic instructions that drive a computer or system. As an example, the processing unit may include at least one arithmetic logic unit (ALU), at least one floating point unit (FPU), such as a math coprocessor or numeric coprocessor, a number of registers, in particular registers configured to provide operands to the ALU and store operation results, and memories, such as L1 cache memory and L2 cache memory. In particular, the processing unit may be a multi-core processor. In particular, the processing unit may be or include a central processing unit (CPU). Additionally or alternatively, the processing unit may be or include a microprocessor, and thus in particular the elements of the processing unit may be contained in one integrated circuit (IC) chip.Additionally or alternatively, the processing unit may be or may include one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs). The processing unit may be specifically configured, such as by software programming, to perform one or more evaluation operations.

[0051] The imaging device may comprise at least one camera with one or more imaging sensors, in particular one or more CCD or CMOS imaging sensors, for acquiring image data of the scene. The imaging device may in particular comprise a one-dimensional or two-dimensional array of imaging sensors, e.g. pixels, which may be arranged on a camera chip. Additionally or alternatively, the imaging device may be or comprise at least one LIDAR-based imaging device. The LIDAR-based imaging device may comprise at least one laser light source for illuminating the object, and at least one localization unit. The localization unit may comprise at least one sensor element configured to detect at least one laser beam emitted from the laser light source and reflected by the object. The localization unit may be configured to determine at least one distance of the illuminated portion of the object from at least one reference point. The determination of the distance, and thus the generation of the image data, may comprise processing the light beam reflected by the object, and / or the at least one reference light beam, and / or a corresponding signal detected by the at least one sensor element. For further options and / or details of the options, reference is made to the description of the imaging device given above.

[0052] The spectrometer device may include at least one detector device including at least one optical element and a plurality of photosensitive elements, the at least one optical element being configured to separate the incident light into a spectrum of constituent wavelength components, each photosensitive element being configured to receive at least a portion of one of the constituent wavelength components and configured to generate a respective detector signal in response to illumination of the respective photosensitive element by at least a portion of the respective constituent wavelength components. Thus, the spectrometer device analyzes the incident light after interaction with the object and generates at least one corresponding detector signal that may form part of the spectroscopic data. The optical element may include at least one wavelength-selective element. The wavelength-selective element may in particular be selected from the group consisting of a prism, a diffraction grating, a linear variable filter, an optical filter, in particular a narrow band-pass filter. The detector device may further include a plurality of photosensitive elements arranged in a linear array, the array of photosensitive elements including a number between 10 and 1000, in particular a number between 100 and 500, in particular a number between 200 and 300, more particularly a number of 256 photosensitive elements. Each photosensitive element is in particular selected from the group consisting of a pixelated inorganic camera element, in particular a pixelated inorganic camera chip, more particularly a CCD chip or a CMOS chip, a monochrome camera element, in particular a monochrome camera chip, at least one photoconductor, in particular an inorganic photoconductor, more particularly an inorganic photoconductor comprising Si, PbS, PbSe, Ge, InGaAs, ext.InGaAs, InSb or HgCdTe. Each photosensitive element may be sensitive to electromagnetic radiation in the wavelength range of 760 nm to 1000 μm, in particular in the wavelength range of 760 nm to 15 μm, more particularly in the wavelength range of 1 μm to 5 μm, more particularly in the wavelength range of 1 μm to 3 μm. The spectrometer device may be or comprise a dispersive spectrometer device, for example as described above, capable of analyzing the radiation of an object illuminated with broadband illumination. However, further configurations and / or arrangements of the spectrometer device are possible, which in particular affect its components, for example the detector and / or the illumination source used. As an example, the object may be illuminated with a limited number of different wavelengths of light. The spectrometer device may include a broadband detector.In particular, the spectrometer device may be a Fourier transform spectrometer, in particular a Fourier transform infrared spectrometer. Thus, a narrowband light source, such as at least one light emitting diode (LED) and / or at least one laser, may be used to illuminate the object. In particular, the spectrometer device may be configured to determine the spectrum by measuring and processing an interferogram, in particular by applying at least one Fourier transform to the measured interferogram.

[0053] The spectrometer device and the imaging device may have a known, particularly fixed, arrangement relative to each other. In particular, the spectrometer device and the imaging device may have a known, particularly fixed, spatial relationship relative to each other. Furthermore, the spatial measurement range of the spectrometer device and the field of view of the imaging device may have a fixed spatial relationship with respect to each other.

[0054] The system may further include at least one light source configured to emit electromagnetic radiation in a wavelength range of 760 nm to 1000 μm, specifically in a wavelength range of 760 nm to 15 μm, more specifically in a wavelength range of 1 μm to 5 μm, more specifically in a wavelength range of 1 μm to 3 μm. The spectrometer device is specifically referred to as a "near infrared spectrometer device".

[0055] The evaluation unit of the system is configured to obtain at least one item of object information relating to the at least one object. The system may include at least one display device configured to provide at least one item of object information relating to the at least one object. The system may further include at least one mobile device, the mobile device comprising at least one spectrometer device and at least one imaging device. Thus, the spectrometer device and the imaging device, e.g. at least one camera, may both be integrated into a mobile device, e.g. a smartphone. The term "mobile device" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art and is not limited to a special or customized meaning. The term specifically refers to, but is not limited to, a mobile electronic device, more specifically a mobile communication device such as a mobile phone or a smartphone. Additionally or alternatively, the mobile device refers to a tablet computer or other type of portable computer having at least one camera. The mobile device in particular has at least one display device, in particular a screen configured to display the items of object information.

[0056] The system may further include at least one control unit. The term "control unit" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to a special or customized meaning. The term specifically refers to a device or combination of devices having the capability and / or configuration to perform at least one computing operation and / or to control at least one function of at least one other device, for example at least one other component of the system for acquiring at least one item of object information, but is not limited to this. The control unit specifically controls at least one function of the spectrometer device, for example the acquisition of spectroscopic data. The control unit specifically controls at least one function of the imaging device, for example the acquisition of image data. The control unit specifically controls an evaluation unit, for example the evaluation of the spectroscopic data and / or at least one item of image information. In particular, the at least one control unit may be embodied as and / or comprise at least one processor, which may be configured to perform one or more operations, in particular by software programming. The term "processor" as used herein is a broad term and is given its ordinary and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term specifically refers to, but is not limited to, any logic circuitry configured to perform the basic operations of a computer or system, and / or generally refers to a device configured to perform calculations or logical operations. In particular, the processor may be configured to process the basic instructions that run the computer or system. As an example, the processor may include at least one arithmetic logic unit (ALU), at least one floating point unit (FPU), such as a math coprocessor or numeric coprocessor, a number of registers, specifically registers configured to provide operands to the ALU and store the results of the operations, and memories, such as an L1 cache memory and an L2 cache memory. In particular, the processor may be a multi-core processor.In particular, the processor may be or include a central processing unit (CPU). Additionally or alternatively, the processor may be or include a microprocessor, and thus in particular the elements of the processor may be contained in one integrated circuit (IC) chip. Additionally or alternatively, the processor may be or include one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs) and / or one or more tensor processing units (TPUs) and / or one or more chips, such as dedicated machine learning optimization chips, or otherwise. The processor may be specifically configured, such as by software programming, to perform and / or execute one or more evaluation operations.

[0057] In a further aspect, a computer program is disclosed, which comprises instructions for causing the system to perform a method disclosed herein, e.g. according to any one of the embodiments described above and / or according to any one of the embodiments described in more detail below, when the program is executed by a control unit of the system. Thus, as an example, the computer program may provide instructions for causing the system to acquire or activate the acquisition of spectroscopic data by using a spectrometer device according to step i., for causing the system to acquire or activate the acquisition of image data of the scene by using an imaging device according to step ii., and for causing the system to perform the evaluation according to step iii. The computer program may in particular comprise instructions for causing the system to perform step iii. of the method. The computer program may further include instructions for causing the system to perform steps i. and ii. of the method, either by its own operation or in response to at least one user action (such as a user interaction such as pressing a start button) that initiates, for example, the acquisition of spectroscopic data in step i. and / or the acquisition of image data in step ii. The computer program may also include instructions for the system to prompt the user to provide certain inputs. Thus, by way of example, the user is prompted to initiate the acquisition of spectroscopic data in step i. and / or is prompted to initiate the acquisition of image data in step ii.

[0058] In a further aspect, a computer readable storage medium is disclosed, which comprises instructions which, when executed by a control unit of a portable spectrometer device disclosed herein, cause the control unit to perform a method disclosed herein, e.g. according to any one of the above-mentioned embodiments and / or according to any one of the embodiments described in more detail below. The term "computer readable storage medium" as used herein specifically refers to a non-transitory data storage means, e.g. a hardware storage medium on which computer executable instructions are stored. A computer readable data carrier or storage medium may specifically be or comprise a storage medium such as a random access memory (RAM) and / or a read only memory (ROM).

[0059] The method of acquiring at least one item of object information and the system of acquiring at least one item of object information disclosed herein provide many advantages over known devices and methods of similar type. In particular, the above-mentioned technical problems are addressed. By combining a spectrometer device with an imaging device, spatially resolved spectroscopic data can be acquired. This allows chemical information to be acquired at different precisely traced locations on the object. In particular, the provided method and system allow accurate spectroscopic data of the object to be acquired despite possible local variations in chemical composition, such as in the case of heterogeneous objects.

[0060] In particular, the proposed method, system, computer program and computer readable storage medium facilitate the correlation of the spectral axis with the spatial arrangement at which the spectroscopic data, in particular the spectrum (also called the spectral curve), was obtained. The system may in particular be embodied as including a sensor fusion of a spectrometer device with an imaging device, in particular an imaging optics and / or an imaging detector. This allows the capture of spatially resolved spectroscopic data, in particular given an imaging device, in particular a visual camera, and a spectrometer device, for example a device such as a smartphone with a NIR spectroscopic sensor. The data thus obtained can be used to obtain chemical information at different precisely traced arrangements of an object, for example a sample.

[0061] The imaging device, specifically a camera, is used to track the current pointing and positioning of the spectrometer device, allowing spatial correlation of the spectroscopic data (specifically spectra) captured at any time. In particular, a user can use the imaging device to capture a wide image and then approach close to individual elements seen in the wide image so that the spectrometer device can obtain the spectroscopic data, specifically spectra, of a particular patch. During the movement of the system, the image, for example the position in the wide image, called the raw image, can be actively tracked using the imaging device, specifically a camera, and the motion tracking software. By repeating this procedure several times at different spots of the raw image, the spectral information of the different patches is obtained. Furthermore, by using the trained model directly, either on the evaluation unit of the system or on the computing cloud, the chemical composition at each spot can be determined and provided to the user.

[0062] An alternative application of the proposed method, system, computer program and computer readable storage medium is the drift scanning technique, which allows a user to move across an object, e.g., a sample, while acquiring image data, particularly image capture, and acquiring spectroscopic data in parallel. The spectroscopic data, particularly spectra and images, can be combined in a mosaic to retrieve chemical information along the scan path.

[0063] By way of summary and without excluding further possible embodiments, the following embodiments can be envisaged.

[0064] Embodiment 1: A method of obtaining at least one item of object information relating to at least one object by spectroscopic measurement, comprising the steps of: i. acquiring spectroscopic data, in particular spectroscopic data of an object, using at least one spectrometer device and within at least one spatial measurement range of said spectrometer device; ii. using at least one imaging device, specifically a camera, to acquire image data of a scene within a field of view of the imaging device, the scene including at least a portion of the object and at least a portion of a spatial measurement range of the spectrometer device; and iii. evaluating the spectroscopic data of step i. and at least one item of image information derived from the image data of step ii. to obtain at least one item of object information relating to the at least one object. The method includes:

[0065] Embodiment 2: A method according to the preceding claims, wherein step iii. further comprises deriving at least one item of image information from the image data of step ii.

[0066]

[0023] Embodiment 3: At least one item of image information is at least one of the following: - at least one image derived from the image data of step ii.; - at least one item of spatial information relating to a spatial measurement range in the scene, in particular an indication of the spatial measurement range in the image for which the spectroscopic data was acquired; - at least one item of identification information for at least one object, in particular identification information for at least one of the following: type of object, boundary of the object in the scene, size of the object, location of the object, color of the object, texture of the object, shape of the object, contrast of the object, volume of the object, area of ​​interest of the object; - at least one item of positioning information relating to at least one object, in particular an indication of the positioning of the spectrometer device with respect to the at least one object, - at least one item of directional information, in particular an indication of the direction between the spectrometer device and at least one object; - at least one item of similarity information about the object, in particular similarity information about at least one shared characteristic shared between different regions of the object; 2. A method according to any one of the preceding claims, comprising:

[0067] Embodiment 4: A method according to any one of the preceding claims, wherein at least one item of image information comprises at least one image derived from image data of step ii., steps i. and ii. are repeatedly performed, and at least one item of object information in step iii. comprises a combination of spectroscopic object information derived from the repetition of step i. and at least one item of spatial information relating to a spatial measurement range within the scene derived from the repetition of step ii., wherein at least one of the spatial information and the spectroscopic object information is displayed in the image.

[0068] Embodiment 5: A method according to the preceding claims, wherein during the repetition of steps i. and ii., at least one of the scene, the field of view and the object is modified.

[0069] Embodiment 6: A method according to any one of the two preceding claims, generating at least one image of a scene having at least two items of spectroscopic object information and corresponding spatial information regarding spatial measurement ranges within the image for each item of spectroscopic object information, in particular the arrangement of the spatial measurement ranges.

[0070] Embodiment 7: A method according to any one of the three preceding claims, wherein the image derived from the image data of step ii. is an image derived from the image data of repetitions of step ii., in particular at least one of a combined image and a selected image of images derived from the image data of repetitions of step ii.

[0071] Embodiment 8: A method according to any one of the preceding claims, wherein at least one item of image information includes at least one item of identification information relating to at least one object, the method comprising applying at least one identification algorithm to the at least one item of image information to derive the at least one item of identification information from the at least one item of image information.

[0072] Embodiment 9: A method according to the preceding claims, wherein the identification algorithm comprises at least one object recognition algorithm for determining the type of the at least one object.

[0073] Embodiment 10: A method according to any one of the two preceding claims, wherein step iii. comprises applying at least one spectroscopic evaluation algorithm to the spectroscopic data of step i., the spectroscopic evaluation algorithm being selected according to an item of identification information, in particular according to at least one type of object.

[0074] Embodiment 11: A method according to the preceding claims, comprising providing a plurality of spectroscopic evaluation algorithms for different items of identification information, in particular for different types of objects.

[0075] Embodiment 12: A method according to any one of the preceding claims, wherein at least one spectroscopic evaluation algorithm includes at least one trained model.

[0076] Embodiment 13: A method according to any one of the preceding claims, further comprising providing at least one item of object information relating to the at least one object, in particular optically providing at least one item of object information relating to the at least one object via a display device.

[0077] Embodiment 14: A method according to any one of the preceding claims, which is at least partially computer-implemented, in particular step iii.

[0078]

[0043] Embodiment 15: A system for obtaining at least one item of object information regarding at least one object by spectroscopic measurement, comprising: I. at least one spectrometer device configured to acquire spectroscopic data (specifically of an object) within at least one spatial measurement range of the spectrometer device; II. At least one imaging device, specifically a camera, configured to acquire image data of a scene within a field of view of the imaging device, the scene including at least a portion of an object and at least a portion of a spatial measurement range of the spectrometer device; and III. At least one evaluation unit, configured to evaluate spectroscopic data acquired by the spectrometer device and at least one item of image information derived from image data acquired by the imaging device to obtain at least one item of object information relating to the at least one object. Including, the system.

[0079] Embodiment 16: A system according to the preceding claims, wherein the imaging device comprises at least one camera having one or more imaging sensors, in particular one or more CCD or CMOS imaging sensors, for acquiring image data of the scene.

[0080] Embodiment 17: A system according to any one of the preceding claims, wherein the imaging device comprises at least one LIDAR-based imaging device for acquiring image data of the scene.

[0081] Embodiment 18: A system according to any one of the preceding claims, referring to a system in which the spectrometer device includes at least one optical element and at least one detector device including a plurality of photosensitive elements, the at least one optical element configured to separate incident light into a spectrum of constituent wavelength components, each photosensitive element configured to receive at least a portion of one of the constituent wavelength components, and configured to generate a respective detector signal in response to illumination of each photosensitive element by at least a portion of each of the constituent wavelength components.

[0082] Embodiment 19: A system according to the preceding claims, wherein the optical element includes at least one wavelength-selective element.

[0083] Embodiment 20: A system according to the preceding claims, wherein the wavelength-selective element is selected from the group consisting of a prism, a diffraction grating, a linear variable filter, an optical filter, in particular a narrow bandpass filter.

[0084] Embodiment 21: A system according to any one of the three preceding claims, wherein the detector device comprises a plurality of photosensitive elements arranged in a linear array, and the array of photosensitive elements comprises a number between 10 and 1000, in particular a number between 100 and 500, in particular a number between 200 and 300, more in particular a number of 256 photosensitive elements.

[0085] Embodiment 22: A system according to any one of the preceding four claims, wherein each photosensitive element is selected from the group consisting of a pixelated inorganic camera element, in particular a pixelated inorganic camera chip, more particularly a CCD chip or a CMOS chip, a monochrome camera element, in particular a monochrome camera chip, at least one photoconductor, in particular an inorganic photoconductor, more particularly an inorganic photoconductor comprising Si, PbS, PbSe, Ge, InGaAs, ext.InGaAs, InSb or HgCdTe.

[0086] Embodiment 23: A system according to any one of the preceding five embodiments, wherein each photosensitive element is sensitive to electromagnetic radiation in the wavelength range of 760 nm to 1000 μm, specifically in the wavelength range of 760 nm to 15 μm, more specifically in the wavelength range of 1 μm to 5 μm, and even more specifically in the wavelength range of 1 μm to 3 μm.

[0087] Embodiment 24: A system according to any one of the preceding claims, referring to a system in which the spectrometer device and the imaging device have a known arrangement, in particular a fixed arrangement, relative to each other.

[0088] Embodiment 25: A system according to any one of the preceding claims, further comprising at least one light source configured to emit electromagnetic radiation in the wavelength range of 760 nm to 1000 μm, specifically in the wavelength range of 760 nm to 15 μm, more specifically in the wavelength range of 1 μm to 5 μm, more specifically in the wavelength range of 1 μm to 3 μm.

[0089] Embodiment 26: A system according to any one of the preceding claims, further comprising at least one display device configured to provide at least one item of object information relating to at least one object.

[0090] Embodiment 27: A system according to any one of the preceding claims, comprising at least one mobile device, the mobile device comprising at least one spectrometer device and at least one imaging device.

[0091] Embodiment 28: A computer program comprising instructions, when executed by a control unit of a system according to any one of the preceding claims referring to a system, that cause the system to perform a method according to any one of the preceding claims referring to a method.

[0092] Embodiment 29: A computer-readable storage medium comprising instructions, which when a program is executed by a control unit of a system according to any one of the preceding claims referring to a system, cause the system to perform a method according to any one of the preceding claims referring to a method. [Brief description of the drawings]

[0093] Further optional features and characteristics are preferably disclosed in more detail in the following description of the embodiments in conjunction with the dependent claims, whereby a person skilled in the art will understand that each optional feature may be implemented individually and also in combination with any optional feature. The scope of the present invention is not limited to the preferred embodiments. The embodiments are illustrated diagrammatically in the figures. The same reference numbers in these figures refer to identical or functionally equivalent elements.

[0094] In the diagram below: [Figure 1] FIG. 1 shows a flow chart of an embodiment of a method for obtaining at least one item of object information. [Diagram 2] 2A and 2B are schematic diagrams showing a system for obtaining at least one item of object information relating to an object (FIG. 2A) and a system for obtaining said object together with a corresponding item of object information (FIG. 2B). [Diagram 3] FIG. 3 shows further objects together with corresponding object information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0095] Detailed Description of the Embodiments Fig. 1 shows an embodiment of a method for obtaining at least one item of object information 110. In Fig. 2A an exemplary embodiment of a system 178 for obtaining at least one item of object information 110 relating to at least one object 112 by spectroscopic measurements is depicted in schematic form. Examples of possible items of object information 110 obtained using this method are illustrated in Fig. 2B and Fig. 3 together with the corresponding object 112, which are described together in the following.

[0096] The system 178 shown in FIG. 2A includes: i. at least one spectrometer device 116 configured to acquire spectroscopic data 114 within at least one spatial measurement range 118 of the spectrometer device 116; ii. at least one imager 120, specifically a camera 122, configured to acquire image data of a scene 124 within a field of view 126 of the imager 120, the scene 124 including at least a portion of the object 112 and at least a portion of a spatial measurement range 118 of the spectrometer device 116; and iii. at least one evaluation unit 180, configured to evaluate the spectroscopic data 114 acquired by the spectrometer device 116 and at least one item of image information 128 derived from the image data acquired by the imaging device 120, to obtain at least one item of object information 110 relating to the at least one object 112; Includes.

[0097] As outlined above, in Fig. 1 an exemplary embodiment of a method for obtaining at least one item of object information 110 relating to at least one object 112 by spectroscopic measurements is shown as a schematic flow chart. The method can in particular make use of the system 178 of Fig. 2A. However, the use of alternative systems is generally possible. The method comprises method steps i., ii. and iii., which are described in detail below. In the flow chart of Fig. 1, method step i. is indicated with reference numeral 130, method step ii. with reference numeral 132 and method step iii. with reference numeral 134.

[0098] The method includes the steps of: i. acquiring spectroscopic data 114 using at least one spectrometer device 116 within at least one spatial measurement range 118 of the spectrometer device 116; ii. acquiring image data of a scene 124 within a field of view 126 of the imaging device 120, using at least one imaging device 120, specifically a camera 122, the scene 124 including at least a portion of the object 112 and at least a portion of a spatial measurement range 118 of the spectrometer device 116; and iii. evaluating the spectroscopic data 114 of step i. and at least one item of image information 128 derived from the image data of step ii. to obtain at least one item of object information 110 relating to the at least one object 112. Includes.

[0099] The method steps may be specifically performed in a given order. However, different orders may also be performed. Furthermore, one or more of the method steps, or all of the method steps, may be performed repeatedly, as outlined in more detail below. Furthermore, the method may include additional method steps not listed here.

[0100] Step i. of the method involves acquiring spectroscopic data 114 using a spectrometer device 116 within a spatial measurement range 118 of the spectrometer device 116. This step is described herein in conjunction with the particular embodiment of a system 178 shown in FIG. 2A.

[0101] Thus, the spectrometer device 116 may be specifically embodied as a portable spectrometer device 116. Specifically, the spectrometer device 116 may be part of a mobile device 136, such as a laptop, a tablet, or specifically a mobile phone, e.g., a smartphone 138. The mobile device 136 may specifically have at least one function different from the spectroscopic function, e.g., a mobile communication function, e.g., a mobile phone function. The spectrometer device 116 may be at least one of integrated with the mobile device 136 and / or attachable thereto. The mobile device 136 shown in FIG. 2A may specifically be embodied as a smartphone 138 with the spectrometer device 116 integrated therein. The spectrometer device 116 may be configured to obtain spectroscopic data 114 of the object 112 as part of the spectroscopic measurement. As part of the spectroscopic measurement, the object 112 may be illuminated with electromagnetic radiation 140, specifically, light 140 in the infrared spectral range, specifically, in the near-infrared spectral range. In particular, the electromagnetic radiation 140 may be in the wavelength range of 760 nm to 1000 μm, in particular in the wavelength range of 760 nm to 15 μm, more particularly in the wavelength range of 1 μm to 5 μm, more particularly in the wavelength range of 1 μm to 3 μm. The light 140 may be generated by a light source 142 configured to emit electromagnetic radiation 140 in particular in the wavelength range of 760 nm to 1000 μm, in particular in the wavelength range of 760 nm to 15 μm, more particularly in the wavelength range of 1 μm to 5 μm, more particularly in the wavelength range of 1 μm to 3 μm. The light source 142 may be part of a mobile device 136, in particular a smartphone 138 as shown in FIG. 2A. However, other options are feasible, such as the use of one or more external light sources or an embodiment without a light source.

[0102] The spectroscopic measurement may further include, in particular, receiving the incident light 140 after interaction with the object 112 and generating at least one corresponding signal that may form part of the spectroscopic data 114. The spectrometer device 116 used in step i. may in particular be a near-infrared spectrometer device 116. Thus, the spectrometer device 116 may in particular be configured to detect electromagnetic radiation 140 in the near-infrared range. The spectrometer device 116 may be configured to perform at least one spectroscopic measurement on the object 112. The spectrometer device 116 may in particular comprise at least one detector device 144 comprising at least one optical element 146 and a plurality of photosensitive elements 148 (e.g. an array of semiconductor photosensitive elements 148), as illustrated in FIG. 2A. The at least one optical element 146 may in particular be configured to separate the incident light 140, in particular the electromagnetic radiation 140 in the near-infrared range, into a spectrum of constituent wavelength components. Thus, by way of example, the at least one optical element 146 may specifically include at least one wavelength-selecting element 184, e.g., at least one of a diffraction grating, a prism, and a filter, e.g., a variable-length filter, having various regions with different wavelength-selective transmittances. Thus, by way of example, the detector arrangement 144 may specifically include an array, e.g., a linear array, of photosensitive elements 148, which are combined with different filters or filter regions of a variable-length filter having different spectral characteristics, such that each combination of a photosensitive element 148 and its corresponding filter or filter region has sensitivity in a different spectral range. Thus, each photosensitive element 148 may specifically be configured in combination with the wavelength-selecting element 184 to receive at least a portion of the constituent wavelength components and to generate a respective detector signal in response to illumination of the respective photosensitive element 148 by at least a portion of the respective constituent wavelength components. The detector signals, and specifically the signal intensities, together with the corresponding wavelengths form part of the spectroscopic data 114, and the detector signals may be part of the spectroscopic data 114.

[0103] The spectroscopic data 114 may include information about at least one optical property or optically measurable property of the object 112, determined for one or more different wavelengths as a function of the wavelength. More specifically, the spectroscopic data 114 may relate to at least one property characterizing at least one of the transmission, absorption, reflection and emission of the object 112. The at least one optical property is determined for one or more wavelengths. The spectroscopic data 114 may specifically take the form of a signal intensity, determined as a function of the wavelength of the spectrum or a division thereof, for example a wavelength interval, which signal intensity is preferably provided as an electrical signal that can be used for further evaluation. In particular, the spectroscopic data 114 is graphically represented in the form of a spectral curve 150, in which the signal intensity I, plotted on the y-axis 152, is shown as a function of the wavelength λ, plotted on the x-axis 154, as illustrated in Figs. 2B and 3. In particular, the signal intensity I corresponds to the intensity of reflected electromagnetic radiation 140, e.g., electromagnetic radiation 140 in the infrared spectral range, as illustrated in Figure 2A, with which the object 112, or at least a portion of the object 112, is illuminated, e.g., with a light source 142 of a smartphone 138. A spectral curve 150 shows the reflected intensity I as a function of wavelength λ, as illustrated in Figures 2B and 3.

[0104] In step i., spectroscopic data 114 is acquired using at least one spectrometer device 116 within a spatial measurement range 118 of the spectrometer device 116. In particular, the spectrometer device 116 may be configured to acquire spectroscopic data 114 based on incident light 140 from within the spatial measurement range 118. As shown in FIG. 2A, the spatial measurement range 118 may in particular be a three-dimensional space, such as a three-dimensional spatial section, e.g., a cone-shaped spatial section, the light content of which is received and analyzed by the spectrometer device 116. As an example, the spatial measurement range 118 is defined as a solid angle or a three-dimensional angular segment in the space, and the object 112 located within the solid angle or angular segment is analyzed by the spectrometer device 116. As an example, the solid angle or angular segment is defined by the geometrical and / or optical properties of the spectrometer device 116. The spectroscopic data 114 acquired by the spectrometer device 116 may include information related to at least one object 112, if the object 112 is located within the spatial measurement range 118 of the spectrometer device 116. Specifically, to spectroscopically analyze the object 112, the spectrometer device 116 may be scanned over a range including the object, or may be placed in close proximity to the object 112, for example at a distance in the range of 0 mm to 100 mm from the object 112, specifically at a distance in the range of 0 mm to 15 mm. In the exemplary embodiment of step i. shown in FIG. 2A, the object 112 located in the spatial measurement range 118 of the spectrometer device 116 may be or may include an apple. However, a wide variety of objects 112 are feasible. Thus, the object 112 may generally be any animate or inanimate object. In particular, the object 112 may be a heterogeneous object 112, for example an object 112 having at least one property, for example at least one of a chemical property, a physical property and a biological property, which varies within the object 112 in a position-dependent manner. In particular, the chemical composition may vary within the object 112 in a position-dependent manner. However, other objects 112 are also feasible, in particular homogeneous objects 112, for example objects 112 having only a small or no variation in chemical composition.The object 112 may specifically be or may include a food item 156, such as a fruit 158 ​​or a vegetable, or a body part 160, such as skin 162. Exemplary objects 112 shown in the figures are an apple in Figures 2A and 2B, a banana in Figure 2B, and the skin 162 of a human hand and arm in Figure 3.

[0105] In step ii. of the method shown in Fig. 1, image data of a scene 124 within a field of view 126 of the imager 120 is acquired using the imager 120. The imager 120 may be or comprise at least one camera 122 having one or more imaging sensors, in particular one or more CCD or CMOS imaging sensors, for acquiring image data, as outlined above. The camera 122 may in particular comprise at least one camera chip, for example at least one CCD chip and / or at least one CMOS chip, configured to record the image 164. The camera 122 may comprise a one-dimensional or two-dimensional array of imaging sensors, for example pixels, arranged on the camera chip. By way of example, the camera 122 may comprise at least 100 pixels in at least one dimension, for example at least 100 pixels in each dimension. By way of example, the camera 122 may comprise an array of imaging sensors comprising at least 100 imaging sensors in each dimension, in particular at least 300 imaging sensors in each dimension. For example, the camera 122 may be a color camera 122 including color pixels, each color pixel including at least three color sub-pixels sensitive to different colors. For example, the camera 122 may include black and white and / or color pixels. Color and black and white pixels may be combined inside the camera 122. The camera 122 may be a camera 122 of a mobile device 136. The present invention is particularly applicable to cameras 122 that are typically used in mobile devices 136, such as laptops, tablets, or particularly mobile phones, for example, smartphones 138. FIG. 2A shows a smartphone 138 with a camera 122 as the imaging device 120. The mobile device 136, particularly the smartphone 138, may further include one or more data processing devices, such as one or more processors 188, as shown in FIG. 2A. The camera 122 may include further elements, such as one or more optical elements, for example one or more lenses (not shown), in addition to at least one camera chip or imaging chip. As an example, the camera 122 may be a fixed focus camera 122 having at least one lens fixedly aligned relative to the camera 122 .However, alternatively, the camera 122 may be equipped with one or more variable lenses that can be automatically or manually adjusted.

[0106] As shown in FIG. 2A, the mobile device 136, specifically the smartphone 138, may comprise both the camera 122 and the spectrometer device 116. Thus, the spectrometer device 116 and the imaging device 120, for example at least one camera 122, may both be integrated into the mobile device 136, for example the smartphone 138. The smartphone 138 may further comprise a housing 161, in which the spectrometer device 116 and the imaging device 120, specifically the camera 122, may be integrated. The smartphone 138 may specifically comprise a front camera 163 and a rear camera 165. In particular, the field of view 126 of the front camera 163 may at least partially overlap the spatial measurement range 118 of the spectrometer device 116, as shown in FIG. 2A. In particular, the front camera 163 may be used to perform the method step ii. Other possibilities are also feasible.

[0107] In step ii., image data of a scene 124 within a field of view 126 of the imaging device 120 is acquired, the scene 124 including at least a portion of the object 112 and at least a portion of the spatial measurement range 118 of the spectrometer device 116. FIG. 2A shows both the field of view 126 of the imaging device 126 and the spatial measurement range 118 of the spectrometer device 116. The image data generated by the imaging device 120 may include spatially resolved optical information related to the object 112 located within the field of view 126 of the imaging device 120. The field of view 126 may be, in particular, a three-dimensional spatial section whose optical content is imaged by the imaging device 120. In particular, the scene 124 included by the field of view 126 is imaged by the imaging device 120. In particular, the scene 124 may include one or more objects 112, such as the object 112 mentioned with respect to step i. above, and at least one object 112 in the scene 124 may be imaged by the imaging device 120. Thus, the scene 124 may specifically include a number of objects 112 having a particular arrangement, and the objects 112 and their arrangement may be imaged by the imaging device 120 to generate at least one image 164. As an example, the image 164 of FIG. 2B shows a scene 124 including an apple and a banana arranged on a plate located on a substrate, the apple and the banana serving as the object 112 from which the spectroscopic data 114 is acquired. In the exemplary embodiment shown in FIG. 2A, the scene 124 includes the top of the apple. The scene 124 further includes at least a portion of the spatial measurement range 118 of the spectrometer device 116, as illustrated in FIG. 2A. Thus, the field of view 126 of the imaging device 120 and the spatial measurement range 118 of the spectrometer device 116 may at least partially overlap, as shown in FIG. 2A.

[0108] As part of method step ii., image data of a scene 124 within a field of view 126 of the imaging device 120 is acquired, the scene 124 including at least a portion of the object 112 and at least a portion of the spatial measurement range 118 of the spectrometer device 116. The object 112 of step i. may be at least partially visible in the image data of step ii. Thus, the field of view 126 of the imaging device 120 and the spatial measurement range 118 of the spectrometer device 116 may at least partially overlap. A spatial relationship between the field of view 126 of the imaging device 120 and the spatial measurement range 118 of the spectrometer device 116 may be known and may be used, for example, in step iii, such as an offset between the field of view 126 of the imaging device 120 and the spatial measurement range 118 of the spectrometer device 116 and / or at least one angle between the field of view 126 of the imaging device 120 and the spatial measurement range 118 of the spectrometer device 116. Thus, the location and / or object 112 within the field of view 126 of the imaging device 120 may be located within the spatial measurement range 118 of the spectrometer device 116, or vice versa. In particular, the at least one object 112, or at least a portion thereof, may thus be located within both the field of view 126 of the imaging device 120 and the spatial measurement range 118 of the spectrometer device 116, as is evident from FIG. 2A . Thus, the at least one object 112, or at least a portion thereof, is not only spectroscopically examined by the spectrometer device 116, but is also at least partially imaged by the imaging device 120.

[0109] In step iii., the spectroscopic data 114 of step i. and at least one item of image information 128 derived from the image data of step ii. are evaluated to obtain at least one item of object information 110 relating to at least one object 112. Step iii. may in particular further comprise deriving at least one item of image information 128 from the image data of step ii. As part of the evaluation, the spectroscopic data 114 and / or the item of image information may be analyzed, for example by applying at least one analysis step, for example an analysis step comprising at least one analysis algorithm applied to the data and / or information. In particular, the spectroscopic data 114 and / or the item of image information may be processed and / or interpreted and / or evaluated as part of the analysis step. As an example, the evaluation of the spectroscopic data 114 may comprise analysing the spectroscopic data 114 to determine at least one peak 166 in the spectroscopic data 114 reflecting a global or local maximum of transmission, absorption, reflection and / or emission of the object 112. The evaluation of the spectroscopic data 114 may further include identifying at least one corresponding wavelength. The evaluation of the spectroscopic data 114 may further include determining a chemical composition of the object 112, for example, by comparing the identified peaks 166 with at least one predefined peak 166 or at least one predefined set of peaks 166. The evaluation of the spectroscopic data 114 is particularly performed using at least one spectroscopic evaluation algorithm. The results of the evaluation of the spectroscopic data 114 are also referred to as spectroscopic object information 167. By way of example, the evaluation of the items of image information 128 may include analyzing the items of image information 128, for example, using at least one identification algorithm, particularly at least one object recognition algorithm, as outlined in more detail below.

[0110] At least one item of image information 128 derived from the spectroscopic data 114 of step i. and the image data of step ii. is evaluated to obtain at least one item of object information 110 relating to at least one object 112. The item of object information 110 specifically relates to at least one property of the object, for example at least one of chemical, physical and biological properties, for example the material and / or composition of the object. As an example, the content of water and / or at least one other target component, for example target components such as fats, sugars, in particular glucose, melanin, lactic acid and / or alcohol, may be determined. In particular, the properties may vary within the object 112 and be characteristic of a particular location or spatial range within the object 112. However, the properties may also show no or only slight variations throughout the object 112. The item of object information describes the properties in a qualitative and / or quantitative manner, for example by means of one or more numerical values. In particular, the item of object information 110 may include chemical information, in particular the chemical composition, of the object 112. The item of object information 110 may include information on the properties as well as spatial information relating to a particular location or spatial range within the object 112 where the property was measured. In this way, the items of evaluated spectroscopic data 114 and evaluated image information 128 are combined or connected, for example in a predefined manner and / or according to a predefined algorithm, to obtain at least one item of object information 110. The at least one item of image information 128 may include at least one of the following: - at least one image 164 derived from the image data of step ii, at least one item of spatial information 168 relating to a spatial measurement range 118 in the scene 124, in particular an indication 170 of the spatial measurement range 118 in which the spectroscopic data 114 was acquired in the image 164; at least one item of identification information of at least one object 112, in particular identification information relating to at least one of the following: type of object 112, boundary of object 112 within scene 124, size of object 112, location of object 112, color of object 112, texture of object 112, shape of object 112, contrast of object 112, volume of object 112, area of ​​interest of object 112, - at least one item of configuration information relating to the at least one object 112, in particular an indication of the configuration of the spectrometer device 116 relative to the at least one object 112; at least one item of directional information, in particular an indication of the direction between the spectrometer device 116 and at least one object 112; - at least one item of similarity information regarding the object 112, in particular similarity information regarding at least one shared characteristic shared between different regions of the object 112; may include:

[0111] As an example of a method for obtaining at least one item of object information 110 relating to at least one object 112 by spectroscopic measurements, the at least one item of image information 128 may comprise at least one image 164 derived from the image data of step ii. The image 164 may in particular comprise the image data mentioned in step ii., or a part thereof, and / or may be derived from the image data or a part thereof. As part of the method, steps i. and ii. may be performed iteratively. The at least one item of object information 110 in step iii. may comprise a combination of spectroscopic object information 167 derived from the repetition of step i. and at least one item of spatial information 168 relating to the spatial measurement range 118 in the scene 124 derived from the repetition of step ii. The method may further comprise displaying at least one of the spatial information 168 and the spectroscopic object information 167 in the image 164. Thus, by way of example, the image 164 may include information about the acquisition location of the spectroscopic data 114 and / or the results of the evaluation of the spectroscopic data 114, e.g. compositional information derived from the spectroscopic data 114. In this way, the image 164 visually shows the scene 124 or a part thereof and information derived from the spectroscopic data 114 acquired in step i., optionally together with positional information about the acquisition location of the information. In this way, the image 164 includes an overlap of at least one object 112 visible in the scene 124 with one or more locations where one or more spectroscopic measurements have been performed, and may optionally include the results of the spectroscopic measurements and / or one or more items of information derived from the spectroscopic measurements. Both Figs. 2B and 3 show examples of one or more images 164 of at least one object 112, with the locations where one or more spectroscopic measurements have been performed on the object 112 marked. Furthermore, information in the form of a spectral curve 150 derived from the spectroscopic measurements is shown. Both figures are explained in more detail below.

[0112] Between possible iterations of steps i. and ii., at least one of the scene 124, the field of view 126, and the object 112 may be changed. Thus, by way of example, the scene 124 may change and / or at least one of the spectrometer device 116, the imaging device 120, and the device including both the spectrometer device 116 and the imaging device 120, e.g., the mobile device 136, may move. In particular, the method generates at least one image 164 of the scene 124, the scene having at least two items of spectroscopic object information 167 and corresponding spatial information 168 regarding the spatial measurement range 118 within the image for each item of the spectroscopic object information 167. Furthermore, the image 164 derived from the image data of step ii. may be an image 164 derived from the image data of the iterations of step ii., in particular at least one of a combined image 164 and a selected image 164 of the images 164 derived from the image data of the iterations of step ii.

[0113] 2B exemplarily illustrates two items of object information 110 relating to two different objects 112, specifically an apple and a banana, as outlined above. The item of object information 110 relating to an apple may include a spectral curve 150 determined by evaluating spectroscopic data 114, which may be obtained, for example, in a spectroscopic measurement of the apple, as shown in FIG. 2A. The item of object information 110 relating to an apple may further include an image 164 showing the apple as part of a scene 124 captured by the imaging device 120. The item of object information 110 may further include an indication 170 showing within the image 164 the location of the spatial measurement range 118 in which the spectroscopic data 114 was obtained. Similarly, the item of object information 110 relating to a banana may include a spectral curve 150 determined by evaluating spectroscopic data 114, which may be obtained, for example, in a spectroscopic measurement of the banana (not shown). The item of object information 110 relating to a banana may further include an image 164 showing the banana as part of the scene 124 captured by the imaging device 120, and an indication 170 showing within the image 164 the location of the spatial measurement range 118 at which the spectroscopic data 114 was acquired. It should be noted that the location of the smartphone 138 when capturing the image 164 showing the apple and the banana may be different from the location at which the spectroscopic data 114 of the apple was acquired, for example. In particular, the imaging device 120 and / or the spectrometer device 116 may move during, and in particular during, any repetition of steps i. and ii. In particular, in a first execution of step ii., image data of the first scene 124 may be acquired at a first distance, and during the repetition of step ii., the imaging device 120 and / or the spectrometer device 116 may move closer to the object 112 such that the captured scene 124 is a subsection of said first scene 124. In particular, image data corresponding to a wide image 164 may be acquired in a first execution of step ii., as shown in Figures 2B and 3. The wide image 164 includes the object 112 completely or nearly completely.For further repetitions of step ii. and / or step i., the distance of the imaging device 120 and / or the spectrometer device 116 relative to the object 112 may be reduced to at least one second distance, which allows acquisition of spectroscopic data 114 of the object 112 by performing step i. In particular, the second distance may be in the range of 0 mm to 100 mm from the object 112, in particular in the range of 0 mm to 15 mm. The image 164 derived from the image data acquired at the second distance may show a subsection of the image 164 derived from the image data acquired in the first execution of step ii. The method may further include tracking the movement of the imaging device 120, for example from the first distance to at least one second distance, using the imaging device 120 and the motion tracking software. In particular, a spatial relationship between the image data and / or the spectroscopic data 114 acquired at the at least one second distance and the image data acquired at the first distance is inferred, for example taking into account the positional information and / or orientation information derived from the image data. An item of object information 110 links spectroscopic object information 167, e.g., a chemical composition determined using spectroscopic data 114, with an item of spatial information 168 that identifies a location in image 164 of object 112 for which the spectroscopic object information 167 is valid. The location of object 112 is identified in image 164 by at least one graphical indicia 170, such as an arrow pointing to the location, or a circle, square, or other type of indicia 170 that surrounds or marks the location. One or more such locations may be marked in image 164 to display the corresponding spectroscopic object information 167.

[0114] As a further example illustrated in FIG. 3, the imaging device 120 and / or the spectrometer device 116 move across the object 112, for example along a scanning path 172, at a fixed distance and / or at a variable distance, while performing repetitions of steps i. and ii. By performing step iii., at least one item of object information 110 is obtained, the object information 110 including, in particular, a number of items of chemical information corresponding to a number of locations along the scanning path 172. Again, image data of the object 112 is acquired, for example in a first execution of step ii., and the scanning path 172 may be included in an image 164 derived from the image data. In particular, the scanning path 172 and / or the spectroscopic object information 167, in particular the chemical information, are shown in the image 164. This allows the chemical information to be retrieved along the scanning path 172. In the example illustrated in FIG. 3, the spectroscopic object information 167 in the form of three spectral curves 166 is shown together with three different locations where the spectral data 114 was acquired.

[0115] As a further example, the items of image information 128 may include at least one item of identification information relating to at least one object 112, in particular identification information relating to at least one of the following: type of object 112, boundary of object 112 within scene 124, size of object 112, location of object 112, color of object 112, texture of object 112, shape of object 112, contrast of object 112, volume of object 112, area of ​​interest of object 112. The items of identification information may be derived, in particular, by using at least one identification algorithm, such as an image recognition algorithm and / or a trained model configured to recognize or identify object 112, for example by using artificial intelligence, such as an artificial neural network. In particular, the at least one item of image information includes at least one item of identification information relating to the at least one object 112, and the method includes applying at least one identification algorithm to the at least one item of image information 128 to derive at least one item of identification information from the at least one item of image information 128. The identification algorithm specifically includes at least one object recognition algorithm for determining a type of the at least one object 112. For example, the object recognition algorithm identifies a type of the object 112. In the example shown in Fig. 2A and Fig. 2B, the types of the object 112 are identified as apple and banana, respectively. In the example shown in Fig. 3, the types of the object 112 are identified as a human hand and arm. In particular, step iii. includes applying at least one spectroscopic evaluation algorithm to the spectroscopic data 114 of step i., the spectroscopic evaluation algorithm being selected according to the item of identification information, specifically according to the type of the at least one object 112. The method may in particular include providing a plurality of spectroscopic evaluation algorithms for different items of identification information, specifically different types of objects 112. Thus, depending on the type of object 112 determined by the identification algorithm, a corresponding spectroscopic evaluation algorithm is selected and information determined from the image data is subsequently used for the evaluation of the spectroscopic data 114 .As an example, the items of image information may include an item of identification information identifying that the object 112 from which the spectroscopic data 114 was acquired is an apple. In this manner, the spectroscopic data 114 may be evaluated using a spectroscopic evaluation algorithm optimized for evaluating apples. Using an application specific spectroscopic evaluation algorithm may improve the accuracy of the evaluation results (e.g. the chemical composition of the object 112) and / or speed up the evaluation process.

[0116] Additionally or alternatively, the items of image information include identifying information regarding the size of the object 112. Thus, the image information may include identifying information regarding both the type and the size of the object 112. Different items of identifying information may be combined to generate added value. As an example, the object 112 may be identified as an apple and the size of the apple may be derived from the image data. Based on these items of information an estimated mass of the apple may be determined. This information may be combined with the chemical composition determined by evaluating the spectroscopic data 114 to obtain an item of object information 110 to infer at least one item of nutritional information, such as nutritional value per portion.

[0117] The method may be at least partially computer-implemented, in particular step iii. The computer-implemented steps and / or aspects of the invention are in particular implemented using a computer or a computer network. As an example, step iii. of the method is fully or partially computer-implemented. Thus, the evaluation of the spectroscopic data is in particular performed using at least one spectroscopic evaluation algorithm. The evaluation of the item of image information may include analyzing the item of image information, for example using at least one identification algorithm. The evaluated spectroscopic data and the evaluated item of image information are combined or connected, for example in a predetermined manner and / or according to a predetermined algorithm, to obtain at least one item of object information. The at least one spectroscopic evaluation algorithm may in particular include at least one trained model. The method may further include providing at least one item of object information 110 relating to the at least one object 112, in particular optically providing at least one item of object information 110 relating to the at least one object 112 via the display device 174. In particular, the items of object information 110 are displayed on a display device 174 , such as a screen 176 of the mobile device 136 , which may include the imaging device 120 and / or the spectrometer device 116 .

[0118] As outlined above, Fig. 2A shows a system 178 for acquiring at least one item of object information 110. The system 178 specifically includes a smartphone 138. Fig. 2A shows a system 178 for acquiring at least one item of object information 110 during execution of step i. of the method for acquiring at least one item of object information 110. An evaluation unit 180 of the system 178 for acquiring at least one item of object information 110 is specifically configured to analyze the spectroscopic data 114 and / or image data, specifically the items of image information 128. The evaluation unit 180 specifically processes and / or interprets and / or evaluates the data and / or information as part of the analysis process. The evaluation unit 180 includes, among other things, at least one processor 182. The processor 182 is specifically configured to perform one or more evaluation operations on the data and / or information, such as by software programming. As shown in FIG. 2A , the system 178 includes at least one display device 174 , such as a screen 176 of a mobile device 136 , configured to provide at least one item of object information 110 relating to at least one object 112 .

[0119] The system 178 further includes at least one control unit 186. The control unit 186 is specifically configured to perform at least one computing operation for obtaining at least one item of object information 110 and / or to control at least one function of at least one other component of the system 178. The control unit 186 specifically controls at least one function of the spectrometer device 116, such as the acquisition of the spectroscopic data 114. The control unit 186 specifically controls at least one function of the imaging device 120, such as the acquisition of image data. The control unit 186 controls at least one function of the evaluation unit 180, such as the evaluation of the spectroscopic data 114 and / or the evaluation of at least one item of image information 128. In particular, the at least one control unit 186 is embodied as and / or comprises at least one processor 188, which is configured to perform one or more operations, in particular by means of software programming.

[0120] List of Reference Numbers 110 Object Information Items 112 Object 114 Spectroscopic Data 116 Spectroscopic equipment 118 Spatial Measurement Range 120 Imaging device 122 Camera 124 Scenes 126 Field of View 128 Image Information Items 130 Step i. 132 Step ii. 134 Step iii. 136 Mobile Devices 138 Smartphone 140 light 142 Light source 144 Detector Equipment 146 Optical Elements 148 Photosensitive element 150 Spectral Curves 152 y-axis 154 x-axis 156 Food 158 Fruit 160 body parts 161 Housing 162 Skin 163 Front Camera 164 images 165 Rear Camera 166 Peak 167 Spectral object information items 168 Spatial Information Items 170 displays 172 Scanning Path 174 Display device 176 screens 178 System for acquiring at least one item of object information 180 evaluation units 182 processors 184 Wavelength Selection Element 186 Control Unit 188 processors

Claims

1. A method for obtaining at least one item of object information (110) relating to at least one object (112) by spectroscopic measurements, comprising the steps of: i. acquiring spectroscopic data (114) using at least one spectrometer device (116) within at least one spatial measurement range (118) of the spectrometer device (116); ii. acquiring image data of a scene (124) within a field of view (126) of the imaging device (120), using at least one imaging device (120), specifically a camera (122), the scene (124) including at least a portion of the object 112 and at least a portion of a spatial measurement range (118) of the spectrometer device (116); and iii. Evaluating the spectroscopic data (114) of step i. and at least one item of image information (128) derived from the image data of step ii. to obtain at least one item of object information (110) relating to at least one object (112), the item of image information (128) including at least one item of similar information of the object (112). Including, The method includes predicting spectroscopic data (114) and / or at least spectroscopically derivable properties for regions of an object (112) where at least one property of the image data is similar to one another.

2. At least one item of image information (128) is at least one of the following: at least one image (164) derived from the image data of step ii; at least one item of spatial information (168) relating to a spatial measurement range (118) in the scene (124), in particular an indication (170) of the spatial measurement range (118) in which the spectroscopic data (114) was acquired in the image (128); at least one item of identification information of at least one object (112), in particular identification information relating to at least one of the following: type of object (112), boundary of object (112) in scene (124), size of object (112), position of object (112), color of object (112), texture of object (112), shape of object (112), contrast of object (112), volume of object (112), area of ​​interest of object (112); at least one item of configuration information relating to at least one object (112), in particular an indication (170) of the configuration of the spectrometer device (116) relative to the at least one object (112); at least one item of directional information, in particular an indication (170) of the direction between the spectrometer device (116) and at least one object (112); at least one item of similarity information regarding the object (112), in particular similarity information regarding at least one shared characteristic shared between different regions of the object (112); The method of claim 1 , comprising:

3. 3. The method of claim 1 or 2, wherein the at least one item of image information (128) comprises at least one image (128) derived from the image data of step ii, and steps i. and ii. are repeatedly performed, and the at least one item of object information (110) in step iii. comprises a combination of spectroscopic object information (167) derived from the repetition of step i. and at least one item of spatial information (168) relating to a spatial measurement range (118) within the scene (124) derived from the repetition of step ii., and wherein at least one of the spatial information (168) and the spectroscopic object information (167) is displayed in the image (128).

4. 3. The method of claim 1 or 2, wherein during the repetition of steps i. and ii., at least one of the scene (124), the field of view (126) and the object (122) is modified.

5. 3. The method of claim 1 or 2, further comprising generating at least one image (128) of a scene (124) having at least two items of spectroscopic object information (167) and corresponding spatial information (168) relating to a spatial measurement range (118) within the image (128) for each item of spectroscopic object information (167).

6. 3. The method according to claim 1 or 2, wherein the image (128) derived from the image data of step ii. is at least one of images (128) derived from the image data of the repetitions of step ii., in particular a combined image (128) of the images (128) derived from the image data of the repetitions of step ii. and a selected image (128).

7. 3. The method of claim 1, wherein at least one item of image information (128) includes at least one item of identification information relating to at least one object (112), and the method comprises applying at least one identification algorithm to the at least one item of image information (128) to derive the at least one item of identification information from the at least one item of image information (128).

8. 3. The method according to claim 1 or 2, wherein step iii. comprises applying at least one spectroscopic evaluation algorithm to the spectroscopic data (114) of step i., the spectroscopic evaluation algorithm being selected according to the item of identification information, in particular according to the type of the at least one object (112).

9. The method of claim 1 or 2, comprising providing a plurality of spectroscopic evaluation algorithms for different items of identification information, in particular different types of objects (112).

10. 3. The method of claim 1 or 2, further comprising providing at least one item of object information (110) relating to the at least one object (112), in particular optically providing at least one item of object information (110) relating to the at least one object (112) via a display device.

11. 1. A system (178) for obtaining at least one item of object information (110) relating to at least one object (112) by spectroscopic measurements, comprising: I. at least one spectrometer device (116) configured to acquire spectroscopic data (114) within at least one spatial measurement range (118) of the spectrometer device (116); II. At least one imaging device (120), specifically a camera (122), configured to acquire image data of a scene (124) within a field of view (126) of the imaging device (120), the scene (124) including at least a portion of the object (112) and at least a portion of a spatial measurement range (118) of the spectrometer device (116); and III. At least one evaluation unit (180) configured to evaluate spectroscopic data (114) acquired by a spectrometer device (116) and at least one item of image information (128) derived from said image data acquired by an imaging device (120) in order to obtain at least one item of object information (110) relating to at least one object (112), the item of image information (128) comprising at least one item of similar information of the object (112), the evaluation unit (180) configured to predict the spectroscopic data (114) and / or at least a spectroscopically derivable characteristic for areas of the object (112) in which at least one characteristic of the image data is similar to one another. A system (178) comprising:

12. The system (178) of claim 11, wherein the spectrometer device (116) and the imaging device (120) have a known, particularly fixed, orientation relative to each other.

13. 12. The system (178) of claim 11, further comprising at least one display device configured to provide at least one item of object information (110) relating to the at least one object (112).

14. A computer program comprising instructions which, when executed by a control unit of a system (178) according to claim 11, cause the system (178) to carry out the method according to claim 1 or 2.

15. A computer readable storage medium comprising instructions which, when executed by a control unit of a system (178) according to claim 11, cause the system (178) to carry out the method according to claim 1 or 2.

Images