Inspection of pharmaceutical objects based on hyperspectral imaging

Abstract

A method for inspecting a plurality of pharmaceutical objects is described, the method comprising: capturing an image of the pharmaceutical object; capturing hyperspectral image data of the pharmaceutical object; selecting one or more hyperspectral image data portions from the hyperspectral image data based on pharmaceutical objects localized within the image; determining a hyperspectral fingerprint based on each of the one or more hyperspectral image data portions, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in the pharmaceutical object; and comparing the one or more hyperspectral fingerprints to a reference fingerprint.

Description

[Technical Field]

[0001] The present invention relates to inspecting pharmaceutical objects, particularly pouches containing pharmaceutical products, based on hyperspectral imaging, and particularly, but not exclusively, to methods and systems for inspecting pharmaceutical objects based on hyperspectral imaging, and computer program products for carrying out such methods. [Background technology]

[0002] Patients are provided with medicines according to prescriptions. In particular, people with chronic diseases need to take the same medicine regularly over an extended period of time. In many cases, patients need to take different medicines, i.e., a combination of pills, tablets, and / or capsules. To facilitate prescribing to patients, the medicines may be packaged in pouches, e.g., clear plastic pouches, blisters, or bags, according to the prescription using an automated packaging system. An error in the packaging of a prescription may result in the patient taking the wrong (combination of) medicines or the wrong dosage of medicines, which may be harmful to the patient's health.

[0003] To reduce the failure rate, pharmaceutical objects are checked by an inspection system configured to inspect the pharmaceutical objects using an image processing system, where the pharmaceutical objects may represent, for example, pills and / or tablets, capsules, ampoules or packets, blisters, or pouches containing the pharmaceutical objects. An example of such an inspection system is known from EP 2951563. To expand the capabilities of such an inspection system, other inspection techniques may be considered. For example, U.S. Patent Application Publication No. 2014 / 0319351 describes an example of an in-line system for inspecting pills placed in blister packages based on near-infrared (NIR) hyperspectral imaging. The inspection system illuminates the pills in the blister package with light from a halogen lamp, and then a hyperspectral image sensor detects 15 response values ​​for 15 bands of the NIR spectrum. The response values ​​are processed to determine the portion of the response value that belongs to the pill's response. These portions are then compared to a reference to determine whether the pill has the correct composition.

[0004] However, building an accurate high-throughput inspection system for pharmaceutical pouches equipped with the above-described hyperspectral analysis capabilities, for example, an inspection system capable of inspecting 10,000 or more pouches per hour, is difficult for several reasons. In contrast to blister packages, in which pills or capsules of a single size, shape, and composition are spatially arranged in a regular manner, multiple pharmaceutical objects within a pharmaceutical pouch may contain different pharmaceutical objects of different sizes, shapes, and compositions spatially distributed in a random order. The pharmaceutical objects may be arranged on their sides, adjacent to each other, or (partially) on top of each other, while the transparent pouch material may introduce errors into the measured data.

[0005] Furthermore, because most pharmaceuticals are primarily composed of the same components (coatings, binder materials, etc.), which often make up a large portion of the pill's mass, the NIR response of pharmaceuticals is a relatively weak signal. Therefore, instead of the 15 values ​​mentioned in the prior art, a large number of spectral response values ​​per pixel, e.g., hundreds or more, are required to distinguish between different pharmaceuticals. In this case, hyperspectral image data typically contains substantial data blocks (data stacks) of data that need to be analyzed in real time, e.g., more than 100 Mbytes per picture. Prior art methods for processing hyperspectral data of imaged pharmaceutical pouches are not suitable for this purpose.

[0006] Therefore, there is a need in the art for improved methods and systems for inspecting pharmaceutical pouches, particularly methods and systems for inspecting pharmaceutical pouches based on hyperspectral imaging in the near-infrared portion of the electromagnetic spectrum that enable accurate, real-time, high-throughput inspection of pharmaceutical pouches. Summary of the Invention

[0007] As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as a system, method, or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware implementation, an entirely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware implementations, all of which may be generally referred to herein as a "circuit," "module," or "system." Functions described in this disclosure may be implemented as an algorithm executed by a microprocessor of a computer. Furthermore, aspects of the present invention may be in the form of a computer program product embodied on one or more computer-readable medium(s), e.g., having computer-readable program code embodied therein.

[0008] The methods, systems, modules, functions, and / or algorithms described with reference to the embodiments herein may be implemented in hardware, software, or a combination of hardware and software. The methods, systems, modules, functions, and / or algorithms may be implemented in a centralized manner in at least one computing system, or in a distributed manner in which different elements are spread across several interconnected computing systems. Any type of computing system or other apparatus adapted to carry out the embodiments (or portions thereof) described herein is suitable. Exemplary embodiments may include one or more digital circuits, such as application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), and / or one or more processors (e.g., x86, x64, ARM, PIC, and / or any other suitable processor architecture), as well as associated support circuits (e.g., storage, DRAM, FLASH, bus interface circuits, etc.). Each individual ASIC, FPGA, processor, or other circuit may be referred to as a "chip," and multiple such circuits may be referred to as a "chipset." In one implementation, the programmable logic device may be provided with high-speed RAM, particularly block RAM (BRAM). Another embodiment may include a non-transitory machine-readable (e.g., computer-readable) medium (e.g., flash drive, optical disk, magnetic storage disk, etc.) having stored thereon one or more lines of code that, when executed by a machine, cause the machine to perform the methods described in this disclosure.

[0009] The flowcharts and block diagrams in the figures may represent the architecture, functionality, and operation of possible implementations of methods, systems, and / or modules for various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code that may be implemented as software, hardware, or a combination of software and hardware.

[0010] It should also be noted that in some alternative implementations, the functions described in the blocks may occur in a different order than that depicted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may, in some cases, be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or a combination of dedicated hardware and computer instructions.

[0011] It is an object of embodiments in the present application to provide an efficient and accurate method for inspecting one or more medical objects, such as a medicine packet containing pills and / or capsules.

[0012] In particular, an objective of embodiments of the present application is to use hyperspectral imaging in a pharmaceutical inspection system so that the system can distinguish between pharmaceutical objects that appear similar and therefore indistinguishable to the human eye (e.g., have the same color and shape) by analyzing image data in the visible spectrum of the pharmaceutical objects. For an accurate pharmaceutical object inspection system, the ability to accurately distinguish between pharmaceuticals based on substance (composition) is critical, since a large number of pharmaceuticals are visually indistinguishable (very often round white tablets).

[0013] The technical advantages of hyperspectral imaging include high spectral resolution (>200 bands instead of the three conventional color bands used with RGB multispectral imaging), which allows for the detection of differences in otherwise similar objects in the visible spectrum. Additionally, it allows for the recognition of various pharmaceuticals based on the invisible (near-infrared) portion of the electromagnetic spectrum.

[0014] In one aspect, the present invention relates to a method for inspecting a plurality of pharmaceutical objects, the method comprising: capturing an image of pharmaceutical objects, preferably pharmaceuticals of different shapes, different sizes, and / or different compositions, randomly arranged in a pouch, the image having a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects in the pouch, the hyperspectral image data having a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels in the image at the first spatial resolution, each of the plurality of blobs of pixels representing one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the plurality of blobs of pixels in the image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, the hyperspectral fingerprint indicative of a spectral response of one or more compounds in the pharmaceutical object; and comparing the hyperspectral fingerprint to one or more reference fingerprints.

[0015] In one embodiment, capturing the hyperspectral image data may include exposing the one or more pharmaceutical objects to light having a continuous spectrum, preferably a continuous spectrum in the visible and / or near-infrared region of the electromagnetic spectrum.

[0016] In one embodiment, the hyperspectral data may comprise a plurality of pixels, where each pixel comprises a plurality of spectral values. is associated with , preferably , the plurality of spectral values ​​are Spectral values ​​in the visible and / or near-infrared regions of the electromagnetic spectrum Contains .

[0017] In one embodiment, one or more single or multi-band images may comprise a 2D grid of pixels, where each pixel is associated with one or a few spectral values, preferably one selected from one or more spectral values, e.g., RGB values ​​and / or IR values.

[0018] In one embodiment, the hyperspectral image data may comprise line-scan hyperspectral image data, wherein the line-scan hyperspectral image data comprises a plurality of lines of a plurality of pixels.

[0019] In one embodiment, the method may further include localizing one or more pixel groups associated with one or more pharmaceutical objects within the one image based on a segmentation algorithm.

[0020] In one embodiment, selecting the one or more hyperspectral image data portions may include mapping each of one or more groups of pixels onto the plurality of pixels of the hyperspectral image data.

[0021] In one embodiment, prior to selecting the one or more hyperspectral image data portions, one or more of the following steps may be performed: removing background pixels (outliers) from the one or more hyperspectral image data using an algorithm, preferably a clustering algorithm; and removing pixels contaminated with specular reflection and / or overexposed pixels from the one or more hyperspectral image data.

[0022] In one embodiment, determining the one or more hyperspectral fingerprints may further include reducing the dimensionality of the one or more hyperspectral image data portions, preferably based on a PCA method; and determining a fingerprint based on at least one of the one or more reduced hyperspectral image data portions.

[0023] In one embodiment, a camera system is used to capture one or more single or multi-band images and hyperspectral image data, preferably the camera system comprises a multispectral camera, and optionally a single band camera or a multi-band camera, e.g., a monochrome camera or a color camera.

[0024] In one embodiment, the hyperspectral image data may be captured using a hyperspectral line scan camera, wherein during the capturing, the pharmaceutical object moves relative to the hyperspectral line scan camera, more preferably, the pharmaceutical object moves through the field of view of the camera system.

[0025] In another aspect, the present invention provides a module for controlling a pharmaceutical inspection device comprising a camera system, the module comprising: a computer readable storage medium having computer readable program code embedded therein; and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein, in response to executing the computer readable program code, the processor captures an image of a plurality of pharmaceutical objects, preferably pharmaceutical objects of different shapes, sizes, and / or compositions, randomly arranged within a pouch, the image having a first spatial resolution; captures hyperspectral image data of the plurality of pharmaceutical objects within the pouch, the hyperspectral image data having a second spatial resolution lower than the first spatial resolution; The pharmaceutical inspection device may be configured to perform executable operations including: determining a plurality of blobs of pixels in the one image at a first spatial resolution, where each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the plurality of blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, where the one hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and comparing the hyperspectral fingerprint to one or more reference fingerprints.

[0026] In a further aspect, the present invention provides a pharmaceutical object inspection apparatus comprising: a camera system; a computer readable storage medium having at least a portion of a program embedded therein; and a computer readable storage medium having computer readable program code embedded therein; and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein in response to executing the computer readable program code, the processor captures an image of a plurality of pharmaceutical objects, preferably pharmaceuticals of different shapes, sizes, and / or compositions, randomly arranged within a pouch, wherein the image has a first spatial resolution; and captures hyperspectral image data of the plurality of pharmaceutical objects within the pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution. determining a plurality of blobs of pixels in the one image at the first spatial resolution, where each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image based on at least one of the plurality of blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, where the hyperspectral fingerprint is indicative of a spectral response of one or more compounds in a pharmaceutical object; and comparing the hyperspectral fingerprint to one or more reference fingerprints.

[0027] In one embodiment, the hyperspectral data may be determined using a hyperspectral camera configured to detect the spectral response of the imaged region in the near-infrared (NIR) portion of the spectrum. In another embodiment, the hyperspectral camera may be configured to detect the spectral response of the imaged region in both the visible and NIR portions of the spectrum. In that case, the hyperspectral camera may generate image data in both the visible and NIR ranges. If the hyperspectral camera is configured to generate both NIR and visible spectral values ​​for each pixel, a separate multispectral camera, such as an RGB or RGB / IR camera, is no longer necessary. In that case, one or more slices of spectral values ​​at one or more wavelengths in the visible spectrum may be obtained from the hyperspectral data stack. Thus, in this embodiment, a monochromatic or multichromatic image can be derived from the hyperspectral image data. Based on this color image, a medical object, such as a pill, can be detected and localized using standard image processing algorithms.

[0028] In one embodiment, the camera system may include a hyperspectral camera and a lamp for illuminating the area imaged by the hyperspectral camera. In one embodiment, the lamp may include a housing and an illumination source. On one side, the housing may include an opening that allows light to exit the housing and illuminate the pharmaceutical object. Typically, the illumination source may be configured to generate continuous spectrum light, such as a halogen lamp, or light. Such illumination sources generate a large amount of heat. Therefore, in some embodiments, the housing may include an outlet that can be connected to a cooling system, such as an air cooling system. In this way, a flow, such as an air flow, can be generated to transport heat from the opening toward the outlet. In this way, the heat generated by the illumination source can be prevented from increasing the temperature around it.

[0029] The present invention may also relate to a method for inspecting pharmaceutical objects, the method comprising: capturing a single band image or a multiband image of a plurality of pharmaceutical objects, preferably pharmaceuticals of different shapes, different sizes, and / or different compositions, randomly arranged in a single pouch; capturing hyperspectral image data of the plurality of pharmaceutical objects in the single pouch; determining a plurality of blobs of pixels in the single band image or the multiband image, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the plurality of blobs of pixels in the single band image or the multiband image; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint is indicative of the spectral response of one or more compounds in the single pharmaceutical object; and comparing the hyperspectral fingerprint to one or more reference fingerprints.

[0030] The invention may also relate to a computer program product comprising software code portions configured to carry out the method according to any one of the method steps described above, when said program product is executed in a computer memory.

[0031] The invention will be further explained with reference to the accompanying drawings, which show, in a simplified manner, embodiments according to the invention, it being understood that the invention is in no way limited to these particular embodiments. [Brief explanation of the drawings]

[0032] [Figure 1] 1 illustrates a pharmaceutical object inspection system according to one embodiment of the present invention. [Figure 2] 1 illustrates a pharmaceutical object inspection scheme based on hyperspectral imaging according to one embodiment of the present invention. [Figure 3] 1 illustrates a flow diagram of a method for inspecting pharmaceutical packets according to one embodiment of the present invention. [Figure 4] 1 illustrates a pharmaceutical object inspection apparatus according to one embodiment of the present invention. [Figure 5] 1 illustrates a system for processing hyperspectral imaging data according to one embodiment of the present invention. [Figure 6] 1 illustrates an example image of a pharmaceutical packet captured by a hyperspectral imaging system. [Figure 7] 7A-7D illustrate images processed based on an image processing method according to an embodiment of the present application. [Figure 8] 8A-8D illustrate images processed based on an image processing method according to an embodiment of the present application. [Figure 9] 1 illustrates an image of a pharmaceutical pouch and a fingerprint of a pharmaceutical object. [Figure 10] 1 illustrates an image of a pharmaceutical pouch and a fingerprint of a pharmaceutical object. DETAILED DESCRIPTION OF THE INVENTION

[0033] FIG. 1 illustrates a pharmaceutical object inspection system in accordance with one embodiment of the present invention. In particular, the figure illustrates an inspection system 100 comprising a transport system 102 for transporting pharmaceutical objects 106, including pharmaceutical pouches containing a plurality of different pharmaceutical objects, through an inspection area configured to inspect the pharmaceutical objects based on an imaging system. The pharmaceutical objects may represent, for example, pills and / or tablets, capsules, ampoules, which may be packaged in packets or pouches, and which may be inspected based on the imaging system. In one embodiment, the imaging system may include one or more camera systems 114 and 116.

[0034] For example, in one embodiment, the first camera system 114 may include one or more image sensors configured to capture a first spatial resolution image of the pharmaceutical object based on a (limited) number of color channels. For example, in one embodiment, the image sensor may include RGB pixels for capturing an RGB color image or an image for each color channel. In a further embodiment, the image sensor may include a spectral channel in the non-visible portion of the electromagnetic spectrum, such as a near-infrared (NIR) channel. The first spatial resolution may be high so that details of the pharmaceuticals in a pouch, including their shape, outline, and text, can be determined very quickly and accurately based on known image processing algorithms. In one embodiment, an NIR camera may be used to obtain high spatial resolution (near-)infrared images of the pharmaceuticals. Such images provide accurate information about the outer contours of the pharmaceuticals in the package. Furthermore, in one embodiment, a color camera may be used to capture high spatial resolution color images of the pharmaceuticals. Based on these images, the location, shape and, for example, color of the pharmaceutical products within the package can be determined very quickly and accurately.

[0035] In further embodiments, the second camera system 116 may comprise a hyperspectral camera system, particularly a hyperspectral camera, that may be configured to perform hyperspectral imaging on pharmaceutical objects. Pharmaceutically active compounds in the pharmaceutical objects respond to near-infrared radiation, particularly near-infrared radiation in the 800-1700 nm range. In this manner, hyperspectral imaging may be a valuable tool for inspecting pharmaceuticals, such as for inspecting pharmaceutical active compounds in pills, tablets, or capsules. Thus, for each pixel of the hyperspectral camera, multiple spectral values, preferably 100 or more, may be detected within a predetermined portion of the electromagnetic spectrum, such as the visible band from 400 nm to 800 nm and / or the near-infrared (NIR) band from 800 nm to 1700 nm. In this manner, the hyperspectral camera may generate a spectral image data stack, where slices of the spectral image data stack at spectral wavelengths may represent an image of a pharmaceutical package at a second spatial resolution, where the second spatial resolution is lower than the first spatial resolution.

[0036] Because the NIR portion of the EM spectrum is particularly suited for determining the response of pharmaceutically active compounds, the spectral values ​​of the hyperspectral image data stack can represent the spectral response of pharmaceuticals captured by the hyperspectral imaging system.

[0037] During hyperspectral imaging, the object may be illuminated using an illumination source 122 that is particularly suitable for hyperspectral imaging. For hyperspectral applications, the illumination source may be selected to have a continuous spectrum in the relevant portion of the spectrum, for example, a continuous spectrum in the UV, visible, and / or near-infrared (NIR) range. Suitable illumination sources for this purpose include incandescent light sources based on highly heated filaments, such as halogen lamps.

[0038] In another embodiment, the hyperspectral camera can be configured to detect the spectral response of the imaged region in both the visible and NIR portions of the spectrum. In that case, the hyperspectral camera can generate image data in both the visible and NIR ranges. If the hyperspectral camera is configured to generate both NIR and visible spectral values ​​for multiple pixels, a separate multispectral camera, such as an RGB or RGB / IR camera, may not be necessary. In that case, one or more slices of spectral values ​​at one or more wavelengths within the visible spectrum can be obtained from the hyperspectral data stack. In some embodiments, a single-band image (e.g., an NIR image) or a multiband image (e.g., an RGB or RGBI image) can be derived from the hyperspectral image data. Based on this image, groups of pixels (blobs) representing medical objects, such as pills, can be detected and localized using standard image processing algorithms.

[0039] A computer 118 may control the imaging system and the transport of the pharmaceutical objects. The computer may further include one or more image processing modules configured to process image data generated by the imaging system so that the pharmaceutical objects can be reliably inspected. The image processing modules may be configured to perform the image processing described with reference to the embodiments of the present application.

[0040] 2 shows a scheme for inspecting pharmaceutical objects based on hyperspectral imaging in accordance with one embodiment of the present invention. In particular, this figure includes a scheme 200 that illustrates the imaging of pharmaceutical objects, in this example pills 201, which may be of different shapes, different sizes, and different compositions and may be randomly arranged in a single pouch. 1~5 , capturing one or more first images, e.g., one or more RGB and / or IR images, at a first spatial resolution of a pharmaceutical pouch 201 containing a portion of a pill, e.g., pill 2012、3 and pills 201 4、5 , may be positioned partially adjacent to or overlapping one another. The one or more first images may be used to localize the pill in the image at a first spatial resolution based on known object detection and segmentation algorithms. In this way, the medicine 201 in the one image may be localized. 1~5 may represent groups of pixels (blobs) within the one image (step 202). Additionally, the pharmaceutical pouch may be imaged with a hyperspectral camera to generate hyperspectral image data at a second spatial resolution lower than the spatial resolution of the one or more first images, i.e., a hyperspectral image data stack.

[0041] The hyperspectral camera can be implemented in various ways. In one embodiment, the camera can be a 2D camera that captures the exposed area including the pouch. Alternatively, in one embodiment, the camera can be a 1D camera, i.e., a line scanner. Such a line scan camera can include an array of light-sensitive pixels that constantly scans a moving object at a high line scan frequency. A two-dimensional image of an object can be generated using the line scan camera as the object moves beneath the camera at a known speed. The data generated by the line scanner can be "stitched" together into a 2D image. The hyperspectral data acquired by a hyperspectral camera can have the form of a "data cube" 204, with the other two dimensions (x and y directions) representing spatial axes and a third dimension representing the spectral response in different parts of the spectrum. In the case of a line scanner, the y axis can be time, as shown in the figure.

[0042] Next, blobs or portions of blobs within the hyperspectral image data may be selected based on groups of pixels, or blobs, localized within the one or more first images. In this manner, hyperspectral data associated with a pill localized in the one or more first images may be determined (step 205). Such hyperspectral blobs may include spectral values ​​206 for a localized pharmaceutical object, such as a pill. These values ​​may represent spectra 208 at pixel locations that are part of the pharmaceutical object. Based on the spectra, a fingerprint may be determined that can be compared to reference fingerprints.

[0043] The high-resolution information in the high-resolution image allows for rapid and accurate differentiation between different medications within a single pouch. Thus, based on a medication localized in the high-resolution image, rapid and accurate selection of hyperspectral image data associated with that localized medication can be achieved. This information can then be used to select the relevant portions of data within the hyperspectral image data required for real-time, high-throughput inspection.

[0044] FIG. 3 is a flow diagram of a method for inspecting pharmaceutical objects according to one embodiment of the present invention. The process may include a first step 300 of capturing one or more first images of a pharmaceutical pouch at a first spatial resolution. In one embodiment, a camera system may be used that includes a high-resolution image sensor, such as a 1440×1080 pixel image sensor, and an optical system that provides a spatial resolution of 0.1 mm per pixel (or approximately 256 pixels per inch, PPI), preferably 0.08 mm per pixel (approximately 317 PPI) or less. In one embodiment, one or more images may be captured while exposing the pharmaceutical pouch to light in one or more portions of the electromagnetic spectrum. Here, at least one of the one or more first images may be an image having a limited number of color channels, such as an RGB image. Furthermore, at least one of the one or more first images may be an infrared (IR) or near-infrared (NIR) image. In further embodiments, such images can be captured using an RGB camera or an RGBI camera, where "I" represents the pixels forming the infrared or near infrared NIR channel.

[0045] In a further step 302, the method may include capturing hyperspectral image data of the pharmaceutical packet. Here, a hyperspectral pixel of the hyperspectral image data may include a plurality of spectral values ​​representing the near-infrared spectral response of the pharmaceutical packet at that pixel location (as described above with reference to FIG. 2). Here, the captured spectral values ​​associated with one wavelength (one slice of the hyperspectral data stack) may form a 2D image with a second spatial resolution, where the second spatial resolution is lower than the first spatial resolution. Typically, the hyperspectral imaging system may include a pixelated image sensor and an optical system providing a spatial resolution at least two times lower than the pixel density associated with the first imaging system, e.g., 0.5 mm per pixel. Due to the lower spatial resolution, it is more difficult to distinguish different objects that are relatively close to each other. In one embodiment, during capture of the hyperspectral image data, the pharmaceutical packet may be exposed to a continuous spectrum of light in the visible and / or near-infrared (NIR) portions of the electromagnetic spectrum.

[0046] The process may further include determining one or more first blobs of a first plurality of pixels representing one or more pharmaceutical products, such as pills and / or capsules, within the one or more first images at the first spatial resolution (step 304). One or more second blobs of a second plurality of pixels may then be selected from the hyperspectral image data based on the locations of the one or more first blobs within the one or more first images (step 306). In step 308, a hyperspectral fingerprint for one of the one or more second pixel groups may be determined, where a hyperspectral fingerprint may indicate the spectral response of one or more compounds in the pharmaceutical product object. The hyperspectral fingerprint may then be compared to a reference fingerprint to determine whether the inspected pharmaceutical product object can be identified as a pharmaceutical product object according to the reference fingerprint (step 310).

[0047] Briefly, therefore, the method provides a very fast, efficient, and accurate method for inspecting pharmaceutical objects based on capturing one image, e.g., a color image of one or more pharmaceutical objects and hyperspectral image data of the one or more pharmaceutical objects. Based on the one or more pharmaceutical objects localized within the high spatial resolution image, one or more hyperspectral image data portions from the hyperspectral image data can be selected, where the hyperspectral image data has a second spatial resolution lower than the first spatial resolution. Thus, the hyperspectral image data portions can be determined quickly and accurately based on the hyperspectral image data. In this manner, hyperspectral pixels associated with the pharmaceutical objects can be determined. Subsequently, the one or more hyperspectral image data portions can be used to determine one or more hyperspectral fingerprints, where the hyperspectral fingerprints indicate the spectral response of one or more compounds within the pharmaceutical objects. These one or more hyperspectral fingerprints are used to determine whether one or more pharmaceutical objects can be identified based on the reference fingerprints.

[0048] 4 illustrates a pharmaceutical inspection device including a hyperspectral imaging system in accordance with one embodiment of the present invention. In particular, the diagram illustrates one or more pharmaceutical objects 402. 1~n4 illustrates an inspection system 400 comprising an imaging system 401 for imaging pharmaceutical products, i.e., one or more pouches containing pharmaceutical products. The system may further comprise a conveying structure 404 comprising a conveying path 406 for guiding one or more pharmaceutical objects through an inspection region of the imaging system. The pharmaceutical objects may include pills, tablets, capsules, ampoules, etc., or packets or pouches containing such pills, tablets, capsules, ampoules, etc., to be inspected based on image data generated by the imaging system. The pharmaceutical objects may be transported to the inspection region via the conveying path when the inspection system is in use. In one embodiment, the pharmaceutical objects may be configured as a series of packets that may be unwound from a first (upstream) reel 4082, guided through the inspection region, and unwound around a second (downstream) reel 4081. Movement of the reels may be controlled by a motor 412.

[0049] Depending on the implementation, the imaging system may include one or more camera systems. For example, in one embodiment, the imaging system may include camera systems 414 and 416 that include one or more multispectral image sensors configured to capture images of the packet based on a (limited) number of color channels. For example, the imaging system may include RGB pixels to capture an RGB color image, or three images for each color channel. In addition, the imaging system may include one or more additional spectral channels, such as a spectral channel in the near infrared (NIR).

[0050] In another embodiment, the imaging system may include a hyperspectral camera system according to any of the embodiments herein. The hyperspectral camera system may include a hyperspectral camera 418 and a lamp 420 for illuminating the imaging area of ​​the hyperspectral camera. In one embodiment, the lamp may include a housing 419 and an illumination source 423. On one side, the housing may include an opening 421 that allows light to exit the housing and illuminate the pharmaceutical object. Typically, the illumination source may be configured to generate continuous spectrum light, such as a halogen lamp. Typically, such illumination sources generate a large amount of heat. Therefore, in some embodiments, the housing may include an outlet 425 that can be connected to a cooling system 422, such as an air-cooling system. In this way, a flow, such as an air flow, can be generated to transport heat from the opening toward the outlet. In this way, the heat generated by the illumination source can be prevented from increasing the temperature of its surroundings. The inspection system may be controlled by a controller 424, e.g., a computer, comprising different modules, e.g., software and / or hardware modules, configured to control the processes required to inspect the pharmaceutical objects.

[0051] In one embodiment, the hyperspectral camera may be configured to detect the spectral response of the imaged region in the near-infrared (NIR) portion of the spectrum. In some embodiments, the hyperspectral camera may also be configured to detect the spectral response of the imaged region in the visible portion of the spectrum. In that case, the hyperspectral camera may generate image data in both the visible and NIR ranges.

[0052] Thus, for each camera pixel, multiple spectral values, preferably 100 or more, can be detected in the near-infrared band, e.g., 900-1700 nm, and / or the visible band, each spectral value thus representing the spectral response of the object, e.g., a pharmaceutical, being imaged by the hyperspectral imaging system.

[0053] The images generated by the first camera system and the second camera system may be processed by an image processing module executed by controller 424. For example, the image data of the first camera system, e.g., a 2D color picture (e.g., an RGB color picture, etc.), may be analyzed using image processing algorithms configured to localize and recognize pharmaceutical objects within the picture based on features, e.g., shape and / or color. Similarly, the image data of the second camera system, e.g., a 3D stack of image data including spectral information (preferably near-infrared spectral information) regarding a pharmaceutical object, may be used to determine a fingerprint of the pharmaceutical object, which may be compared to reference fingerprints in the database to derive information regarding the composition of the pharmaceutical object.

[0054] The hyperspectral camera may be implemented in different ways. For example, in one embodiment, the camera may be a 2D imager. In another embodiment, the camera may be implemented as a line scanner. In the case of a 2D imager, the camera may include a 2D grid of light-sensitive pixels configured to generate 2D hyperspectral image data. The 2D hyperspectral image data may include a plurality of pixels of the imaged area, where each pixel is associated with a plurality of spectral response values. In the case of a line scan camera, the camera may include an array of light-sensitive pixels that scans an area at a high line scan frequency to generate 1D hyperspectral image data for each scan. Two-dimensional images of an object can be generated using a line scan camera when the object moves below the camera at a known speed, or when the camera moves above the object at a known speed. In that case, the 1D hyperspectral image data (lines of pixel data, where each pixel data comprises multiple spectral values) generated by the line scanner may be "stitched" together into 2D hyperspectral image data comprising multiple pixels of the imaged area, where each pixel is associated with multiple spectral response values. Thus, the data acquired by the hyperspectral camera may have the form of a "data cube," with two other dimensions (x and y directions) representing spatial and time axes, respectively, and a third dimension representing the spectral response in different parts of the spectrum.

[0055] In one embodiment, the hyperspectral camera may be configured to generate spectral values ​​in at least the near-infrared (NIR) range of the electromagnetic spectrum (selected wavelengths between approximately 900 nm and 1700 nm). In other embodiments, the hyperspectral camera may be configured to generate spectral values ​​in both the NIR and visible ranges, or only in the visible range. Furthermore, a typical data acquisition for a line scanner may correspond to a "line" of 600 to 1000 pixels, each approximately 200 to 300 μm long. The width of the pixels varies according to the field of view of the lens, but in our case is approximately 300 to 600 μm. All such spatial pixels may contain over 200 spectral values ​​equidistantly spread across a bandwidth between 900 nm and 1700 nm. This diagram is merely a non-limiting example of a hyperspectral imaging system that may be used in a pharmaceutical inspection system according to various embodiments described herein.

[0056] A motor, such as a stepper motor, driving a transport structure (e.g., a conveyor belt) can serve as a trigger mechanism for the camera. At each step of the motor, the camera can be triggered to acquire a pixel line. The conveyor belt can be controlled at a speed of 100-200 mm / s, triggering the hyperspectral camera approximately 300 times per second, thus scanning the object at 300 fps. This means that, taking into account the time required to transfer the data, there is a maximum of 3.3 ms between the acquisition of two consecutive lines, and therefore a maximum exposure time of 3 ms or less.

[0057] The processing of the hyperspectral data may include identifying data related to specular reflections and overexposed areas in the hyperspectral image data (at the packet level) and removing the identified hyperspectral data. Then, in a further step, one or more hyperspectral fingerprints may be determined (at the pill level), where each detected pharmaceutical object (pill, capsule, tablet) may be represented by a blob on the x-y plane of a hyperspectral cube. Overexposed pixels and / or pixels contaminated by specular reflections may be detected so that these values ​​can be excluded from the calculation of the hyperspectral fingerprint. The detection of pixel values ​​that became overexposed during acquisition may be based on a threshold. For example, in one embodiment, a reflectance signal may be determined to be overexposed if it is equal to the maximum value of the sensor's dynamic range. These pixels may be easily filtered from the raw data because their reflectance values ​​are equal to the maximum value of the dynamic range across all spectral bands.

[0058] Pixels contaminated by specular reflections primarily reflect light back to the camera like a mirror, obscuring the object underneath. Figure 6 shows such reflections (e.g., the white areas indicated by reference numerals 602 and 604) in a hyperspectral scan of pouches where the pills in one pouch are not visible due to the pouch's reflection. The reflectance spectrum in those areas may essentially be equivalent to the spectral power distribution (SPD) of the light source itself, which is equivalent to the reflection of the total amount of light emitted.

[0059] Known algorithms can be used to detect such regions. For example, target detection techniques, such as constrained energy minimization (CEM), can be used to detect such regions. CEM is a finite impulse response filter designed to maximize the response of a known target profile while simultaneously suppressing the response of an unknown complex background, thus matching only the known target spectrum. The target spectrum may be the SPD of the light source, which may be approximated based on the reflectance of a white calibration target with a reflectance grade of >95% across the spectrum. The unknown complex background can be represented as a correlation or covariance matrix of all pixels in the xy plane, giving the CEM detector the following equation:

[0060]

number

[0061] where d is the illuminant of the target profile, x is the spectrum of a single pixel, and R is the composite background correlation or covariance matrix. Figures 7A-7D schematically illustrate the process of detecting specular and overexposed pixels and subsequently removing these pixels from the hyperspectral image data, as shown in Figure 6. Here, in Figure 7A, specular reflections are detected based on the target detection technique described above. Similarly, in Figure 7B, overexposed pixels can be determined based on a threshold. Next, both the pixels affected by specular reflections and the pixels affected by overexposure are used to form a pixel mask, as shown in Figure 7C, to identify pixels (and associated spectral values) to be removed from the spectral image data. Figure 7D illustrates the result of applying the pixel mask to the hyperspectral image data. Based on these data, a hyperspectral fingerprint can be determined.

[0062] Extracting hyperspectral fingerprints of individual pharmaceutical objects within a pouch may involve a first step of localizing pharmaceuticals, such as pills, within one or more high-resolution images of the pharmaceutical pouch. Image processing of these images prior to the hyperspectral processing may already provide robust pill detection and segmentation. The contour of a detected blob representing a pharmaceutical can be used to localize the pharmaceutical object inside the pouch. The resolution and pixel size of the high-resolution image may be different compared to the resolution and pixel size of the hyperspectral image, and therefore the contour coordinates need to be scaled so that they can be used to localize the blob of pixels within the hyperspectral data (hyperspectral blob) representing the pharmaceutical object. The scaling factor may be constant for all pouches, resulting in very fast calculation of the tablet's coordinates on the x-y plane of the hyperspectral image.

[0063] Next, outliers (background pixels) can be removed from the hyperspectral blobs. The hyperspectral blobs may contain background pixels because the mapping of coordinates from the high-resolution image to the hyperspectral image may not be accurate. In addition, the position of the pouch or pharmaceutical object within a pouch may change slightly when transported from the exposure area of ​​the color camera to the exposure area of ​​the hyperspectral camera. In such cases, using all pixels specified by this mapping will result in some background pixels being taken into account in the calculation of the pharmaceutical fingerprint. To solve this problem, the selected hyperspectral image data may be clustered into two groups according to their spectral characteristics. For this purpose, in one embodiment, a clustering algorithm, such as a k-means clustering algorithm with k=2 clusters, may be used separately for each blob. In one embodiment, the centroids of the two clusters may be defined as the spectral mean of the entire pouch, representing the background cluster, and the center of mass of the mapped blob, representing the pharmaceutical object. After the clustering algorithm is run, the pixels assigned to the drug cluster can be used for all subsequent calculations.

[0064] Further steps involve denoising and normalising the pixels within the hyperspectral blob. For the remaining valid pixels, the thermal noise of the camera can be subtracted. This can be achieved based on the raw reflectance values. This noise is essentially the signal received by the sensor when the camera shutter is closed (total absence of light). To obtain a robust measurement of the noise, multiple scans can be taken with the shutter closed, and the values ​​for each wavelength can be averaged. The average noise profile thus obtained can be subtracted from the reflectance of each individual pixel. Subsequently, the spectral characteristics of the light source can be removed. This is done to ensure that only the reflectance characteristics of the pharmaceutical object are used in determining the fingerprint. This can be achieved by dividing the reflectance value of each pixel by the average reflectance of the white calibration target described above.

[0065] For every pixel, the logarithmic derivative can be calculated to make the hyperspectral fingerprint invariant to light intensity. The logarithmic derivative of spectrum p in spectral band i can be calculated as follows:

[0066]

number

[0067] where ε is a small positive constant that ensures that division by zero does not occur. This form of derivative is called logarithmic because it uses the ratio between consecutive spectra instead of their difference. Logarithmic derivatives can highlight small structural differences between nearly identical spectra. The logarithmic derivative of the spectrum can be smoothed with a filter, such as a Savitzky-Golay filter, that performs piece-by-piece fitting of a polynomial function, e.g., a second-order polynomial function, to the input signal. The average of the smoothed logarithmic derivatives of all valid pixels for each spectral bin can be calculated, thus reducing the data to a single reflectance spectrum per drug and averaging out noise.

[0068] At this stage, the pharmaceutical objects may be represented by vectors of a given dimension, e.g., 150 or more dimensions. Each dimension may correspond to a different wavelength in the range of 930-1630 nm, and many wavelengths may not have significant discriminatory power between different pharmaceutical objects. Such redundant dimensions contribute nothing to successfully matching pharmaceuticals; in fact, they often degrade the performance of the matching algorithm.

[0069] To obtain the minimum number of dimensions that conveys the maximum amount of discriminatory information, a dimensionality reduction algorithm, such as a PCA dimensionality reduction algorithm, can be used. Such algorithms can be used to detect nonlinear structures in the original data and unfold them into linearly separable projections. In one embodiment, a cosine kernel can be used, which essentially means that the data is projected into a new feature space based on a matrix of pairwise cosine distances between hyperspectral profiles in the reference set. This step may require defining a set of reference pouches in advance, since it is this set that is used to calculate the kernel PCA transform. The wider and more complete the set of reference pouches, the more robust the kernel PCA model will be, especially for a small number of reference patches. After a certain number of pouches, the projection of the feature space "learned" by the kernel PCA algorithm changes little, but this number is estimated to be several hundred pouches.

[0070] FIG. 5 illustrates a method for processing hyperspectral image data according to one embodiment of the present invention. Examples of images during image processing are shown in FIGS. 8A-8D and 9 and 10. In particular, this figure illustrates a method for processing hyperspectral image data based on the steps described above. The method may include capturing an image at a first spatial resolution of a pharmaceutical packet, localizing one or more pharmaceutical objects within the image, and capturing hyperspectral image data from the pharmaceutical packet (step 500). Next, multiple image processing steps may be applied to the hyperspectral data. These steps may include removing background pixels (outliers) from one or more hyperspectral image data portions using an algorithm, such as a clustering algorithm (step 502). Additionally, the method may include removing pixels contaminated with specular reflections and / or overexposed pixels from one or more hyperspectral image data portions (step 504).

[0071] FIG. 8A illustrates an example of a pill localized in a color image. Similarly, FIG. 8B illustrates a hyperspectral image of the pill, and FIG. 8C illustrates an image in which pixels containing specular reflections and overexposed pixels have been removed. Next, one or more hyperspectral image data portions may be determined by mapping one or more localized pharmaceutical objects in the image to the hyperspectral image data (step 506). This step is illustrated by FIG. 8D, which shows the selection of a blob of pixels from the hyperspectral image data based on the pill localized in the color image. In a further step, the dimensionality of the one or more hyperspectral image data portions may be reduced (step 508), preferably based on a PCA method. A fingerprint may be determined based on at least one of the one or more reduced hyperspectral image data portions (step 510).

[0072] 9 and 10 illustrate example fingerprints of two pills of the same pharmaceutical composition, where the fingerprints are calculated based on the data processing steps described with reference to embodiments of the present disclosure. These results demonstrate that the process provides reliable and reproducible results, allowing for accurate testing of pharmaceutical objects.

[0073] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, such as a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset) including the above. Various components, modules, or units are described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined into a codec hardware unit, along with appropriate software and / or firmware, or may be provided by a collection of interacting hardware units, such as a collection of interacting hardware units including one or more processors as described above.

[0074] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that the words "comprise" and / or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0075] The corresponding structure, material, acts, and equivalents of all means-plus-function or step-plus-function elements in the appended claims are intended to include any structure, material, or acts for performing the function as specifically claimed in combination with other claimed elements. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or to limit the invention to the form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The embodiments were chosen and described to best explain the principles and practical applications of the invention and to enable those skilled in the art to understand the invention in various embodiments with various modifications suited to the particular uses contemplated. The present invention may be configured as follows. [Section 1] 1. A method for inspecting a plurality of pharmaceutical objects, comprising: capturing a single image of a plurality of randomly arranged pharmaceutical objects within a single pouch, preferably pharmaceuticals of different shapes, sizes, and / or compositions, wherein the single image has a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects within the one pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels within the one image at the first spatial resolution, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and comparing said hyperspectral fingerprint to one or more reference fingerprints; The method comprising: [Section 2] Item 11. The method of item 1, wherein capturing the hyperspectral image data comprises exposing the one or more pharmaceutical objects to light having a continuous spectrum, preferably a continuous spectrum in the visible and / or near infrared region of the electromagnetic spectrum. [Section 3] 3. The method of claim 1 or 2, wherein the hyperspectral image data comprises a grid of pixels, where each pixel is associated with a plurality of spectral values, and each spectral value is associated with a wavelength in the visible and / or near-infrared region of the electromagnetic spectrum. [Section 4] 4. The method of any one of clauses 1 to 3, wherein the one image of the one or more pharmaceutical objects comprises a 2D grid of pixels at the first spatial resolution, each pixel being associated with at least one spectral value, preferably an RGB value and / or an IR value. [Section 5] 5. The method of any one of claims 1 to 4, wherein the hyperspectral image data comprises line-scan hyperspectral image data, wherein the line-scan hyperspectral image data comprises a plurality of lines of a plurality of pixels, each pixel associated with a plurality of spectral values. [Section 6] 6. The method of any one of paragraphs 1 to 5, wherein determining one or more blobs of pixels in an image is based on an object detection and segmentation algorithm. [Section 7] 7. The method of any one of claims 1 to 6, wherein selecting the one or more portions of hyperspectral image data comprises mapping one or more blobs of pixels in the one image onto pixels of the hyperspectral image data. [Section 8] prior to selecting the one or more hyperspectral image data portions; removing background pixels (outliers) from the one or more hyperspectral image data using an algorithm, preferably a clustering algorithm; and Removing pixels that are contaminated with specular reflections and / or overexposed from said one or more hyperspectral image data. 8. The method according to any one of items 1 to 7, comprising carrying out one or more steps of: [Section 9] determining the one or more hyperspectral fingerprints reducing the dimensionality of the one or more hyperspectral image data portions, preferably based on a PCA method; and determining a fingerprint based on at least one of the reduced hyperspectral image data portions; Item 9. The method according to any one of items 1 to 8, further comprising: [Section 10] 10. The method of any one of clauses 1 to 9, wherein a camera system is used to capture the image of the one or more pharmaceutical objects, preferably the camera system comprises a monochrome camera or a multi-band camera, e.g. a color camera. [Section 11] 11. The method of any one of clauses 1 to 10, wherein the hyperspectral image data is captured using a 2D hyperspectral camera or a hyperspectral line scan camera, and wherein during the capturing, the pharmaceutical object moves relative to the hyperspectral line scan camera, more preferably the pharmaceutical object moves through the field of view of the camera system. [Section 12] A module for controlling a pharmaceutical inspection device equipped with a camera system, comprising: the module comprises a computer-readable storage medium having computer-readable program code embedded therein, and a processor, preferably a microprocessor, connected to the computer-readable storage medium; wherein in response to executing the computer readable program code, the processor: capturing a single image of a plurality of randomly arranged pharmaceutical objects within a single pouch, preferably pharmaceuticals of different shapes, sizes, and / or compositions, wherein the single image has a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects within the one pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels within the one image at the first spatial resolution, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and comparing said hyperspectral fingerprint to one or more reference fingerprints; The module is configured to perform executable actions including: [Section 13] 1. A pharmaceutical object inspection apparatus, comprising: camera systems; a computer-readable storage medium having at least a portion of a program embedded therein; and a computer-readable storage medium having computer-readable program code embedded therein; and a processor, preferably a microprocessor, connected to the computer-readable storage medium, wherein in response to executing the computer-readable program code, the processor: capturing a single image of a plurality of randomly arranged pharmaceutical objects within a single pouch, preferably pharmaceuticals of different shapes, sizes, and / or compositions, wherein the single image has a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects within the one pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels within the one image at the first spatial resolution, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and comparing said hyperspectral fingerprint to one or more reference fingerprints; The pharmaceutical object inspection device is configured to perform executable operations including: [Section 14] A computer program product comprising software code portions configured to perform the method according to any one of claims 1 to 11 when the computer program product is executed in the memory of a computer.

Claims

1. 1. A method for inspecting a plurality of pharmaceutical objects, comprising: capturing a single image of a plurality of randomly arranged pharmaceutical objects within a single pouch, preferably pharmaceuticals of different shapes, sizes, and / or compositions, wherein the single image has a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects within the one pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels within the one image at the first spatial resolution, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and comparing said hyperspectral fingerprint to one or more reference fingerprints; The method comprising:

2. 10. The method of claim 1, wherein capturing the hyperspectral image data comprises exposing the one or more pharmaceutical objects to light having a continuous spectrum, preferably a continuous spectrum in the visible and / or near infrared region of the electromagnetic spectrum.

3. 3. The method of claim 1 or 2, wherein the hyperspectral image data comprises a grid of pixels, where each pixel is associated with a plurality of spectral values, each spectral value being associated with a wavelength in the visible and / or near-infrared region of the electromagnetic spectrum.

4. 4. The method of any one of claims 1 to 3, wherein the one image of the one or more pharmaceutical objects comprises a 2D grid of pixels at the first spatial resolution, each pixel being associated with at least one spectral value, preferably an RGB value and / or an IR value.

5. 5. The method of claim 1, wherein the hyperspectral image data comprises line-scan hyperspectral image data, wherein the line-scan hyperspectral image data comprises a plurality of lines of a plurality of pixels, each pixel associated with a plurality of spectral values.

6. The method of any one of claims 1 to 5, wherein determining one or more blobs of pixels in an image is based on an object detection and segmentation algorithm.

7. 7. The method of claim 1, wherein selecting the one or more portions of hyperspectral image data comprises mapping one or more blobs of pixels in the one image onto pixels of the hyperspectral image data.

8. prior to selecting the one or more hyperspectral image data portions; removing background pixels (outliers) from the one or more hyperspectral image data using an algorithm, preferably a clustering algorithm; and Removing pixels that are contaminated with specular reflections and / or overexposed from the one or more hyperspectral image data. The method according to any one of claims 1 to 7, comprising carrying out one or more of the steps:

9. determining the one or more hyperspectral fingerprints reducing the dimensionality of the one or more hyperspectral image data portions, preferably based on a PCA method; and determining a fingerprint based on at least one of the reduced hyperspectral image data portions; The method of any one of claims 1 to 8, further comprising:

10. 10. The method of any one of claims 1 to 9, wherein a camera system is used to capture an image of said one of the one or more pharmaceutical objects, preferably the camera system comprising a monochrome camera or a multiband camera, e.g. a color camera.

11. 11. The method of any one of claims 1 to 10, wherein the hyperspectral image data is captured using a 2D hyperspectral camera or a hyperspectral line scan camera, and wherein during the capturing, the pharmaceutical object moves relative to the hyperspectral line scan camera.

12. A module for controlling a pharmaceutical inspection device equipped with a camera system, comprising: the module comprises a computer-readable storage medium having computer-readable program code embedded therein, and a processor, preferably a microprocessor, connected to the computer-readable storage medium; wherein in response to executing the computer readable program code, the processor: capturing a single image of a plurality of randomly arranged pharmaceutical objects within a single pouch, preferably pharmaceuticals of different shapes, sizes, and / or compositions, wherein the single image has a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects within the one pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels within the one image at the first spatial resolution, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and Comparing the hyperspectral fingerprint to one or more reference fingerprints. The module is configured to perform executable actions including:

13. 1. A pharmaceutical object inspection apparatus, comprising: camera systems; a computer-readable storage medium having at least a portion of a program embedded therein; and a computer-readable storage medium having computer-readable program code embedded therein; and a processor, preferably a microprocessor, connected to the computer-readable storage medium, wherein in response to executing the computer-readable program code, the processor: capturing a single image of a plurality of randomly arranged pharmaceutical objects within a single pouch, preferably pharmaceuticals of different shapes, sizes, and / or compositions, wherein the single image has a first spatial resolution; capturing hyperspectral image data of the plurality of pharmaceutical objects within the one pouch, wherein the hyperspectral image data has a second spatial resolution lower than the first spatial resolution; determining a plurality of blobs of pixels within the one image at the first spatial resolution, wherein each of the plurality of blobs of pixels represents one of the plurality of pharmaceutical objects; selecting at least one hyperspectral image data portion from the hyperspectral image data based on at least one of the blobs of pixels in the one image at the first spatial resolution; determining a hyperspectral fingerprint based on the hyperspectral image data portion, wherein the hyperspectral fingerprint indicates the spectral response of one or more compounds in a pharmaceutical object; and Comparing the hyperspectral fingerprint to one or more reference fingerprints. The pharmaceutical object inspection device is configured to perform executable operations including:

14. A computer program product comprising software code portions configured to perform the method according to any one of claims 1 to 11 when said computer program product is executed in a memory of a computer.

Images