Surface Profiler

Abstract

An imaging device for measuring a profile of an object is described. The device includes a plurality of moving parts, each having a first end that contacts the object and a second end that includes a fluorescent or phosphorescent material. The device further includes a housing that holds the moving parts, such that the moving parts can move parallel to one another within the housing to change their respective positions according to the profile of the object with which they are in contact. The device further includes a light source that stimulates emission of light from the material, and a camera configured to obtain an image of the second ends of the moving parts based on the emission of light.

Description

[Technical field]

[0001] The present invention relates generally to an imaging device for measuring an image profile of an object, particularly, but not exclusively, for measuring subcutaneous tumors. [Background technology]

[0002] Preclinical scientists and technicians are accustomed to and comfortable using handheld calipers for tumor volume measurement (despite their current inaccuracies). Other imaging-based tools generally require changes to workflow, for example, either bringing the subject animal to the instrument or taking images of a moving subject. Both approaches are currently expensive.

[0003] The use of calipers generally involves taking two distance measurements, for example along the short and long axes, and then estimating the 3D volume. It is usually assumed that the short axis corresponds to height, and that the tumor is a regular sphere, so that the appropriate equations for this can be applied. However, tumors are not regular spheres, and often have lumps and bumps, and can be flat or elevated. This results in imprecise measurements and variability. The variability is especially high when two different people measure the same tumor, for example because they apply different amounts of pressure. Caliper measurements are therefore subject to low precision, high variability, and the need for user training. This variability can lead to reduced reliability of scientific results, and can also mean that more animals (e.g. mice) are needed in a given research study to reach statistically significant results.

[0004] More specifically, in the field of preclinical cancer research, estimating volume through multiple length measurements using caliper-like tools (either manual or digital) is often limited to two of the three dimensions due to accessibility issues, and this third dimension is assumed to be equal to the shortest axis measured. Combined with the variation in pressure applied to the tissue by different operators, this leads to a large variability in volume estimates in preclinical oncology research, especially between different users. When optical reconstruction of 3D surfaces is attempted, many external factors can affect the optical reconstruction of the measured surface, such as environmental light, surface properties, hair / fur, as well as the object not remaining stationary during the measurement.

[0005] It is conceivable to use tools with an array of pins to measure the surface profile, with each pin physically connected to a microprocessor to determine the relative change in the z dimension. However, this may physically limit the pin density and may come at a significant economic cost. Such tools may rely on the user providing downward pressure as well as locking the pins in place.

[0006] In this regard, it is noted that the pins, which are free to move within the array, generally undergo lateral displacement upon extension and therefore generally need to contact the surface to be profiled from the side or bottom. However, subcutaneous tumors in laboratory mice are more easily contacted from above. In addition, due to the small size of the living subject, only a certain level of pressure can be safely applied during measurements.

[0007] For use in understanding the present invention, - Patent Document 1 (Crew et al.) on 3D Profile Gauge US Patent No. 5,399,363 (Rebiere et al.) relating to a method for plotting a three-dimensional shape, in particular a plantar arch, and for operating a system for manufacturing an orthopedic shoe or sole. - Patent document 3 (Bengtson et al.) on the measurement and evaluation of tissue structures such as breasts - Patent document 4 (Larkins et al.) on obtaining data characterizing three-dimensional objects. - Non-Patent Document 1 Please refer to the disclosure. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] International Publication No. 9423605 [Patent Document 2] U.S. Patent No. 6,160,264 [Patent Document 3] US Patent Publication No. 20090112130 [Patent Document 4] International Publication No. 2018 / 109453 [Non-patent literature]

[0009] [Non-Patent Document 1] A pin-array method for capturing tissue deformation under defined pressure distributions and its application to prosthetic socket design, Prince, Kenney & Howard. Medical Engineering and Physics, 84 (2020) pp. 136-143 Summary of the Invention [Problem to be solved by the invention]

[0010] Thus, there remains a need in the field of surface profilers to improve the speed, consistency, comfort, cost, convenience, accuracy, resolution and / or reliability of volumetric estimation of profiled surface features (e.g., in mice, e.g., subcutaneous tumors), capture of larger portions of the surface feature profile, reduced need for user training to perform measurements, reduced number of subjects (e.g., mice and / or tumors) to achieve statistically distinct study results, etc. [Means for solving the problem]

[0011] According to a first aspect of the present invention, there is provided an imaging device for measuring a profile of an object. The device includes a plurality of moving parts, each having a first end in contact with the object and a second end including a fluorescent or phosphorescent material. The device also includes a housing for holding the moving parts, such that the moving parts can move parallel to each other within the housing to change their respective positions according to the profile of the object with which they are in contact. The device further includes a light source for stimulating light emission from the material, and a camera configured to obtain an image of the second ends of the moving parts based on the light emission.

[0012] The device allows for easy imaging of subcutaneous tumors in mice instead of using traditional ultrasound scanners or micro-computed tomography (micro-CT) scanners, which require subjects to be tightly restrained or anesthetized -- traditional techniques that are time-consuming and expensive.

[0013] The moving parts can be posts, for example pins. In any case, these parts preferably slide parallel to each other.

[0014] In a preferred embodiment, the apparatus further includes an objective lens between the moving part and the camera to create a telecentric system. (In an embodiment, such an objective lens can form a telecentric system with the objective lens of the camera when placed at an appropriate distance.) Since a telecentric system generally provides an orthogonal projection and provides the same magnification at all distances, in an embodiment, height-dependent variations can be removed from the image of the moving part, e.g., a pillar such as a pin. This leads to improved performance, since crosstalk that may be introduced by adjacent, e.g., pins, that are physically obstructing each other, is reduced.

[0015] In some implementations, the apparatus further includes a window between the telecentric system and the moving part, the window being transparent to the wavelengths of light emitted by the light source and the material. Such a window may be advantageous for use with a reset mechanism for the moving part, such as a pin reset mechanism.

[0016] In some implementations, the device further includes a mechanism for resetting the position of the moving part, which preferably includes a spring and a switch attached to the window. The switch, when actuated, can cause the spring to reset the position of the moving part to the baseline. Other pin (or other type of moving part) reset mechanisms include, but are not limited to, blowing / pumping compressed air into the housing to set the position of the pin to the baseline, or using a magnetic material attached to the end of the pin and then applying a magnetic field to return the pin to the baseline.

[0017] In some implementations, the moving parts include pin-like posts, and these parts are preferably made of metal, plastic, or fluorescent / phosphorescent material. The use of metal, plastic, or fluorescent / phosphorescent material itself may be based on the cost and / or ease of manufacturing the imaging device. These parts are preferably loose enough that the user does not exert too much pressure on the object to be measured, but not so loose that they fall out of the housing. Lateral movement of parts, e.g., pins, can preferably be avoided to prevent them from colliding with each other.

[0018] In some implementations, the housing includes an array of holes corresponding to each of the components (e.g., pins) so that the components can move freely longitudinally along the axis of the housing. It is noted that in some embodiments, the housing need not be solid, but may be hollow, for example, to reduce friction against the components. This may be advantageous, for example, if the pins bend or tilt sideways, for example, when placed on an object with steep sides.

[0019] In some implementations, the housing includes a stop that limits movement of the movable part out of the housing.

[0020] In some implementations, the fluorescent or phosphorescent material comprises a homogenous layer or coating, preferably sprayed or painted on. However, the fluorescent or phosphorescent material can also be patterned to include, for example, concentric rings or crosses.

[0021] In some implementations, the first end of the moving part that contacts the object includes a rotatable element, preferably a ball bearing. Using this technique allows for better surface contact between the device and the object.

[0022] In some implementations, the first end of the moving part of the imaging device that contacts the object includes a sensor operable to determine a property of the object to be measured, preferably the applied pressure or temperature. In one embodiment, the sensor / mechanism can be mounted to ensure that all users engage the profiler with the target animal with equal force, thereby reducing inter-operator variability. In one embodiment, a thermistor, thermocouple or resistance temperature detector (RTD) can be mounted on each moving part, which can allow for the measurement of the surface temperature for each point in the array. This information can be used to assess the variation in temperature across the surface, allowing for the identification of inflamed areas of the joint.

[0023] In some implementations, the imaging device further comprises at least one spring attached to each of the movable parts and the housing, which allows for determining the force generated to push the parts to their respective positions. Using the imaging device in this way allows for access to the tensile properties of the tissue under examination.

[0024] In some implementations, the imaging device further includes a cap configured to provide a barrier between the moving parts and the object, preferably the cap is disposable.

[0025] In some implementations, the imaging device uses a monochrome camera. However, it is possible to use a color camera with multiple channels that can improve the image. For example, if the fluorescent or phosphorescent material (e.g., paint) of an embodiment emits in the green range, extracting only the green channel from the image can improve the background / noise ratio and / or reduce, e.g., prevent, chromatic aberrations that can affect the focus of light of certain wavelengths.

[0026] According to a second aspect of the present invention, there is provided a method for modelling a profile of an object based on images of moving parts in contact with the object, the method comprising the steps of: (a) obtaining images of illuminated ends of moving parts using a camera; (b) determining, for each said moving parts, a measure of blur; and (c) determining, for each said moving parts, a height or distance of the moving parts based on the determined measure of blur.

[0027] Preferably, the method comprises the steps of: performing a calibration procedure using said object having a predetermined profile, the calibration procedure comprising at least steps (a) and (b), generating a relationship between the degree of blur and the distance or height of at least one moving part based on each determined degree of blur and the predetermined profile; and performing the method using an object having an unknown profile, wherein step (c) is performed based on the generated relationship.

[0028] It is further described that the imaging device or method can be used to obtain a 3D image or model of tissue / tumor excised from a living body.

[0029] Further described is the use of an imaging device or method to obtain a 3D image or model of a subcutaneous mass, preferably of a human, for example where the mass is a cyst and / or is located in the human neck.

[0030] Additionally, the use of the imaging device or method to identify and process objects containing 3D QR codes or Braille is described.

[0031] Further described is the use of an imaging device and method in which the sensor is preferably a temperature sensor operable to measure the surface temperature of a human body to identify inflamed areas of a joint.

[0032] Additionally, the use of an imaging device on the tissue to access the tensile properties of the tissue under examination is described.

[0033] However, the use of the embodiments of the above aspects is not limited to the above applications.

[0034] The invention further provides processor control code for implementing any of the above-mentioned aspects, for example on a general purpose computer system or on a digital signal processor (DSP). The code may be provided on a carrier such as a disk, a microprocessor, a CD or DVD-ROM, a programmed memory such as a non-volatile memory (e.g. flash) or a read-only memory (firmware). Code (and / or data) for implementing embodiments of the invention may include source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code. As will be appreciated by those skilled in the art, such code and / or data may be distributed among several coupled components that communicate with each other.

[0035] According to a related aspect of the invention, there is provided a non-transitory data carrier carrying processor control code which, when executed on a processor, causes the processor to implement the above-mentioned method.

[0036] For a better understanding of the present invention and to show how it may be carried into effect, reference will now be made, by way of example, to the accompanying drawings in which: [Brief description of the drawings]

[0037] [Figure 1] FIG. 1 is an example of an in silico mechanism for testing equivalent variable pin densities and variable measurements. [Diagram 2] 13A-13C are examples of in silico volume estimation using a variety of regular and irregular shapes. [Figure 3-1] Figure 3B shows an example design of a pin-based mechanical profiler with an optical process to determine the relative position of the pin in three-dimensional space. The inset in Figure 3B shows the phosphorescent sphere zoomed in for both the 0 mm and 12 mm images, where the blurring can be clearly observed. [Figure 3-2] Figure 3B shows an example design of a pin-based mechanical profiler with an optical process to determine the relative position of the pin in three-dimensional space. The inset in Figure 3B shows the phosphorescent sphere zoomed in for both the 0 mm and 12 mm images, where the blurring can be clearly observed. [Figure 4] 13 shows results for a single row of pixels at each stage in the pipeline for an example feature extraction algorithm on a synthetic tumor image. [Diagram 5] 13 shows the results for a single row of pixels at each stage in the pipeline for an example feature extraction algorithm on an image of a flat calibration surface. [Figure 6] An example of measurement accuracy evaluation using a defined 3D object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] In general, it describes how a three-dimensional model of a surface is generated through arranging an array of moving parts, e.g., posts, on an object such that the profile of the object is captured in the profile of the parts. In an embodiment, an image of the tops of these, e.g., posts or "pins," is taken and used to estimate the distance each post / pin has traveled, e.g., relative to a given location. This is then used to reconstruct the distance from the ends of the pins at various heights (e.g., relative to a given location) or across the array to the camera, and thus the object being measured in three-dimensional space.

[0039] The embodiment allows for the measurement and quantification of the physical representation of the surface through an array of pins, preferably (i.e., optionally) by capturing a profile of the entire (or substantially entire) tumor surface. In contrast to non-telecentric systems, the height or distance of the pins can be inferred based on images taken from above the pin array through a telecentric system, where objects generally remain the same size regardless of distance from the camera. In general, the image of the ends of the pins does not change with height, but the relative loss of focus does. Thus, the use of a telecentric system may mean that the ends of the pins appear "blurry" as they move towards the camera. The degree of "blurry" may represent the gradient of the transition of image intensity from a baseline or background level (which may be considered a "no signal" level) to a feature intensity level (which may be considered a "signal" level, corresponding to the center of the pin). Thus, edge enhancement for blur determination employs the derivative of the image intensity profile, the Laplacian (i.e., second derivative), or other filters commonly used for edge enhancement, to determine how steeply the intensity decays at each pin to where the edges of each pin are, and how blurred the image is. A blurry image will have a more gradual transition compared to a sharp, focused image. In this regard, it is noted that blurring occurs in both directions, i.e., both as the pin approaches the camera and as it moves away from the camera, e.g., depending on changes in convexity or concavity. Using a set of objects of known but variable heights allows each pin to be calibrated based on the degree of "blurriness" observed. The array can then be placed over the object to be measured, images captured, and the calibration data used to infer height / distance. Since each pin is part of a physical array, the approximate height of each pin can then be used to reconstruct the three-dimensional surface of the object.

[0040] (It is noted that the term "blur" is used generally in this application; however, this can be replaced with, for example, "degree of blur," "extent of blur," or conversely, the "degree" or "level" of "focus.")

[0041] Typically, the tool is configured to have a baseline position for the pin such that image blurring occurs only for pin movement in a single direction. This can be achieved based on the geometry of the optics, including at least the focal length (e.g., 78 mm). The baseline position can be set based on a desired offset of, e.g., 1 mm, so that it generally cannot be moved toward the "wrong" side of focus. The base position can be, e.g., about 1 mm away from the position of the sharpest image, so that the image will gradually become blurrier as the pin moves closer to the camera, and mechanically the pin cannot go any further. A calibration step prior to image capture can provide a reference for each set of one or more measurements on the surface, e.g., a single calibration image used for a set of 100 images. A calibration curve can be provided for each pin, and each pin can have a slightly different curve. Thus, the calibration step can generate a calibration curve for each pin.

[0042] An embodiment may be calibrated based on, for example, 1 mm increments, which may correspond to the height / distance measurement resolution of the device.

[0043] It is further noted that the use of a telecentric system may mean that one pin does not interfere with the imaging of another. In a non-telecentric system, if one pin is taller and therefore has a larger image, it may physically interfere with the imaging of adjacent pins. In a telecentric system, real objects do not appear larger or smaller based on distance, but more or less out of focus. Thus, in the algorithm, each pin position is fixed relative to the other, but after placing the device on a perfectly flat surface (0 mm) and a flat object of known height (12 mm), blurring is seen as in the example of Figure 3B, where the image changes from more focused to more blurred as you get closer to the lens.

[0044] In silico testing identified a preferred density of pins and also confirmed that the approach may be reproducible even when the center of the object is changed relative to the pin array, e.g., simulating different users, and that the approach may avoid optical interference.Embodiments may enable rapid and / or reliable volume estimation of subcutaneous tumors.

[0045] In addition to or instead of using a telecentric system, other embodiments may use other characteristics of the pin array image, such as shape, size and / or intensity.

[0046] To aid in the capture of the image, many strategies can be employed with regard to illumination and / or other energy sources, detection, light path and / or positioning. The inventors have achieved good results by utilizing the properties of phosphorescent and / or fluorescent molecules coated on particles (e.g. spheres or beads, more preferably flat particles, e.g. 1 mm in diameter) attached to the end of the pin. (The optional use of particles can enhance the consistency of the coating of phosphorescent / fluorescent paint compared to directly applying paint to the preferably flat pin end. Alternatively, however, the pin can be formed of phosphorescent or fluorescent material so that paint is not required). Subsequent capture of the image following activation of the molecules by a light source led to a low background image without optical aberrations due to internal reflections seen with other illumination / detection methods. (By using phosphorescence or fluorescence instead of illuminating the end of the pin with a light source and measuring the reflected light from the pin, reflections from internal features of the tool, e.g. from the lens, that generally affect the image are reduced). The molecules can be applied to the end of the pin by painting (e.g. by a brush) mixed with an adhesive or by spraying an aerosol type paint with the molecules. The molecules can be applied to an element, such as a ball, which is attached (eg, glued) to the end of a pin either before or after painting.

[0047] In this regard, it is noted that the images and therefore measurements obtained by an embodiment may depend on the loading (e.g., concentration and / or surface area) of phosphorescent material and the intensity of the light used to excite the molecules. Multiple coatings of material may be applied to increase the concentration and therefore the emission from the end of the pin.

[0048] In embodiments, the reservoir preferably provides enough resistance to hold the pin in place without significant lateral movement and / or difficulty in returning to baseline using a return mechanism such as using a spring-loaded platform within the device itself to physically push the pin back, which may be desirable for contacting a surface to be profiled, such as a subcutaneous tumor in a laboratory mouse, from above and / or applying lower pressure to the surface.

[0049] In this respect, it is desirable for the movement of the pin to be as straight as possible to avoid it coming off the axis and interfering with other pins. Preferably, there is little resistance to the movement of the pin so that the surface, i.e. the object, to be measured is not crushed. However, the pin is preferably not so loose that it moves too much and falls out of the block. A pin material with an appropriate coefficient of friction can be selected. In the home position, the pin protrudes as far as possible before the next measurement is taken. In a preferred embodiment, the tolerances allow for a sliding pin arrangement where the pins fall easily, for example with a striking aid (e.g. by tapping the tool), so that the pins all fall to their home position. Preferably, there is sufficient friction to resist the weight of each pin. However, there may be debris over time, and there may be wear. In addition, a mechanism may be provided to prevent the pins from sliding back towards the camera, in case the device is rotated beyond the horizontal. In this respect, FIG. 3 shows a flat element 12, which is spring-suspended by a spring 13. A user can press a button 14 to push at least a portion of the internal assembly forward to return the pin to its home position. The flat element 12 is preferably transparent, e.g. a glass window. Such a transparent element, provided for pushing the pin back to its original position, may nevertheless serve in embodiments to enable imaging of the phosphorescence, i.e. to enable the pin to be pushed back to its original position without optically interfering with the measurement.

[0050] Advantageously, for example, in embodiments the ends of the freely moving pins are imaged using a camera so that no direct electrical or other connections to each pin are required to measure changes in pin height. This can be combined with optional measurement of additional metrics such as including surface temperature and / or spring on pins with known tensile properties (or other sensors at the pins measuring other biochemical / physiological properties of the underlying tissue) to provide information in addition to the inferred 3D profile.

[0051] The outer ends of the pins can be shaped to allow good contact with the surface to be profiled. The ends can be flat or rounded. A rotatable element, such as a ball bearing, can be provided at each end to allow the tool to glide as it moves over the surface without potentially catching on the skin, for example. This can allow the tool to continuously cover large surfaces and / or to better capture the contours of the surface. Such an embodiment can be coupled with a gyroscope and / or position monitoring system to allow the orientation and / or position of the device to be determined during capture of the surface image. Such information can allow reconstruction of large surfaces.

[0052] In a preferred embodiment, the acquisition of measurements is initiated by non-manual means (e.g., a foot switch or voice) to allow the preclinical scientist / technician to maintain the orientation of the subject and profiler in a stable and safe configuration.

[0053] In a preferred embodiment, the system incorporates a "live mode" in which a series of images are captured rapidly (preferably as close to real-time as possible) to allow improved placement of the profiler over the underlying tissue.

[0054] 3A shows one embodiment having an array of posts or "pins" (1). In this regard, the inventors have found that a minimum number of contact points over a given area is preferred for good measurement accuracy and is easier to manufacture. In an embodiment, this may be, for example, about 50-100 pins over a 2 cm square area.

[0055] The next section relates generally to the mechanical design and overall process design. From the "in silico" data described below, the inventors have determined that an array of approximately 90-100 pins will allow accurate reconstruction of a 3D object. A system built to these specifications is described below.

[0056] FIG. 3A describes one embodiment including an array of, for example, 1.5 mm diameter posts or "pins" (1), suitably made of metal or plastic (polymers may offer advantages, for example, in terms of shape, form and / or cost). The pins are positioned through a block (2) containing a corresponding array of holes so that they have sufficient freedom to move up and down. The shape of the pins and / or the addition of fixed stops inhibit or prevent them from slipping out from either side. In the current embodiment, a plastic sphere (3) spray-painted with three layers of phosphorescent material (e.g., approximately 25-30 microns thick) that emits light in the green (e.g., about 550 nm) is fixed to the camera end of each of the pins. In the current embodiment, this involves a uniform distribution of phosphorescent material across the sphere, but it may be across other shapes and in various patterns (e.g., crosses or concentric circles). The phosphorescent material emits green light after exposure to a light source, allowing the timing of image capture to be appropriately delayed to eliminate internal illumination / reflections on undesirable sides of the assembly.

[0057] The block (2) is mounted in an external housing tube (4) such that one end of the pin array can contact the surface to be measured and the other end is enclosed by the housing. A shroud (5) covers the pin array to protect the pin assembly and can be designed to be of different lengths to expose different lengths of the pin array if required. An additional cap (6) containing a flexible membrane (7) is pressed or locked onto the end of the device to present a protective barrier between the object and the device as well as a more suitable surface for interacting with the pin array. The end of the housing contains a camera (8) with a conventional 8 mm lens and a blue (445 nm wavelength) LED light source (9) with associated microelectronics that is at a specific distance from the pin array when in its lowest position. In the current embodiment, the camera (8) is a CMOS monochrome 5 megapixel camera. However, the use of a color and / or high resolution camera in future embodiments may allow for improved image analysis, for example using the green channel in particular and more reliably detecting blur, respectively. A cable (10) connects the camera to a computer system (not shown). When a measurement is performed, a light source (eg, an LED) is turned on and off to excite the molecules immediately prior to imaging.

[0058] The cap (disposable in embodiments) can be an external, removable (e.g. clip on / off) part. It can provide a membrane across the surface that is preferably attached and removed after measurement. This can provide a flexible membrane material as a barrier between the surface and the pins. The cap can serve multiple purposes such as reducing the ingress of hair and dust into the mechanism, providing a sterile protective barrier between the subject and the device to reduce transfer of pathogens (e.g. between mice), and / or reducing the effect of sticky movement of the pins' surface. Thus, when the cap is attached, the pins can have improved movement.

[0059] Between the camera and the pin array is an objective lens (11) which is combined with the camera lens (8) spaced apart by the focal length of the objective lens (11) to form a telecentric system, which makes the captured image of an object appear sharper or blurrier as it moves axially towards / away from the camera (8), as opposed to a traditional lens where the object appears larger or smaller as it moves towards / away from the camera. With a non-telecentric system, the pins may obstruct adjacent pins when at their highest position, but telecentric lenses generally avoid this problem. In the current embodiment, the objective lens (11) is a plastic aspheric lens with a diameter of 35 mm and a focal length of 53 mm. In another embodiment, the objective lens and / or phosphorescent paint are omitted, such that the ends of the pins are imaged and the height of the pins can be inferred by the relative size of each pin end, relative to its distance from the camera. Between the lens and the pin is a transparent disk (12) connected via a large spring (13) to a button or lever (14) at the end of the housing, which pushes the pin back to its lowest position when the button or lever is depressed.

[0060] To show the change in "blur" when moving the pin closer towards the camera, the device can be placed, for example, on a perfectly flat surface (0 mm) and on a flat object of known height (12 mm), leading to an individual change in the height of the pin and therefore the "blur" (Figure 3B, top two panels). The device is placed on a series of calibration objects at incremental heights (e.g., 1 mm steps) to obtain a numerical value for "blur", which can be displayed as a line graph showing the correlation for each pin between the numerical feature and the distance. Once the distance is calculated for each pin individually, the entire array can be reconstructed using its known x and y coordinates and its interpolated z coordinate (Figure 3B, bottom two panels). A proposed preferred sequence is shown in Figure 3C, which includes a calibration step in which the device is placed on a series of calibration pieces to generate a standard curve for each pin in terms of blur and distance, and a subsequent acquisition step in which the device is placed on the object to be measured.

[0061] However, the design of the embodiments may be modified, for example, with respect to one or more of the following: - Pins (e.g. number, arrangement, shape and / or material of the pins). - Methods for measuring pin height: e.g. standard imaging (size of object = distance to camera), multiple images taken at multiple points in any dimension (x, y, z), physical or projected image distortion. - Assembly: for example contained within one solid unit or as separate units with pins locked in place, extruded or covered. - Optics: including additional light / energy sources, additional detectors, including e.g. mirrors or filters; and / or - Processing: For example using different pin features or variations of them instead of "blurry", using specific channels, cropping or modifying the image before processing, using machine learning / AI to identify features.

[0062] With regard to processing the camera output images, a preferred method involves making height or distance estimates using the "blur" measurements as parameters. In one embodiment, calibration metrics can first be created, including, for example, known "blur" values ​​and known height / distance ranges. These can be used to determine where each pin is located in real space based on an image of the pin's back end. This allows for an accurate determination of where the pin's center is in the physical array. From this, the height / distance (z) as well as the xy position can then be calculated from the interpolation, and a reconstructed 3D volume or 3D image of, for example, a tumor, can then be calculated as the final metric. The interpolation can actually use a 3D graph based on the xy values ​​and blur calculations, which includes lines fitted to interconnect points in space so that the volume can be calculated.

[0063] The blur-based method can be used whether phosphorescent / fluorescent molecules are used or not, and can be applied when, for example, the reflectance of light from a light source by the end of a pin is used. However, a telecentric system is preferably present. However, the molecules can be advantageous to reduce noise, allowing in the embodiment to capture only the emitted light, i.e. not including light that is internally reflected around the inside of the device. Thus, the signal-to-noise ratio of the image can be improved, leading to a clearer image as input to the algorithm.

[0064] The algorithm for determining the distance or height of the pins is now described in more detail. The algorithm generally aims to generate a degree of blur to correlate with the distance or height. In an embodiment, the degree of blur for each pin can be determined by detecting (e.g., using an edge detection process) any areas of high degree or rate of change in intensity (e.g., gradient or curvature) within the image area that may be associated with the pin. In an embodiment, the sum of the degree / rate over the measured local areas can identify the maximum blur for each pin as well as its center location.

[0065] Thus, in some embodiments, for each pixel, the intensity change values ​​(e.g., curvature) measured in a local region (e.g., 10 or 16 pixels) around that pixel are summed, such that the value is at its maximum when the local region includes all of the pixels coming from one pin. This corresponds, for example, to the peak seen in the sixth panel in Figures 4 and 5, which is the degree of "blurriness" (in embodiments, the final reported version of the degree). In the graph, it can be seen that after the summation step, the peak is approximately 0.0007 to 0.2 on the y-axis, as a result of all the local regions around the pixel being summed.

[0066] To further understand the summation, it is noted that in an embodiment, an extensive step preceding the summation can quantify the degree or percentage of intensity change (preferably intensity curvature) at each pixel, and instead of a luminance value for each pixel, we have for example a "degree or percentage of change" value for each pixel based on for example a Laplacian step. After clipping these values ​​to positive values, the summation step can involve summing this clipped value over a preferably disk-shaped area with a radius of a predetermined number of pixels, for example 16, for each pixel, such that as the pixel being calculated moves closer to the center of the pin in the image, the highest value can be obtained when this summation area covers most / all of the pixels associated with the pin. Preferably, there is no selection or removal of any low degree / percentage (e.g. curvature) pixels in the summation step, and the image region is evaluated as a whole. (For any such clipping, it is preferred to sum all positive values. Preferably, absolute values ​​are summed if an embodiment uses second derivatives, otherwise there may be a risk of everything canceling out in the summation operation (note the roughly equal sized positive and negative peaks in subplot 3 of FIG. 4). In general, edges always produce both positive and negative spikes in the Laplacian.)

[0067] The detection of "high" e.g. curvature may be relative to a threshold. As alluded to above, in a preferred embodiment, the detection of sub-regions of high curvature may be performed by using a Laplacian filter. The summation may be performed, for example, using a top-hat filter. A dilation process may be performed so that it does not matter if the pins are in slightly different locations in any calibration image relative to the measurement image.

[0068] In this regard, it is noted that blurred images generally cannot change intensity very rapidly. In one embodiment, the image may therefore first be filtered to highlight areas where the intensity is changing rapidly. Note that there are various options for doing this, e.g., the Laplacian, and looking at the second derivative may give a more different response between sharp and blurred edges. However, other operations, e.g., absolute gradient, higher order derivatives, or other common edge detection filters, may work just as well.

[0069] If the method for detecting sharp edges is the second derivative, for example using the Laplacian as described above, this may also highlight any random noise in the image. Therefore, noise filtering, for example using a Gaussian filter, can be applied to the image as a first step to smooth any noise before performing the Laplacian. However, if a uniform pin image is captured by the camera, such noise filtering may not be very advantageous. Therefore, the degree of Gaussian filtering may be reduced or not used at all, which in turn may provide more sensitive edge detection, especially near the focus point.

[0070] Consistent with the comments above, after creating an image that highlights the "high variation areas," an embodiment may then sum all of the high variation areas associated with a particular pin. However, it is noted that there are many ways to do this. For example, the algorithm could run a blob detection algorithm and then use the center of the blob as the center of the pin, and sum all the pixels around the estimated pin center. The algorithm could then correlate the pin center from the measurement image with the pin center in the calibration image to find the most likely match and select the most appropriate calibration curve. However, a computationally quick approach may be to sum the surrounding pixels of every pixel using a top-hat filter. At every point, the top-hat filtered image can include the sum of the surrounding pixels in the edge-enhanced image. The most intense points in this image will generally be those near the center of the pin, since at these points the top-hat filter will overlap with the most points of high variation and therefore sum them.

[0071] A further, and sometimes final, step is to perform a dilation, which essentially fills in the peaks from the summation step. So instead of explicitly matching the pin locations from the calibration image to the pin locations in the measurement image, the algorithm can simply take the pin locations from the calibration data and look at their exact locations in the dilated measurement image. It doesn't matter if the algorithm doesn't look in exactly the right place, because the dilation step has been performed.

[0072] The next section generally concerns the "in silico" data mentioned above.

[0073] To test the potential of the array of measurements to estimate the volume of an object, we generated a pipeline using ImageJ that utilizes an iterative resize:scale method to simulate the resolution of the array.

[0074] Greyscale shapes from the 3D object were used to correlate the intensity gradient to the height (z dimension) of the object. (In an embodiment, absolute intensity is correlated (e.g., linearly) to height, rather than the intensity gradient (e.g., differential). In this in silico modeling example, the brightness of the test pixel is therefore used as a proxy for the height determined by blur). A 2 cm by 2 cm array area was used in a 200 pixel by 200 pixel array, resulting in 10 pixels / mm. Shapes were constructed in Photoshop, saved as tiff images and processed in ImageJ. Images were resized without averaging or interpolation to 20x20, 10x10 and 5x5 pixel arrays, then scaled back (again, without interpolation) to a 200x200 pixel image to mimic the low resolution of a pin (i.e., 1 pixel=1 pin, FIG. 1A). Using the average intensity, area, known height and maximum intensity, the in silico volume can be estimated using the following equation: Volume (mm 3 ) = (area x average intensity) x (known height / maximum intensity)

[0075] Standard area 400mm 2 Given an object's height = 1 cm and maximum intensity of a grayscale pixel = 255, the volume can be estimated. The center point of the object in the image was radially translated around the original center point to mimic placement of the measurement array at different locations (i.e. inter / intra-operator variability, central location shown in Figure 1B) and the intensity of the object was measured. This was compared to a caliper equivalent length measurement where the "short" and "long" distances were measured and used to estimate the volume using the following equation: - Volume (mm 3 )=π / (6a 2 b) - where a = "short" axis and b = "long" axis, and the height of the object is assumed to be equal to a, just like in modern caliper measurements; or - Volume (mm 3 )=π / (6abc) - where a=the "short" axis and b=the "long" axis, c is the height set to 1, and the height of the object is entered manually as a comparison to the equation above.

[0076] Multiple "short" and "long" measurements were taken around what were considered reasonable measurement points.

[0077] Baseline comparisons in this test used two distance measurements as well as caliper measurements where the height of the object is not measured. The 200 × 200 array gave identical repeated measurements and was the most accurate measurement method. However, in practice, an array of 40,000 pins is not possible. Interestingly, reducing the array size to 20 × 20 or 10 × 10 also provided high precision and minimal variability. The 5 × 5 array had the highest variability, but was still similar to distance measurements of known height and still better than distance measurements where the height is estimated. Overall, the measurement arrays performed well across multiple shape types, with the 10 × 10 array being sufficient to maintain precision and repeatability as shown in Figure 2. Distance-based assessment of volume, such as that used by calipers, can sometimes accurately estimate tumor volume for regular ellipses when the height is known, but generally overestimates the volume if a third dimension is assumed (Figure 2, top left panel, first two rows). However, the ability to accurately estimate irregular shapes containing gaps, additional lumps or lobes is compromised when using distance-based methods, regardless of whether the height is known or assumed (Figure 2, remaining four panels). The measurement array, however, was able to estimate volume with minimal error across all shapes tested.

[0078] The following generally relates to the tumor 3D profile reconstruction algorithm. In one example outline (any one or more of the steps can be omitted): 1. Calculate the degree of "blurriness" for each pin. Create a calibration curve that maps "blurriness" to distance. 2. Calculate the transformation from pixel space to "real" space for each pin. 3. For each pin, use the calibration curve and a transformation from pixel space to real space to calculate the z and x, y coordinates respectively. 4. Perform an interpolation, for example a linear interpolation, between the x, y, z coordinates of the pins and then integrate to get the final volume.

[0079] Additionally, we provide an overview of the calculation of an example of a "blurring" measurement by detecting areas of high intensity curvature and summing these over the area of ​​the pin (one or more of the steps that may be more preferably omitted are indicated by an asterisk). 1. Gaussian Blur (less sensitive to noise)* 2. Laplacian filter to highlight areas of the image with strong edges 3. Clip to positive values ​​to focus on the outer edge of the pin where the strength curvature is positive. 4. Local sum via convolution with a circular top-hat filter. Essentially summing over all areas of positive curvature around the pin. 5. Dilation - Morphological Dilation 1 (fill in the peaks so it doesn't matter if you don't know exactly where the pins are) - call the result of steps 1-5 Image A. (https: / / en.wikipedia.org / wiki / Dilation_(morphology)As noted above, dilation, usually represented by ○+, is one of the fundamental operations in mathematical morphology. A dilation operation typically uses a structuring element to explore and dilate the shapes contained in the input image.)* 6. Sum the raw image without the Laplacian filter. Go back to the original image and perform the same Gaussian blur as in step 1, followed by the filter in step 4, then the dilation in step 5. Call the resulting image B.* 7. Normalization - Divide image A by image B and call this image C. This step is intended to normalize for changes in lighting levels.* 8. Blur degree lookup - Look up pixel values ​​in image C at the expected locations of the pins. The pixel values ​​at these locations are the "blur degree" for each pin.

[0080] Figure 4 shows the results for a single line of pixels across the center of the array at each stage in the algorithm. The final value of the metric is found by looking at the image at the expected location of the pin. The expected location is determined from the calibration image and is shown by the dotted line in the last subplot. The blurriness value is the intersection of the dotted and solid lines in the bottom subplot. The flat areas in this plot can help ensure that there is substantial tolerance to changes in pin location relative to the calibration sequence.

[0081] FIG. 5 shows the results for a single row of pixels at each stage in the pipeline, eg, feature extraction algorithm, for an image of a flat calibration surface.

[0082] To perform step 8 in the above algorithm, it is desirable to know where to find the center of the pin. An example algorithm for this is as follows: 1. Collect N calibration images from a calibration piece that has a flat surface. 2. The N calibration images are summed to form an average image. 3. Perform blob detection using the "Laplacian of Gaussian" method. (https: / / scikit-image.org / docs / dev / api / skimage.feature.html#skimage.feature.blob_log) 4. Use the center of the blob as the pin center.

[0083] It is desirable to know where the pins are in "real" space rather than pixel space so that the 3D volume can be reconstructed. Below is an outline of one example transformation from pixel space to real space: 1. Calculate the center position of the pin in the image as shown above. 2. Load the expected pin positions in "real space" from the CAD data. 3. Run a global optimizer to compute the rotations, scaling and translations that aim to optimally map the image space to the real space. 4. Discard rotation and translation but retain scaling for use in 3D reconstruction.

[0084] Alternatively, if the mechanical variations between parts are small, this conversion can be omitted and a fixed value for scaling is used instead.

[0085] To reconstruct the 3D pin positions, the points for the 3D reconstruction can be calculated by the approach in the following example. 1. Calculate the degree of blur as described above and then calculate the z-score by comparing it with a calibration curve. 2. Calculate the x,y values ​​for each pin by applying a pixel space to real space transformation.

[0086] Finally, the 3D volume can be calculated through linear interpolation between each x, y, z coordinate for each pin.

[0087] We have further explored testing accuracy and repeatability. To test the accuracy of the system in measuring and reconstructing 3D shapes, we placed the device on different shapes with known dimensions. The first object contained 10 "steps", each approximately 1.2 mm high, ranging in height from 1.2 mm to 12 mm (Figure 6A). A calibration image set was captured before placing the device on the object, and images were captured to calculate the height of each pin. The average height detected for each "step" was plotted on an xy graph against the known height of each "step" (Figure 6A). The second object was a bilobed, hemisphere, produced using 3D printed plastic, with a volume of 1059.6 mm. 3 The device was used to measure the objects in repeated measurements (8 independent off / on interactions) and rotational measurements (8 independent interactions with successive 45° angular shift repetitions between images) and the results are described (Figure 6B). The statistics are further described in Table 1.

[0088] [Table 1]

[0089] With regard to use cases, the embodiments may generally be used for clinical evaluation of biological surfaces and materials and / or to analyze three-dimensional geometries designed to store data. The embodiments are discussed herein only in the context of tumors, such as subcutaneous tumors (which generally involve skin-based profile measurements) or excised tumors. However, the embodiments may be applicable to measurements of other human / animal tissues and / or to non-animal / human surfaces, such as plants or other inanimate surfaces.

[0090] More specifically, applications outside of preclinical oncology can include other medical applications such as measuring joint inflammation as part of a rheumatological evaluation, as well as general external surface evaluation such as palpable skin lumps (e.g., a lump on the neck). It should also be noted that embodiments may be useful as a way to decode information stored in 3D geometry in a manner similar to a QR code, but using multiple points along the z-axis to define variables. For example, a standard QR code uses black or white boxes arranged in a pattern that corresponds to information such as a binary sequence. However, 3D QR codes can be considered. By having each box have multiple Z positions, the amount of information that can be stored in each "box" is increased.

[0091] Technical details of examples of clinical evaluations of biological surfaces and materials are described below.

[0092] Surface profiles of externally accessible human tissues, such as skin, skin-based growths or tumors, bones, joints and bony prominences or lesions, or profiles of tissues removed after surgery (i.e. ex vivo) can be generated using current embodiments of the technology using the described methods. This can be used to document basic information about the measured tissue shape and size that may aid in clinical evaluation of the tissue. For ex vivo tissues, multiple pin arrays can also be used to evaluate the entire tissue surface.

[0093] Further information can be obtained by integrating existing technologies into the device. To assess the tensile properties of the tissue under test, a spring with a known spring force can be attached between the pin (1) and the block (2) such that the degree of pin movement is directly proportional to the applied force. If a continuous image of the pin height is obtained during application, the extrapolated force applied can be measured at each step. Furthermore, the addition of a thermistor, thermocouple, or resistance temperature detector (RTD) to each pin (1) allows the measurement of the surface temperature for each point in the array. This information can be used to assess the temperature variation across the surface and aid in identifying key areas of interest (e.g. areas of inflammation in a joint).

[0094] Thus, for example, measurements of joint inflammation in connection with arthritis monitoring may be accomplished by embodiments.

[0095] Technical details of an example of how to analyze data-encoded 3D structures are described below. Embedding coded information within a visual format has been successfully utilized in barcodes and "quick response" (QR) codes. A typical QR code uses black and white blocks to represent an 8-block binary code that encodes specific characters. In contrast to previous iterations of physical codes detected by a 2-dimensional image sensor, a 3-dimensional profile can be captured as previously demonstrated and then digitally analyzed using a programmed processor. In the current embodiment, each pin can be used to distinguish, for example, 128 separate locations per pin, which can then be used to encode information within each pin. For example, a generic 7-bit encoding scheme can be used to encode data such as a URL within a 3-dimensional array of columns. Further information such as orientation, error correction, and encoding format can also be included by using the physical layout, shape, or color of the pins. Custom storage and encoding of data can also be included. For example, using 40 distinct locations, information can be captured using a "base 40" system where each location corresponds to 40 digits, and a group of three pins read in one sequence can represent up to 65,640 digits. Using customized encoding formats, these digits can be used to encode entire words, word families, unique characters or sequences. Applications can also include identification / monitoring of manufactured equipment or products with "3D" QR codes as well as the transfer of information within physically interactive points of interest such as museums or exhibits, potentially interfacing with Braille technology.

[0096] It is noted that the use of blur is a feature of many embodiments as described above. In any of these embodiments, blur can generally be calculated using a series of chained image processing techniques that detect the ends of moving parts (e.g., pins), preferably remove image artifacts / noise, and extract and measure features of interest, such as extrema, gradients and / or intensities. These features or measurements can then be used, alone or in any combination, as or to determine measurements for blur.

[0097] Many other effective alternatives will no doubt occur to those skilled in the art, and it will be understood that the invention is not limited to the described embodiments, but encompasses modifications apparent to those skilled in the art that are within the spirit and scope of the claims appended hereto. [Explanation of symbols]

[0098] 1 pin 2 Blocks 3 Plastic balls 4 Outer Housing Tube 5. Shroud 6 Cap 7 Flexible membrane 8. Camera 9 LED light source 10 Cable 11 Objective lens 12 Disc, flat element 13. Spring 14 Levers, buttons

Claims

1. 1. An imaging device for measuring a profile of an object, comprising: a plurality of moving parts, each having a first end that contacts the object and a second end that includes a fluorescent or phosphorescent material; a housing that holds the movable parts, the movable parts being able to move parallel to one another within the housing to change their respective positions according to the profile of the contacting object; a light source for stimulating light emission from the material; a camera configured to obtain an image of the second end of the movable part based on the emitted light; an imaging device comprising:

2. The imaging device of claim 1 , including a telecentric system between the moving part and the camera.

3. The imaging device of claim 2 , wherein the imaging device includes a window between the telecentric system and the moving part, the window being transparent to wavelengths of light emitted by the light source and the material.

4. 4. The imaging device of claim 1, wherein the imaging device has a mechanism for resetting the position of the movable part, the mechanism preferably including a spring and a switch attached to the window, the switch causing the spring to reset the position when actuated.

5. 4. The imaging device of claim 1, wherein the movable part comprises a pin-like post, the movable part preferably being made of metal, plastic or fluorescent / phosphorescent material.

6. 4. The imaging device of claim 1, wherein the housing includes an array of holes corresponding to each of the movable parts so that the movable parts can move freely longitudinally along the axis of the housing.

7. The imaging device of claim 1 , wherein the housing includes a stop that limits movement of the movable part out of the housing.

8. 4. An imaging device according to any one of claims 1 to 3, wherein the fluorescent or phosphorescent material comprises a homogeneous layer or coating, preferably sprayed or painted.

9. 4. An imaging device according to claim 1, wherein the first end of the movable part that contacts the object comprises a rotatable element, preferably a ball bearing.

10. 4. The imaging device of claim 1, wherein the first end of the movable part that contacts the object includes a sensor operable to determine a property of the object to be measured, preferably an applied pressure or a temperature.

11. 4. The imaging device of claim 1, wherein the imaging device includes at least one spring attached to each of the movable parts and the housing, the spring making it possible to determine the force generated to push the movable parts towards their respective positions.

12. 4. The imaging device of claim 1, wherein the imaging device includes a cap configured to provide a barrier between the moving parts and the object, preferably the cap being disposable.

13. The imaging device according to claim 1 , wherein the camera is a monochrome camera.

14. 4. An imaging system including the imaging device of claim 1, configured to determine, for each moving part, a degree of blur based on the image, and to determine a distance or height of the moving part based on the determination.

15. Use of an imaging device as described in claim 10, wherein the sensor is preferably a temperature sensor operable to measure the surface temperature of a human body and identify areas of inflammation in a joint.

16. 12. Use of an imaging device according to claim 11 on tissue under examination to access tensile properties of said tissue.

17. 1. A method for modeling a profile of an object based on images of a moving part in contact with the object, comprising: (a) obtaining an image of an illuminated end of the moving part using a camera; (b) determining a degree of blur for each said moving part; (c) for each said moving part, determining a height or distance of said moving part based on said determined degree of blur; A method comprising:

18. performing a calibration procedure using the object having a predetermined profile, the calibration procedure including at least steps (a) and (b), and generating a relationship between the degree of blur and the distance or height of at least one moving part based on each determined degree of blur and the predetermined profile; performing the method using an object having an unknown profile, wherein step (c) is performed based on the generated relationship; 18. The method of claim 17, comprising:

19. 19. Use of an imaging device according to any one of claims 1 to 3 or a method according to claim 17 or 18 for obtaining a 3D image or model of a subcutaneous tumor.

20. 19. Use of an imaging device according to any one of claims 1 to 3 or a method according to claim 17 or 18 for obtaining a 3D image or model of tissue / tumour excised from a living organism.

21. Use of an imaging device according to any one of claims 1 to 3 or a method according to claim 17 or 18 for obtaining a 3D image or model of a lump, preferably a subcutaneous lump in a human being, for example a cyst and / or in the human neck.

22. Use of an imaging device according to any one of claims 1 to 3 or a method according to claim 17 or 18 for identifying and processing objects containing 3D QR codes or Braille.

Images