Instruments and methods for characterizing porous materials

Abstract

A system (100) for automated characterization of a porous material sample (200), the system (100) comprising at least one light source (110), a optical sensor (120), and a software module (131), where the at least one light source (110) is arranged to illuminate the porous material sample, where the optical sensor (120) is arranged to capture one or more images of the illuminated porous material sample, and where the software module (131) is arranged to, when executed by a control unit, perform segmentation of the one or more images into areas corresponding.

Description

[0001] INSTRUMENTS AND METHODS FOR CHARACTERIZING POROUS MATERIALS

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to instruments and methods for characterizing porous materials, such as polymeric cellular structures in various foam materials.

[0004] BACKGROUND

[0005] Cellular polymers such as plastic foams and the like are used in many diverse applications, such as thermal insulation, crash absorbers, mattresses, transport packaging and bike helmets. Foam materials can be open or closedcell foams, cross-linked PE (polyethylene) foams, rubber foams, PVC (polyvinyl chloride) foams, polyurethane foam, and so on.

[0006] The properties of a polymeric cellular structure are of course largely determined by the polymer itself, but also of the cell geometry in the polymeric cellular structure. Different cell structure geometries provide different material properties, such as mechanical rigidity and durability, even if the polymer used to create the different structures is the same.

[0007] The cell structure geometry is sometimes challenging to control during production of a porous material such as a polymeric cellular structure, and it is often desired to verify that the cell structure in a produced material is as desired and meets requirements.

[0008] There is a need for more efficient methods to characterize porous materials, such as cellular polymers.

[0009] SUMMARY

[0010] It is an objective of the present disclosure to provide an instrument which allows for automated characterization of porous material samples, such as samples of different types of cellular polymers. This objective is at least in part obtained by a system for automated characterization of a porous material sample. The system comprises at least one light source, an optical sensor, and a software module. The at least one light source is arranged to illuminate the porous material sample and the optical sensor is arranged to capture one or more images of the illuminated porous material sample. The software module is arranged to, when executed by a control unit, perform segmentation of the one or more images into areas corresponding to cell interiors and areas corresponding to cell walls.

[0011] Segmentation of the image into cell interiors and cell walls enables the use of image processing algorithms to determine the cell structure geometry of the sample. The automatic segmentation of the one or more images described herein can advantageously be used in place of applying a dye to the porous material sample, i.e. the process known as inking. This enables a faster characterization process as the step of applying the dye is removed. The automatic segmentation process also can produce results that are more consistent compared to conventional inking, as the automatic segmentation does not depend on applying a dye evenly over the sample surface.

[0012] The software module is executed by a control unit comprising processing circuitry, as will be discussed in more detail below. The processing circuitry can be located in connection to the optical sensor and light source, such as within the same housing or in a nearby location, or remote from the optical sensor and light source, e.g. on a server.

[0013] The software module may be arranged to perform segmentation by means of a machine learning algorithm, where the machine learning algorithm has been trained using samples of images of porous structures and corresponding cell segmentations. According to some aspects, the machine learning algorithm comprises a structure such as a neural network. Machine learning algorithms and structures will be discussed at length below. It is however appreciated that many of the herein disclosed techniques for characterization of a porous material sample can be practiced without machine learning techniques, instead relying on more traditional image processing methods.

[0014] According to some alternatives, the software module is further arranged to characterize the porous material sample in terms of its cell structure geometry, based on the segmented image. This way an automated characterization of a porous material sample is enabled, which allows efficient verification of a produced material. A further advantage of the system is that it provides a more deterministic, operator-independent and repeatable characterization compared to when porous material samples are characterized using manual methods such as inking followed by manual inspection of the sample.

[0015] The software module may be arranged to characterize the porous material sample in terms of cell distribution, cell to material ratio, coalescence, and / or cell size distribution. These metrics all provide valuable information about the porous material sample, allowing an operator to determine if the material meets expectations or not.

[0016] The software module may furthermore be arranged to compare at least one parameter of the determined cell structure to one or more ranges of allowable parameters, and to output the result of the comparison as a verification test result, thus allowing for automated verification of a produced material. The one or more ranges of allowable parameters can, for instance, be selected by the control unit in dependence of a type of porous material sample.

[0017] According to other alternatives, the software module is solely arranged to perform segmentation of the one or more images. Further analysis of the segmented image can then be performed e.g. by another software module or through inspection by a machine operator.

[0018] The optical sensor can for example be any of a camera, a stereo camera, a line-scan camera, and a LIDAR. A regular camera has the advantage of being a simple and relatively cheap sensor, while a stereo camera can provide information about the structure of the sample in three dimensions and a line- scan camera may provide an increased resolution and sensitivity. LIDAR sensors can be arranged to provide detailed three-dimensional information about the structure of the sample. The sensor type can be selected in dependence of the type of porous material sample to be characterized.

[0019] The vision-based sensor is preferably a camera having a resolution of at least 20 pm per pixel in the captured image. A high-resolution image sensor is generally preferred over a lower resolution image sensor. It is an advantage to capture at least five or more pixels per cell in the sample, or preferably at least ten or more pixels per cell.

[0020] According to some aspects, the software module is arranged to adjust the image captured by the optical sensor based on one or more pre-determined sensor calibration parameters. This allows for fine-tuning of the captured image, which improves the result of the segmentation process and the end characterization results. Adjustment of the image can for example be used to achieve more consistent end results with less variation over different batches of processed samples. The pre-determined sensor calibration parameters optionally comprise any of size and scaling aspects, brightness, and contrast over the captured image.

[0021] According to some examples, the optical sensor is arranged to capture two or more images, and the software module is arranged to combine at least some of the two or more images into a combined image and perform segmentation on the combined image. Advantageously, this can reduce detrimental effect of random image artifacts, noise in the optical sensor, etc. The software module may be arranged to combine at least some of the two or more images by performing an averaging operation on the images.

[0022] The at least one light source is optionally arranged to illuminate a surface of the porous material sample from an angle of between 25-65 degrees, or even between 0-65 degrees, and preferably about 45 degrees, measured relative to a normal of the surface. These angles of illumination have been shown to work well with most porous materials. It is an advantage that the light sources of the instrument are adjustable such that the illumination can be tailored for a given sample or for a given porous material. The light sources can also be motorized, such that the control unit can adjust the position of the light source and / or the angle of the light source relative to the porous material sample in an automated manner.

[0023] The instrument preferably comprises two or more light sources arranged to illuminate the surface of the porous material sample from different angles, measured relative to the normal of the surface. The two or more light sources can thus be used to provide optimal illumination, which is an advantage. According to some examples, the two or more light sources are arranged to illuminate the surface of the porous material sample in sequence, and the optical sensor is arranged to capture one or more images during the time that each light source illuminates the surface of the porous material sample. The images captured with illumination from different light sources can subsequently be combined before segmentation, or each image can be segmented and the results of the segmentation can be compared for a more detailed characterization of the porous material sample.

[0024] The at least one light source comprises a light source arranged to emit any of: white light, red light, and blue light. The light source can be a color light source or arranged to emit a blend of two or more colors. It may be an advantage in some cases to adjust the color of the light, since some materials reflect certain colors better than other colors. Glare and unwanted reflection may also be avoided by changing the properties of the light source.

[0025] The system may also comprise a neutral density filter and / or a polarizing filter arranged between the porous material sample and the light source and optical sensor. Such filters may also contribute to avoiding glare and unwanted reflections. The instrument optionally comprises a transparent shelf arranged to hold the porous material sample. In this case the at least one light source and the visionbased sensor can be arranged on an opposite side of the transparent shelf compared to the porous material sample, where they are protected from dust and debris from the sample and from the ambient environment. The transparent shelf also gives a deterministic placement of the porous material sample relative to the one or more light sources and to the vision-based sensor, which is an advantage. The transparent shelf may also comprise the neutral density filter and / or a polarizing filter mentioned above, which improves the quality of the captured image, at least in some cases. A filter integrated with the transparent shelf may be particularly effective in reducing effects of glare and unwanted reflection.

[0026] According to some alternatives, the system comprises a mounting means for mounting the optical sensor and at least one light source. The mounting means is arranged to position the optical sensor so that it faces a surface of the porous material sample. For example, the mounting means may be arranged to position the optical sensor next to one side of the porous material sample, or to hold the optical sensor suspended above a top surface of the porous material sample. Such setups can be integrated as part of a production line, which is an advantage.

[0027] The mounting means may be connected to a linear actuator arranged to move the mounting means along the porous material sample. The optical sensor and one or more light sources can then be moved along the sample to capture images of different parts of the surface. This yields a more complete picture of the sample structure, particularly for samples that are much larger than the field of view of the optical sensor.

[0028] According to some aspects, the control unit on which the software module is executed may be part of the system. That is, the system may comprise a control unit arranged to perform segmentation on the captured images by executing the software module, and optionally also to characterize the porous material sample in terms of its cell structure geometry based on the one or more captured images. The system may further comprise a main body that houses at least some of the components of the system.

[0029] Herein, when the system comprises a main body that houses at least some components of the system, the system can also be referred to as an instrument.

[0030] The control unit may be arranged as an on-board control unit to perform image processing locally. Alternatively, the control unit may be at least partly arranged separated from the main body, i.e. , as a remote processing resource. A remote processing station may be made more powerful, and it can be re-used to process images from more than one instrument, which is more cost efficient compared to local processing. A drawback of having a remote processing station such as a cloud-based processing resource, is that large amounts of image data may need to be communicated between the instrument and the remote processing station. Image compression algorithms may be used with advantage locally to reduce the amounts of data that needs to be communicated between the instrument and the remote processing station.

[0031] The system preferably comprises an access hatch arranged in the main body which hatch is arranged to permit insertion of the porous material sample into an internal volume of the main body. The access hatch may be formed as a top-fed access hatch arranged on a top side of the instrument in use. The hatch allows an operator to insert a sample in a convenient manner.

[0032] The results of the characterization may be displayed directly on the instrument, i.e., on a display unit, or be communicated to some remote location for further analysis.

[0033] The instrument may furthermore comprise a display unit arranged to display a cell segmentation image of the characterized porous material sample. The image displayed on the instrument allows an operator to quickly get an idea of the properties of the analyzed porous material sample. Methods, computer programs, computer readable media, and computer program products associated with the above discussed advantages are also disclosed herein.

[0034] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / an / the element, apparatus, component, means, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated. Further features of, and advantages with, the present invention will become apparent when studying the appended claims and the following description. The skilled person realizes that different features of the present invention may be combined to create embodiments other than those described in the following, without departing from the scope of the present invention.

[0035] BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above, as well as additional objectives, features and advantages, will be better understood through the following illustrative and non-limiting detailed description of exemplary embodiments, wherein:

[0037] Figures 1A-C schematically illustrate systems for porous material characterization,

[0038] Figures 2A-C show an instrument for porous material characterization,

[0039] Figure 3 illustrates a transparent shelf in an example instrument,

[0040] Figures 4,5 illustrate example cell segmentations of cellular polymers,

[0041] Figure 6 is a flow chart illustrating a method,

[0042] Figure 7 schematically illustrates a control unit, Figure 8 shows an example computer program product, and

[0043] Figure 9 is a flow chart illustrating a method.

[0044] DETAILED DESCRIPTION

[0045] The present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments are shown. The disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness. Like reference character refers to like elements throughout the description.

[0046] Figures 1A-C schematically illustrate systems for characterization of porous materials according to the present invention. Figure 1A shows the system 100 comprising at least one light source 110, an optical sensor 120, and a software module 131 . The at least one light source 110 is arranged to illuminate a porous material sample 200, and the optical sensor 120 is arranged to capture one or more images of the illuminated porous material sample 200. The software module 131 is arranged to, when executed by a control unit, perform segmentation of the one or more images into areas corresponding to cell interiors and areas corresponding to cell walls.

[0047] Figure 1A also shows the control unit 130 by which the software module is executed, as well as a data storage unit 135 that is connected to the control unit 130. According to some alternatives, the control unit 130 and I or the data storage unit 135 may be considered part of the system 100. In addition to executing the software module 131 , the control unit 130 can also be arranged to control the operation of at least the optical sensor 120 and the at least one light source 110.

[0048] The figure also shows a main body 150. A main body is herein considered to be a part of the system on or in which at least some parts of the system are mounted. In general, at least the optical sensor 120 and the at least one light source 110 will be mounted on or in the main body.

[0049] The control unit 130 can, according to some examples, be arranged within the main body 150 of the system 100. According to other examples, the control unit 130 can be arranged outside of the main body 150 but in close proximity to the rest of the system 100. The control unit 130 can also be arranged in a place that is more remote from main body 150, light source 110, and optical sensor 120. This will be discussed in more detail below.

[0050] An optical sensor 120 can be any type of sensor using electromagnetic radiation in the optical spectrum to create an image. The optical spectrum is herein considered to include the infrared, visible, and ultraviolet spectra. The optical sensor can be any of a camera, a stereo camera, a line-scan camera, and a LIDAR. However, other types of vision-based sensors can also be used, such as black and white camera sensors, ultra-violet camera sensors, and the like. In some case more than one camera can be used to capture images from different locations and angles. If the system comprises a transparent shelf 140 as shown in Figure 1 A and Figure 3, it is normally preferred to capture an image at a viewing angle that is normal to the transparent shelf 140.

[0051] Note that herein, the terms “optical sensor” and “vision-based sensor” are used interchangeably when referring to sensors such as cameras, stereo cameras, and line-scan cameras.

[0052] According to one example, the optical sensor 120 is a camera having a resolution of at least 20 pm, micrometer, per pixel in the captured image. That is, one side of a pixel in the captured image corresponds to a distance of at most 20 pm on the sample. The size of a pixel in the captured image depends on factors such as the resolution of the camera, the focal length of the camera lens or lens assembly used, and the distance between the camera and the sample. Preferably, one or more of the abovementioned factors is adjusted to achieve a desired resolution in the captured image for a given sample or type of sample. For example, a sample where the expected cell size is on the order of a hundred micrometers may only require a resolution of 20-30 pm per pixel, while a sample where the expected cell size is on the order of 20 pm may require 5 pm per pixel or less.

[0053] Here, the size of a cell in the porous material sample is a distance between two points on the circumference of the cell along a straight line. Preferably, the cell size is taken to be a maximum distance between two points on the circumference of the cell along a straight line. For a circular cell, the size would thus refer to the diameter of the cell.

[0054] The resolution of the captured image in terms of micrometers per pixel may advantageously be such that the expected cell size is at least five times the length of one side of a pixel in the captured image. That is, if the expected cell size is 50 pm, one side of a pixel in the captured image corresponds to a distance of at most 10 pm on the sample. This is herein referred to as having at least five pixels per cell of the porous material. It is preferable to have at least 10-20 pixels per cell. In some cases, the resolution may be only 1 -5 pm per pixel.

[0055] Figures 1 B and 1 C show an example system 100 according to another alternative from two different angles. Here, the main body 150 comprises a mounting means 400 for mounting the optical sensor 120 and at least one light source 110. The mounting means 400 is arranged to position the vision-based sensor 120 so that it faces a surface of the porous material sample 200.

[0056] In the example system 100 shown in Figures 1 B and 1 C, the mounting means 400 serve to position the optical sensor such that it faces a vertical surface of the sample 200, at a suitable distance from the sample horizontally (in the x direction). Here, that the optical sensor faces a surface of the porous sample 200 is taken to mean that the optical sensor is pointed towards the surface so that it can capture an image. For example, if the optical sensor 110 is a camera the lens of the camera is directed towards the surface of the sample.

[0057] According to some aspects, the sample 200 in Figures 1 B-C may be placed on a conveyor device arranged to move the sample to and from the system 100. For example, the sample 200 could be placed on a conveyor belt moving along the z direction of Figures 1 B-C, and the system 100 could be set up to capture images of the sample at intervals as the sample passes in front of the optical sensor. Preferably, the conveyor device is configured so that the porous material sample 200 is stationary in front of the optical sensor 110 during the time when images are captured.

[0058] The sample could also be placed on a conveyor moving along the x direction, i.e. approaching the optical sensor 100. After images are captured by the optical sensor 110, the sample could be moved away along the negative x direction or along the positive or negative z direction. According to some examples, additional processing steps such as cutting the sample may also be performed. Cutting the sample before the image is captured makes it possible to characterize the internal structure of the sample.

[0059] As an alternative, the optical sensor 120 and at least one light source 110 could be suspended above a conveyor device on which one or more porous material samples 200 are transported, with the optical sensor facing down toward an upper surface of the sample 200. The system 100 could then capture images of the one or more samples as they pass beneath the optical sensor 120.

[0060] The mounting means 400 may be connected to a linear actuator 410 arranged to move the mounting means along the porous material sample 200. In Figures 1 B-C, this is shown as the mounting means being able to move along the z direction. Moving the mounting means is particularly useful if the porous material sample 200 is much larger than the field of view of the optical sensor 120 in at least one dimension. In such a case, only a limited part of the sample surface can be captured in a single image. A series of images from different points along the surface then enables a more complete characterization of the porous material sample 200.

[0061] The linear actuator 410 is here shown as controlled by the control unit 130, but it may also be controlled through some other means.

[0062] Segmentation of the one or more images into areas corresponding to cell interiors and areas corresponding to cell walls comprises identification of the boundaries between the cells of the porous sample that is being investigated using the instrument. A porous sample, such as a cellular polymer sample has cavities or bubbles that are separated by walls. The cell structure can be seen if a piece of porous material is cut to form a surface of the sample with visible cells. Segmentation can be done in different ways, such as by edge detection algorithms.

[0063] In many existing characterization methods, the segmentation is facilitated by application of a dye to the sample surface, a process known as inking. Compared to inking, automatic segmentation as described herein yields more consistent and reproducible results and is not dependent on being able to apply an even coating of dye to the surface. Furthermore, it can be done more quickly and without modifying the sample.

[0064] According to some alternatives, the software module 131 is arranged to perform segmentation by means of a machine learning algorithm or machine learning structure, where the machine learning algorithm has been trained using images of porous structures and corresponding cell segmentations.

[0065] Thus, it is understood that the control unit 130, when executing the software module 131 , may perform the porous material characterization at least partly by using a machine learning technique. The machine learning technique involves generation of training data. For supervised learning techniques, generation of training data generally comprises a plurality of different images of porous material samples constituting input to the machine learning structure and corresponding cell segmentation images constituting desired output from the machine learning structure. Thus, supervised training is performed on sets of pairs, where each pair comprises an image of a sample and a corresponding cell segmentation. The machine learning structure can be trained using most known training methods. Supervised learning methods have, for instance, shown good results.

[0066] The machine learning algorithm or machine learning structure may comprise a neural network. The neural network may be a convolutional neural network (CNN), i.e. a neural network arranged to extract features from input data such as images by consecutive application of a plurality of filters. Image processing using CNN is known in the art. According to some aspects, the neural network may be of the ll-net type. A ll-net type neural network is a type of CNN comprising one part that performs feature extraction and one that constructs output data based on the extracted features.

[0067] Visual transformer structures and masked autoencoders are other known machine learning structures that may be used with advantage in this application. Such machine learning structures can advantageously be trained using unsupervised learning methods.

[0068] The training of the machine learning structure is terminated when an acceptable cell segmentation performance has been reached. Various metrics for determining when training has been completed can be used. For instance, a ground truth histogram of cell sizes or of some other characterization metric can be obtained, e.g., from manual inspection or from historical data, and training can be terminated when the machine learning structure outputs a histogram that is similar enough to the ground truth. An experienced operator may also be tasked with comparing the results of the machine learning structure to his or her experience and terminate the training when sufficiently good results are obtained. A set time limit for training can also be used, such as training for 24 hours or some other time period. Alternatively, training can be ended when the performance improvement between each consecutive training step is smaller than a predetermined threshold value. When generating the training data for the machine learning structure, it may be advantageous to add noise and distortion to the input data, while maintaining the cell segmentations of each pair. This way a single pair can be duplicated into a plurality of pairs by adding various forms of noise and distortion to the image corresponding to the image captured by the vision-based sensor. The added noise may, e.g., be configured to emulate saw marks, ruggedness and other defects that can be expected in real world samples.

[0069] The software module 131 may further be arranged to characterize the porous material sample 200 in terms of its cell structure geometry based on the segmented image. This means that the software module 131 comprises one or more software routines that are used to process the one or more captured images of the sample after segmentation in order to determine various aspects of the geometry of the cell structure of the sample. The characterization operations performed by the software module and the control unit are automated characterization algorithms which are performed with little or no interaction with the operator. This is an advantage since it saves time and also produces a more reliable and reproducible characterization result compared to if the operator should have performed the characterization manually. The software module may for example be arranged to, when executed by the control unit 130, characterize the porous material sample in terms of cell distribution, cell to material ratio, coalescence, and / or cell size distribution.

[0070] The cell distribution of a sample is a measure of how the cavities in the material are distributed over the sample surface. The cell distribution can be even or less even. A less even cell distribution is characterized by a larger number of cavities in some parts of the sample compared to in other parts of the sample. A measure of cell distribution can be obtained by measuring the number of cavities per area unit of the porous material sample.

[0071] The cell to material ratio is a measure of the thickness of the walls in between cells. A high cell to material ratio is indicative of thin cell walls, while a low cell to material ratio is indicative of thicker cell walls. The distribution of cell to material ratio over the sample may also be of interest. It is normally preferred that the cell to material ratio is even over the sample.

[0072] The coalescence metric is indicative of how often two neighboring cells have merged into a common cavity due to damage in the cell wall in between the cells.

[0073] The cell size distribution is a measure of the cross-section area of the cells in the sample. It is preferably presented as a histogram of cell sizes in the sample. Figures 4 and 5 show two different cell segmentations for two different porous material samples. The example 510 has relatively large cells of varying cell size. The sample has rather evenly distributed cells over the sample, i.e. , there are no places on the sample where cells are sparser compared to the average cell density. The cell sizes in the example 520 are smaller compared to the example 510. Thus, the two samples are distinguishable in terms of their cell size distribution.

[0074] The software module 131 may be arranged to compare at least one parameter of the determined cell structure to one or more ranges of allowable parameters, and to output the result of the comparison as a verification test result. The one or more ranges of allowable parameters can be selected by the control unit in dependence of a type of porous material sample, and if of course configurable by an operator of the instrument.

[0075] According to aspects, the software module 131 is arranged to adjust the image captured by the vision-based sensor based on one or more pre-determined sensor calibration parameters. The pre-determined sensor calibration parameters may comprise any of size and scaling aspects, brightness, and contrast over the captured image.

[0076] The optical sensor 120 may be arranged to capture two or more images, and the software module 131 may then be arranged to combine at least some of the two or more images into a combined image and perform segmentation on the combined image. Combining at least some of the two or more images may be achieved by performing an averaging operation on the images.

[0077] Capturing two or more images and combining them into one image has the benefit that stochastic noise in the optical sensor and other random image artifacts may be removed, or at least reduced, in the resulting combined image. Optionally, combining images could also comprise removing an image that is an outlier from the set of images. For example, if five images are obtained and one is significantly less well-lit due to a fluctuation in the light source 110, the less well-lit image could be removed. Such outlier removal can be automated using known image processing algorithms.

[0078] Performing an averaging operation on the images could for example comprise averaging the intensity values for each pixel.

[0079] According to some aspects, the system 100 comprises two or more light sources 1 10 arranged to illuminate the surface of the porous material sample 200 from different angles, measured relative to the normal of the surface of the sample. This way the quality of the captured image is improved, which in turn improves the end characterization result. Various types of light sources can be chosen, and it is appreciated that some types of light are more suitable for characterization of a given material than others. The light sources in the instrument 100 can therefore be configured as replaceable light sources, that can be disconnected in a convenient manner and replaced by another type of light source more suited for a material to be characterized. The at least one light source may for instance comprise an LED-based light source arranged to emit any of: white light, red light, and blue light. A combination of two or more colors of light is beneficial for some materials, such as a combination of blue and red light at different angles.

[0080] The light sources may be motorized light sources. This allows the control unit to change the positions of the light sources relative to the sample, and also the illumination angle of each light source, in order to capture a plurality of images with different illumination settings.

[0081] The two or more light sources 110 may be arranged to illuminate the surface of the porous material sample 200 in sequence, and the optical sensor may be arranged to capture one or more images during the time that each light source 110 illuminates the surface of the porous material sample 200. Put differently, the control unit may, e.g., be configured to step through a sequence of different light source settings, including potentially changing the color of the light, and capture an image for each light source setting in the sequence. Thus, a series of images can be obtained where the sample is illuminated from different angles, with different colors, etc.

[0082] According to some examples, a neutral density filter and / or a polarizing filter is arranged such that the porous material sample 200 is on one side of the filter and the light source 110 and the optical sensor 120 are on the other side. A neutral-density filter, or ND filter, is a filter that reduces or modifies the intensity of all wavelengths, or colors, of light equally, giving no changes in hue of color rendition. It can be a colorless (clear) or grey filter. A polarizing filter can be used to manage reflections and to suppress glare from the surface of the porous material sample. ND filters and polarizing filters are well known in the art and will therefore not be discussed in more detail herein.

[0083] In the alternative implementation of the system shown in Figure 1 B, such filters may for example be arranged mounted on the mounting means 400 such that they extend in front of the light source 110 and optical sensor 120.

[0084] In an implementation as shown in Figure 1A and Figure 3, the system 100 comprises a transparent shelf 140 arranged to hold the porous material sample. The at least one light source 110 and the vision-based sensor 120 are arranged on an opposite side of the transparent shelf 140 compared to the porous material sample. The shelf thus functions as a window through which the light source and the vision-based sensor can interact with the sample. The shelf also protects the light sources and the vision-based sensor from dust and debris, which may otherwise hamper operation. Here, the transparent shelf may have an integrated neutral density filter and / or a polarizing filter which improves the quality of the images captured by the vision-based sensor.

[0085] Figures 2A-C show different views of an instrument 100 for characterization of a porous material sample, such as a sample from a cellular polymer material. It may be noted that herein, the term “instrument” will occasionally be used to refer to the system 100, particularly when describing alternatives that comprise a control unit 130 and a main body 150.

[0086] Figure 2A is an isometric view, Figure 2B is a side view, and Figure 2C is a cross section view along the line A-A indicated in Figure 2B. The instrument 100 illustrated in Figures 2A-C constitutes an example embodiment of the techniques discussed herein. It is however possible to practice the methods and techniques for characterization of porous materials using other instrument designs. The instrument can, for instance, be integrated into a production line for plastic foam material or products. The instrument can then be arranged in connection to a conveyor belt or the like, where it is positioned to perform automated characterization of materials or products transported on the conveyor belt.

[0087] The instrument 100 comprises at least one light source 110, a vision-based sensor 120, a control unit 130, and a main body 150. A porous material sample, for instance a cellular polymer sample such as a piece of plastic foam, is inserted into the instrument 100, where the at least one light source 110 illuminates the porous material sample, i.e. , at least one surface of the sample. The vision-based sensor 120 then captures one or more images of the illuminated porous material sample. Only one image is sufficient for characterization of a porous material sample. However, additional images of the same sample may improve the end results of the characterization, since more images potentially carries more information about the cell structure on the sample, especially if the captured images have been illuminated in different ways. A drawback of capturing many images is of course an increase in the amount of data that needs to be handled by the instrument.

[0088] The actions of the at least one light source 110 and the vision-based sensor 120 are preferably controlled by a control unit 130 in an automated manner. Thus, once a sample has been inserted into the instrument, a user can trigger a characterization, e.g., via the display unit 170, whereupon an automated characterization process ensues, that comprises activation of the one or more light sources, capturing of one of more images, and subsequent processing of the captured images. The result of the characterization can then be displayed directly on the instrument 100 using the on-board display unit 170, and also sent to a remote location for further analysis.

[0089] This characterization is the result of a computation performed using the image captured of the illuminated sample of porous material placed in the instrument 100. The result of the characterization can be displayed on the display 170 and / or transmitted to some external system.

[0090] The instrument 100 comprises an access hatch 160 arranged in the main body 150. The access hatch 150 is arranged to permit insertion of the porous material sample into an internal volume of the main body 150. The access hatch is preferably a top-fed access hatch arranged on the top side of the instrument 100 in use, as illustrated in Figures 2A-C. A top-fed access hatch presents a convenient way to load samples into the instrument. The hatch and the main body are preferably formed in an opaque and durable material, in order to reduce the impact of ambient light on the characterization process and also to protect the components of the instrument from damage.

[0091] A ventilation opening 180 may be arranged in the instrument main body 150. This opening facilitates cooling of the at least one light source 110 and the control unit 130. The vision-based sensor 120 may also need cooling. An interface 190 provides electrical power and potentially also a high data speed communications interface to the instrument, allowing communication with the instrument from a remote location at high data rate.

[0092] The at least one light source 110 (Figures 2A-C show an example with two separate light sources) is arranged to illuminate a surface of the porous material sample from an angle of between 25-65 degrees, and preferably about 45 degrees, measured relative to a normal of the surface. Some light sources may be arranged to emit light at an angle that ranges between 0-65 degrees. A surface normal is a vector that extends perpendicular to a point on the surface. The porous material samples placed in the instrument are normally rectangular blocks of material, although other forms of samples can also be placed in the instrument 100. The surface normal in the example illustrated in Figures 2A-C is directed downwards in use, perpendicular to the transparent shelf 140. In some example realizations the light source or light sources are motorized, allowing the control unit 130 to control both the location of the light source relative to the porous material sample as well as the illumination angle measured relative to the sample surface normal. This allows the control unit to acquire images of the same object but with different illumination configurations, which may improve the cell segmentation result in some cases.

[0093] The control unit 130 may be comprised in the instrument itself, i.e., enclosed by the main body structure 150 of the instrument. However, additional advantages may be obtained if the control unit 130 is at least partly arranged separated from the main body 150 of the instrument 100, as illustrated in Figure 1 A. In this case the control unit 130 may form part of a remote server in a cloudbased solution. An advantage of this arrangement is that the same server can be used in many different instruments, possibly all over the world. A cloudbased solution may allow for more powerful processing of the captured images and is also more easily upgraded. A high-speed data interface 190 is an advantage in case a remote server is used for processing of the captured image data. Image compression by an on-board control unit may also be an advantage since this reduces the amount of image data to be communicated to the remote server.

[0094] The display unit 170 can be arranged to display a cell segmentation image 510, 520 of the characterized porous material sample as shown in Figures 4 and 5. This displayed cell segmentation surface may be useful to an operator, in particular in case the operator is an experienced operator.

[0095] The different operations of the instrument 100 during a characterization of a porous material sample will now be summarized in an example. After the porous material sample is illuminated and one or more images are captured of it, the software module and control unit perform pre-processing steps before a segmentation is created.

[0096] The pre-processing step depends on the type of optical sensor that is used. For a camera, the captured image is first transformed given a pre-determined set of camera calibration parameters. The calibration can, for instance, be done by taking a picture of a checkerboard pattern. Parameters that yield an undistorted image can then be calculated from this image having knowledge of the dimensions of the checker-board pattern. The calibration images and parameters can for example be obtained during set-up of the instrument. The pixel to millimeter ratio must be measured during the set-up of the instrument as well. This ratio is needed in the last step when calculating the sizes of all cells. The pixel to millimeter ratio sets a limit on how small the cells can be in order to be detected by the camera. The pixel to millimeter ratio is preferably configured to be significantly smaller than the smallest cell. If the camera lens is changed, new camera calibration parameters can be obtained and a new pixel to millimeter ratio can be determined.

[0097] For other optical sensors such as stereo cameras and line-scan cameras, the calibration procedure is adapted to the sensor type. For stereo cameras in particular, calibration includes determining the distance between the two cameras comprised in the stereo camera as well as their relative rotation. This can also be accomplished using a checkerboard pattern.

[0098] According to one alternative, at least one light source 1 10 could be arranged to provide illumination with structured light. This could be used to determine the displacement and angle of the sample relative to a vision-based sensor such as a camera or stereo camera. Such information can for example be used to ascertain that the sample is correctly positioned, as well as to obtain the pixel to millimeter ratio.

[0099] The brightness and contrast are also adapted according to predetermined specification. This is to obtain an increased contrast but also an equal contrast over the whole captured image.

[0100] The captured image is then fed to a machine learning structure such as a convolutional neural network that outputs a segmented picture. In the segmented picture the cells are represented, e.g., by white and the wall material by black, as exemplified in Figures 4 and 5. From this segmentation it is straight-forward to determine the characteristics of each cell, and to obtain the desired statistics of the cell structure which make up the characterization of the porous material sample.

[0101] An important part in the characterization of a porous material sample is the processing by the machine learning structure. This machine learning structure has been trained a-priori on a large amount of input output pairs of captured images or different porous material samples with corresponding cell segmentation images. The input data thus comprises a set of pairs, where each pair comprises an image of a porous material, and a corresponding segmentation.

[0102] To increase the amount of training data, the data can undergo data augmentation. This includes e.g., rescaling, noise adding, brightness change, blurring, rotation etc. The un-augmented input images are samples of a wide range of materials with cells, not only plastic foams. The rationale behind this is to increase the generality of the model. The machine learning structure can be a neural network such as a deep convolutional network of ll-net type. The inference time and accuracy are highly dependent on the number of layers and size of the layers. The training of the machine learning structure can vary, although it has been found that standard supervised learning works well, where the model when given an input image should output a segmentation as close as possible to the one provided in the training data.

[0103] The data used for training can be obtained in at least three different ways. The first method is to take an initial image of a material. This is the input image. The material is then pressed against ink which marks the outer cell walls as in Figures 4 and 5. This cell segmentation is then photographed and postprocessed to get an image with cells being white and cell walls being black. This is the output image.

[0104] The second method is to synthetically create data. To create the segmentation, a mesh is selected. Preferably, the mesh should comprise a plurality of irregular polygons. Optionally, the mesh may be based on a Voronoi diagram. The edges of the polygons in the mesh are then represented as cell walls in the segmentation. Subsequently, a 3D model of a material with a corresponding cell structure is created through 3D rendering. The material can be rendered with different surface properties and under different lighting conditions in order to provide diverse training data. A 2D image of the 3D rendered model is then obtained. This 2D image along with the segmentation then forms a pair of input and output images for training the neural network.

[0105] The third method is to feed input images of materials to a model trained on data obtained using the two previous methods. These outputs are finetuned and then added to the training data.

[0106] There are also other methods for segmenting cells in materials that can be used by the control unit 130, either as an alternative or as a complement to the machine learning structure. These methods include known thresholding methods and flood-fill methods.

[0107] The above discussion can be summarized in terms of the methods illustrated by the flow charts in Figures 6 and 9. Figure 6 illustrates a method for automated characterization of a porous material sample. The method comprises illuminating S1 the porous material sample by at least one light source 110, capturing S2 one or more images of the porous material sample by a vision-based sensor 120, and characterizing S3 the porous material sample in terms of its cell structure, based on the one or more captured images, by a control unit 130.

[0108] It is appreciated that the different hardware aspects and optional features of the instrument 100 can also be cast as corresponding method steps. Thus, the method may comprise:

[0109] Illuminating S11 a surface of the porous material sample from an angle of between 25-65 degrees, and preferably about 45 degrees, measured relative to a normal of the surface.

[0110] Illuminating S12 the porous material sample by two or more light sources 110 from different angles, measured relative to the normal of the surface.

[0111] Illuminating S13 the porous material sample by a light source arranged to emit any of white light, red light, and blue light, or a combination thereof.

[0112] Capturing S21 the image of the sample by a camera having a resolution of at least 20 pm per pixel in the captured image.

[0113] Arranging S22 the porous material sample on a transparent shelf 140, where the at least one light source 110 and the vision-based sensor 120 are arranged on an opposite side of the transparent shelf 140 compared to the porous material sample. The method may also comprise integrating a neutral density filter and / or a polarizing filter in the transparent shelf. Characterizing S31 the porous material sample in terms of cell distribution, cell to material ratio, coalescence, and / or cell size distribution.

[0114] Comparing S32 at least one parameter of the determined cell structure to one or more ranges of allowable parameters and outputting the result of the comparison as a verification test result. In this case the one or more ranges of allowable parameters can be selected by the control unit in dependence of a type of porous material sample.

[0115] Displaying S33 a cell segmentation image 510, 520 of the characterized porous material sample on a display unit of the instrument 100.

[0116] Adjusting S34 the image captured by the vision-based sensor based on one or more pre-determined sensor calibration parameters. In this case the predetermined sensor calibration parameters may comprise any of size and scaling aspects, brightness, and contrast over the captured image.

[0117] Characterizing S35 the porous material sample by using a machine learning structure, where the machine learning structure has been trained using samples of images of porous material samples and corresponding cell segmentations.

[0118] The method may also comprise generating training data SO and configuring the machine learning structure of the control unit 130 based on the training data. The training data generally comprises pairs, where each pair comprises an image representation of a sample (corresponding to images captured by the vision-based sensor) and cell segmentation images (corresponding to the output of the machine learning structure). The image pairs may be created in various ways, such as taking photos of different porous material samples and performing a manual cell segmentation by inking, or by manually marking cell edges. Computer-based methods may also be used, comprising rendering virtual images of samples and corresponding cell segmentations.

[0119] Each pair of captured image and corresponding cell segmentation can be duplicated into several pairs by adding various forms of noise and distortions to the image of the porous material image, while keeping the cell segmentation of the original captured image. The added noise and distortion may comprise, e.g., blurring, reflections, glare, saw marks, scratch marks, and ruggedness. This way the machine learning structure can be trained to disregard these artefacts in the image data obtained from the vision-based sensor 120.

[0120] There is also herein described a method as shown in Figure 9 for automated characterization of a porous material sample, comprising the steps of illuminating SA1 the porous material sample by at least one light source 110, capturing SA2 one or more images of the porous material sample by an optical sensor 120, and performing SA3 segmentation of the one or more images into areas corresponding to cell interiors and areas corresponding to cell walls by a software module 131.

[0121] The method may also comprise characterizing SA4 the porous material sample 200 in terms of its cell structure geometry based on the segmented image by the software module 131 , as well as generating SA0 training data and configuring a machine learning algorithm comprised in the software module 131 based on the training data as described above. Generating SA0 training data may comprise generating SA01 a segmented image from a mesh and rendering SA02 a 3D model corresponding the mesh.

[0122] Furthermore, the method may comprise any of:

[0123] Illuminating SA11 a surface of the porous material sample from an angle of between 25-65 degrees, and preferably about 45 degrees, measured relative to a normal of the surface.

[0124] Illuminating SA12 the porous material sample by two or more light sources 110 from different angles, measured relative to the normal of the surface.

[0125] Illuminating SA13 the porous material sample by a light source arranged to emit any of white light, red light, and blue light, or a combination thereof. Illuminating SA the surface of the porous material sample with two or more light sources in sequence, and capturing one or more images during the time that each light source illuminates the surface of the porous material sample 200.

[0126] Capturing the image of the sample using any of a camera, a stereo camera, a line-scan camera, and a LIDAR.

[0127] Capturing SA21 the image of the sample by a camera having a resolution of at least 20 pm per pixel in the captured image.

[0128] Arranging a neutral density filter and / or polarizing filter between the porous material sample and the optical sensor and light source.

[0129] Arranging SA22 the porous material sample on a transparent shelf 140, where the at least one light source 110 and the vision-based sensor 120 are arranged on an opposite side of the transparent shelf 140 compared to the porous material sample. The method may also comprise integrating a neutral density filter and / or a polarizing filter in the transparent shelf.

[0130] Arranging SA23 the optical I vision-based sensor 120 mounted on a mounting means 400 arranged to suspend the optical sensor above the surface of the porous material sample. Optionally, arranging the at least one light source 110 mounted on the same mounting means 400.

[0131] Arranging SA24 the optical sensor to capture two or more images, and arranging a software module 131 and / or control unit 130 to combine at least some of the two or more images into a combined image before performing segmentation. Obtaining the combined image may comprise performing an averaging operation on two or more images.

[0132] Performing SA31 segmentation of the one or more obtained images using a machine learning algorithm, where the machine learning algorithm may comprise a neural network. Adjusting SA32 the image captured by the vision-based sensor based on one or more pre-determined sensor calibration parameters. In this case the predetermined sensor calibration parameters may comprise any of size and scaling aspects, brightness, and contrast over the captured image.

[0133] Characterizing SA41 the porous material sample in terms of cell distribution, cell to material ratio, coalescence, and / or cell size distribution.

[0134] Comparing SA42 at least one parameter of the determined cell structure to one or more ranges of allowable parameters and outputting the result of the comparison as a verification test result. In this case the one or more ranges of allowable parameters can be selected by the control unit in dependence of a type of porous material sample.

[0135] Displaying SA43 a cell segmentation image 510, 520 of the characterized porous material sample on a display unit of the instrument 100.

[0136] Characterizing SA44 the porous material sample by using a machine learning structure.

[0137] Figure 7 schematically illustrates, in terms of a number of functional units, the components of a control unit 130, 600 according to embodiments of the discussions herein. Processing circuitry 610 is provided using any combination of one or more of a suitable central processing unit CPU, graphics processing unit GPU, tensor processing unit TPU, multiprocessor, microcontroller, digital signal processor DSP, etc., capable of executing software instructions stored in a computer program product, e.g., in the form of a storage medium 630. The processing circuitry 610 may further be provided as at least one application specific integrated circuit ASIC, or field programmable gate array FPGA. Particularly, the processing circuitry 610 is configured to cause the control unit 130, 600 to perform a set of operations, or steps, such as the methods discussed in connection to Figures 6 and 9, and generally herein. For example, the storage medium 630 may store the set of operations, and the processing circuitry 610 may be configured to retrieve the set of operations from the storage medium 630 to cause the control unit 130, 600 to perform the set of operations. The set of operations may be provided as a set of executable instructions. Thus, the processing circuitry 610 is thereby arranged to execute methods as herein disclosed.

[0138] The storage medium 630 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory.

[0139] The control unit 130, 600 may further comprise an interface 620 for communications with at least one external device. As such the interface 620 may comprise one or more transmitters and receivers, comprising analogue and digital components and a suitable number of ports for wireline or wireless communication.

[0140] The processing circuitry 610 controls the general operation of the control unit 130, 600, e.g., by sending data and control signals to the interface 620 and the storage medium 630, by receiving data and reports from the interface 620, and by retrieving data and instructions from the storage medium 630. Other components, as well as the related functionality, of the control node are omitted in order not to obscure the concepts presented herein.

[0141] Figure 8 illustrates a computer readable medium 710 carrying a computer program comprising program code means 720 for performing the methods illustrated in Figures 6 and 9 and the techniques discussed herein, when said program product is run on a computer. The computer readable medium and the code means may together form a computer program product 700.

Claims

CLAIMS1. A system (100) for automated characterization of a porous material sample (200), the system (100) comprising at least one light source (110), an optical sensor (120), and a software module (131 ), where the at least one light source (110) is arranged to illuminate the porous material sample (200), where the optical sensor (120) is arranged to capture one or more images of the illuminated porous material sample (200), and where the software module (131 ) is arranged to, when executed by a control unit, perform segmentation of the one or more images into areas corresponding to cell interiors and areas corresponding to cell walls.

2. The system (100) according to claim 1 , wherein the software module (131 ) is arranged to perform segmentation by means of a machine learning algorithm, where the machine learning algorithm has been trained using images of porous structures and corresponding cell segmentations.

3. The system (100) according to claim 2, where the machine learning algorithm is a neural network.

4. The system (100) according to any previous claim, wherein the software module (131 ) is further arranged to characterize the porous material sample (200) in terms of its cell structure geometry based on the segmented image.

5. The system (100) according to claim 4, where the software module (131 ) is arranged to characterize the porous material sample in terms of cell distribution, cell to material ratio, coalescence, and / or cell size distribution.

6. The system (100) according to claim 4 or 5, where the software module (131 ) is arranged to compare at least one parameter of the determined cell structure to one or more ranges of allowable parameters, and to output the result of the comparison as a verification test result.

7. The instrument (100) according to claim 6, where the one or more ranges of allowable parameters is selected by the control unit in dependence of a type of porous material sample.

8. The system (100) according to any previous claim, wherein the optical sensor (120) is any of a camera, a stereo camera, a line-scan camera, and a LIDAR.

9. The system (100) according to claim 8, wherein the optical sensor (120) is a camera having a resolution of at least 20 pm, micrometer, per pixel in the captured image.

10. The system (100) according to any previous claim, where the software module (131 ) is arranged to adjust the image captured by the vision-based sensor based on one or more pre-determined sensor calibration parameters.

11. The system (100) according to claim 10, where the pre-determined sensor calibration parameters comprise any of; size and scaling aspects, brightness, and contrast over the captured image.

12. The system (100) according to any previous claim, wherein the optical sensor (120) is arranged to capture two or more images, and the software module (131 ) is arranged to combine at least some of the two or more images into a combined image and perform segmentation on the combined image.

13. The system (100) according to claim 12, wherein the software module (131 ) is arranged to combine at least some of the two or more images by performing an averaging operation on the images.

14. The system (100) according to any previous claim, comprising two or more light sources (110) arranged to illuminate the surface of the porous material sample (200) from different angles, measured relative to the normal of the surface.

15. The system (100) according to claim 14, wherein the two or more light sources (110) are arranged to illuminate the surface of the porous materialsample (200) in sequence, and the optical sensor (120) is arranged to capture one or more images during the time that each light source (110) illuminates the surface of the porous material sample (200).

16. The system according to any previous claim, wherein a neutral density filter and / or a polarizing filter is arranged such that the porous material sample (200) is on one side of the filter and the light source (110) and the optical sensor (120) are on the other side.

17. The system (100) according to any previous claim, comprising a transparent shelf (140) to hold the porous material sample, where the at least one light source (110) and the optical sensor (120) are arranged on an opposite side of the transparent shelf (140) from the porous material sample.

18. The system (100) according to any of claims 1 to 16, the system comprising a mounting means (400) for mounting the optical sensor (120) and at least one light source (110), where the mounting means (400) is arranged to position the optical sensor (120) so that it faces a surface of the porous material sample (200).

19. The system (100) according to claim 18, comprising a linear actuator (410) arranged to move the mounting means (400) along the porous material sample (200).

20. The system (100) according to any of claims 1 to 17, wherein the optical sensor (120) is a vision-based sensor, and the system (100) comprises a control unit (130) and a main body (150), where the control unit (130) is arranged to characterize the porous material sample in terms of its cell structure geometry, based on the one or more captured images.

21. The system (100) according to claim 20, where the at least one light source (110) is arranged to illuminate a surface of the porous material samplefrom an angle of between 0-65 degrees, such as between 25-65 degrees, and preferably about 45 degrees, measured relative to a normal of the surface.

22. The system (100) according to any of claims 20 or 21 , where the at least one light source comprises a light source arranged to emit any of: white light, red light, and blue light.

23. The system (100) according to any of claims 20 to 22, where the control unit (130) is at least partly arranged separated from the main body (150) of the instrument (100).

24. The system (100) according to any of claims 20 to 23, comprising an access hatch (160) arranged in the main body (150), where the access hatch (150) is arranged to permit insertion of the porous material sample into an internal volume of the main body (150).

25. The system (100) according to claim 24, where the access hatch is a top-fed access hatch arranged on a top side of the system (100) in use.

26. The system (100) according to any previous claim, comprising a display unit (170) arranged to display a cell segmentation image (510, 520) of the characterized porous material sample.

27. A method for automated characterization of a porous material sample such as a cellular polymer sample, the method comprising illuminating (SA1 ) the porous material sample by at least one light source (110), capturing (SA2) one or more images of the porous material sample by an optical sensor (120), and performing (SA3) segmentation of the one or more images into areas corresponding to cell interiors and areas corresponding to cell walls by a software module (131 ),28. The method according to claim 27, comprising characterizing (SA4) the porous material sample (200) in terms of its cell structure geometry based on the segmented image by the software module (131 ).

29. A computer program product (700) comprising program code for performing, when executed by a control unit (130), the method of claims 27 or28.

30. A non-transitory computer-readable storage medium (710) comprising instructions, which when executed by a control unit (130) comprising processing circuitry, cause the processing circuitry to perform the method claim 27 or 28.

Images