Model-based spring shape measurement

Abstract

Images of the wire spring (10) are acquired from multiple cameras (150) mounted at different positions. Then, a parametric 3D shape model is fitted to the acquired images. The parametric 3D shape model is defined by a plurality of reference points which, by said fitting, are aligned with a course of the wire spring (10) in the images.

Description

[0001] Model-based spring shape measurement

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to methods and systems for measuring wire springs.

[0004] BACKGROUND OF THE INVENTION

[0005] Wire springs are used in various kinds of products, for example in mattresses or other kinds of bedding or seating products. Depending on the product, the wire springs may be produced with various characteristics and spring geometries, e.g., as helical coil springs with cylindrical, hourglass-like, or barrel like outer contour. The desired geometry is typically achieved by winding a metallic wire according to a correspondingly defined winding process, which may be performed by an automated or semi-automated spring winding machine.

[0006] During production of wire springs, it may however occur that the produced spring deviates from its desired characteristics. This may for example be due to the wire spring deviating from its intended spring geometry and / or due to imperfect design of the spring geometry.

[0007] WO 2022 / 017785 A1 describes a method of producing coil springs by spring winding, in which images of the coil spring are acquired during the winding process and are used for adapting a parametric three-dimensional (3D) helix model of the coil spring, and the 3D helix model is then used as a basis for obtaining measurement values for describing the geometry of the coil spring, such as pitch, diameter or extended length. Such way of modelling a wire spring may however not be applicable to certain types of wire springs, e.g., having a more complex geometry or exhibiting severe faults causing significant deviation from the intended helical spring geometry. Further, in some scenarios measurement of wire springs may also need to be done independently of the manufacturing process, e.g., at a site where finished wire springs are assembled to a product.

[0008] Accordingly, there is a need for techniques which allow for flexibly and accurately measuring the geometry of wire springs. BRIEF SUMMARY OF THE INVENTION

[0009] The present invention provides a machine according to claim 1 and to a method according to claim 10. The dependent claims define further embodiments.

[0010] According to an embodiment, the present invention thus provides a method of measuring shape of a wire spring. According to the method, images of the wire spring are acquired from multiple cameras mounted at different positions. Then, a parametric 3D shape model is fitted to the acquired images. The parametric 3D shape model is defined by a plurality of reference points which, by said fitting, are aligned with a course of the wire spring in the images.

[0011] By using multiple cameras, e.g., two, three, or even more, which are mounted at different positions, the images depict the wire spring from different perspectives, thereby allowing for an accurate assessment of the spring shape. Further, it becomes possible to capture the images without specific restriction on positioning of the spring while the images are captured. Multiple images from different perspectives can be captured simultaneously, which allows for doing the measurement at a specific time instance, or even in a time resolved manner. Still further, the measurement can be done while the wire spring is at rest. The parametric 3D shape model is defined by a plurality of reference points allows for efficiently and accurately modelling the wire spring, without excessive restrictions on the nominal geometry of the wire spring to be modelled. For example, also non-helical wire springs or wire springs exhibiting a significant deviation from an ideal helix geometry can be modelled in an efficient and accurate manner.

[0012] According to an embodiment each reference point defines a reference sphere which, by said fitting, is aligned with the course of the wire spring in the images. This allows for a highly efficient process of aligning the reference points with the course of the wire spring in the images, because the projection of the reference sphere to the two- dimensional image is remains circular, irrespective of the perspective of the image. As result, processing requirements for performing the alignment can be alleviated. It is however noted that, also depending on the cross-sectional geometry of wire used for forming the wire spring, other kinds of geometric bodies could be used instead of speres, e.g., reference ellipsoids, reference cubes, or the like.

[0013] As mentioned above, the number of the cameras can be three or more. However, in some scenarios usage of only two cameras can also provide satisfactory results. In other scenarios, further improvement of modelling accuracy can be achieved by using four or even more cameras.

[0014] According to an embodiment, the images are captured while the wire spring is at rest. In this way it can be avoided that the measurement is affected by changes of the spring shape which are caused by movement of the spring while capturing the images. Alternatively, one or more of the images could also be captured while the spring is moving. Further, it would be possible to move the spring between capturing two of the images.

[0015] The number of the reference points is 50 or more, 100 or more, or 200 or more. In tests on typical wire springs utilized in commercial bedding or seating products, numbers in the range of 100-200 were found to provide a reasonable balance of processing effort and modelling accuracy.

[0016] The parametric 3D shape model may be based on a three dimensional spline curve with knots corresponding to the reference points. The fitting may be based on minimizing a deviation of parametric 3D shape model from the course of the wire spring in the images. I some embodiments, the fitting may be performed in an iterative manner, based on a gradient of the deviation between the positions of the reference points and wire spring in the images.

[0017] In some embodiments, the fitting comprises fitting multiple intermediate parametric 3D shape models to the images. Then, the parametric 3D shape model may be generated by combining the intermediate parametric 3D shape models, e.g., by selection, averaging, and / or concatenating. In some cases, this may involve that each of the intermediate parametric 3D shape models is fitted to different subsets of the images. Alternatively or in addition, for each of the intermediate parametric 3D shape models, the fitting may be based on a different set of starting conditions. In this way, the overall fitting process may flexibly consider different kinds of assumptions and conditions.

[0018] In some embodiments, a number of parameters of the parametric 3D shape model may be adapted during said fitting. For example, the fitting could start with a relatively simple version of the parametric 3D shape model, with a low number of parameters, and the number of parameters could then be increased, e.g., when convergence of the fitting process has reached a given degree with the lower number of parameters. The parameters of the simple version of the parametric 3D shape model can then be used as starting values for fitting a refined version of the parametric 3D shape model, with a higher number of parameters. As example of such adaptive parameters, the number of the reference points could be adapted.

[0019] In some scenarios, the wire spring may be regarded as having a central portion and end portions, and such central portion and end portions may also differ with respect to their geometry. For example, the end portions could have a smaller or larger outer diameter than the central portion. In some embodiments, the fitting may comprise first fitting the parametric 3D shape model to the central portion and then expanding the fitting of the parametric 3D shape model to the end portions. In this way, it can be utilized than in many cases the geometry of the central portion of the wire spring is more regular, so that quicker convergence of the fitting process can be expected. Then expanding the fitting process to the end portions may allow for using the at least partially converged fit in the central portion as a starting point for the fit in the end portions.

[0020] The fitted parametric 3D shape model can be used in various ways. For example, it can be used for assessing quality of wire springs produced by a spring winding machine. This may be done on-site of the spring winding machine, e.g., as part of manufacturing quality control, or at some other site than the spring winding machine, e.g., at the site of assembling wire springs provided by a third party to a product, e.g., a mattress or some other bedding or seating product. In such cases, the measurement method could be implemented independently of the spring winding machine. In some embodiments, the fitted parametric 3D shape model could also be used as input for controlling a spring winding machine, e.g., for setting or re-adjusting spring winding parameters.

[0021] According to a further embodiment, an apparatus for measuring shape of a wire spring is provided. The apparatus comprises processing circuity configured to acquire images of the wire spring from multiple cameras mounted at different positions. Further, the processing circuitry is configured to fit a parametric 3D shape model to the acquired images. The parametric 3D shape model is defined by a plurality of reference points which, by said fitting, are aligned with the features of the wire spring in the images. The apparatus may be configured to perform the above method of measuring a wire spring. In some embodiments, the apparatus may also comprise the cameras from which the images are acquired. According to a further embodiment, a computer program or computer program product is provided, e.g., in the form of a non-transitory storage medium. The computer program or computer program product comprises program code which, when executed by a computer, causes the computer to perform the above-mentioned method.

[0022] BRIEF DESCRIPTION OF DRAWINGS

[0023] Embodiments of the invention will be described with reference to the accompanying drawings.

[0024] Figs. 1A and 1 B schematically illustrate a measurement system according to an embodiment.

[0025] Figs. 2A, 2B, and 2C schematically illustrate fitting of a parametric 3D shape model according to an embodiment.

[0026] Fig. 3 schematically illustrates a model fitting process according to an embodiment.

[0027] Fig. 4 schematically illustrates combination of intermediate models according to an embodiment.

[0028] Fig. 5 shows a flowchart for illustrating a method according to an embodiment of the invention.

[0029] DETAILED DESCRIPTION OF EMBODIMENTS

[0030] Exemplary embodiments of the concepts illustrated herein will now be described with reference to the drawings. In particular, the following detailed description will describe such embodiments by referring to an exemplary measurement system 100 and measurement methods which may be performed by using the measurement system 100. In the illustrated examples, it is assumed that the measurement system is used for measuring a wire spring 10 with generally helical geometry. It is however noted that the illustrated concepts could also be applied to various other spring geometries, in particular to various kinds of spring geometry which can be obtained by bending or otherwise deforming a wire or other type of elongated body. Fig. 1A schematically illustrates components of the measurement system 100. Fig. 1 B schematically illustrates a functional architecture of the measurement system 100. As illustrated, the measurement system 100 may include a measurement table 110 on which the wire spring 10 to be measured is placed. Further, a number of cameras 150 is provided for capturing images of a measurement space extending above the measurement table 110. The cameras 150 are positioned and oriented to capture the images from different perspectives. For this purpose, the measurement system 100 could for example include a rack for mounting the cameras 150 and / or the table 110. In some examples, the positions and / or orientation of the cameras 150 can be adjustable, e.g., by manual adjustment and / or by electronically controlled adjustment. In some examples, the measurement table 100 may include optical markers, which may for example assist in evaluation of the images captured by the cameras 150 and / or be used for calibration of the cameras 150. The cameras 150 may correspond to commercially available industrial cameras.

[0031] A further illustrated in Fig. 1 B, the measurement system includes a processing and control subsystem 160. The cameras 150 are coupled to the processing and control subsystem 160 to enable transfer of the images captured by the cameras 150 to the processing and control subsystem 160, e.g., in the form of image data files which each correspond to an individual image and optionally metadata associated with the image, such as time stamp, camera identifier, or the like. In some cases, the cameras 150 could however also provide raw image data, e.g., pixel data, to the processing and control subsystem 160, and the processing and control subsystem 160 could then generate the images from the raw image data and optionally also metadata associated with the image, such as time stamp, camera identifier, or the like. The processing and control subsystem 160 may also control the cameras 150, e.g., by providing control data to the cameras 150. Such control may for example involve triggering of capturing the images by the cameras 150 and / or setting of one or more image acquisition parameters, such as exposure time, aperture, white balance, optical zoom, digital zoom, or the like. Further, such control could also include electronic adjustment of position and / or orientation of the camera 150. The processing and control subsystem 160 may also include a user interface through which an operator can interact with the measurement system 100 and control various operations as described herein. Such user interface could for example be based on a graphical user interface (GUI), and user inputs from various kinds of human interface device (HID), such as computer mouse, touchscreen, keyboard, graphics tablet, stylus pen, or the like. The processing and control subsystem 160 may be implemented by suitably programmed general-purpose computer hardware, by dedicated processing hardware, e.g., based on one or more ASICs (applicationspecific integrated circuits), and / or a combination of general-purpose computer hardware and dedicated processing hardware.

[0032] As further illustrated, the measurement system 100 may further include an illumination subsystem 170. The illumination subsystem 170 may include one or more light sources for illumination of the wire spring 10 when capturing the images. In this way, light conditions for capturing the images may be controlled and stabilized as compared to using environment light. For example, one or more of such light sources could be mounted on the same rack which is also used for mounting the cameras 150 and / or the table 110. In some scenarios, the illumination subsystem 170 could be configured for capturing the images with backlight illumination, e.g., with the light source(s) and the cameras 150 arranged on opposing sides of the measurement space. In other scenarios, the illumination subsystem 170 could be configured for capturing the images with frontlight illumination, e.g., with the light source(s) and the cameras 150 arranged on the same side of the measurement space. In some cases, the illumination subsystem 170 could also be re-configurable to change between backlight illumination and frontlight illumination, e.g., by modifying location of one or more light sources and / or by switching between different light sources. Such switching between different light sources could also be based on control data provided by the processing and control subsystem 160. Usage of backlight illumination may help to improve quality of the images captured by the cameras 150, e.g., by reducing reflections and / or image contrast. Usage of frontlight illumination may in turn allow for a simplified hardware setup, e.g., if the available space is not sufficient to mount the light source(s) on the opposing side of the cameras 150. The processing and control subsystem 160 may also control the illumination subsystem 170, e.g., by providing control data to the illumination subsystem 170. Such control may for example involve activation or deactivation of the light source(s), controlling light intensity, or switching between different light sources. Further, such control could also include electronic adjustment of position and / or orientation of the light source(s) and / or control of change between one or more of backlight illumination, frontlight illumination, environmental illumination.

[0033] As further illustrated, the measurement system 100 may further include an actuator subsystem 180. The actuator subsystem 180 may include one or more actuators, e.g., for adjustment of position or orientation of the cameras 150. Further, if the measurement system 100 includes the above-mentioned table 110, the actuator subsystem 180 could include one or more actuators for adjustment of position or orientation of the table 110. Further, if the measurement system 100 includes the above-mentioned illumination subsystem, the actuator subsystem 180 could include one or more actuators for adjustment of position or orientation of the light source(s) of the illumination subsystem 170. Further, actuator subsystem 180 could include one or more actuators of a robotic system, e.g., for automatically placing the wire spring 10 on the table 110 or for feeding the wire spring 10 to the measurement system 100.

[0034] As further illustrated, the processing and control subsystem 160 may also be provided with an interface to a spring winding machine 200. In some cases, the measurement system 100 could be co-located or integrated with the spring winding machine 200. In such cases, the interface to the spring winding machine could be used for providing evaluation results to the spring winding machine, e.g., for adjusting spring winding parameters or other configurations of the spring winding machine. Further, the interface could be used for providing information on the wire spring 10 to be measured to the measurement system 100. Such information may for example include information on the type of the wire spring 10 or information on a nominal spring geometry of the wire spring 10. The measurement system 100 may then utilize such information for enhancement of the measurement process, e.g., be selecting corresponding input conditions for the evaluation.

[0035] In the illustrated concepts, the measurement system 100 measures the wire spring 10 by fitting a parametric 3D shape model to the images provided by the cameras. The related data processing may be performed by the processing and control subsystem 160. The parametric 3D shape model, in the following also briefly termed as “model”, is based on a plurality of reference points which, in the fitting process, are aligned with the course of the wire spring 10 in the images provided by the cameras 150. Figs. 2A, 2B, and 2C schematically illustrates an example of such 3D parametric shape model 20. In the illustrated example, the modelled wire spring is assumed to have generally helical geometry, e.g., like the above-mentioned wire spring 10. A result of the fitting process is shown in Fig. 2A, which illustrates a schematic perspective view of the parametric 3D shape model 20. The reference points are illustrated by solid dots. In this example, the reference points correspond to knots of a three-dimensional spline curve which approximates the course of the wire spring as observed in the images provided by the cameras 150. Parameters of the parametric 3D shape model 20 may for example include, for each reference point: 3D coordinates of the reference point, spline curve orientation at the reference point, and / or spline curve curvature at the reference point. Figs. 2B and 2C further illustrate the fitting process. In the illustrated example, it is assumed that each reference point of the model defines a reference sphere 21 , with the 3D coordinates of the reference point corresponding to the center of the reference sphere 21 . When projected to a 2D image of the wire spring, the reference sphere defines a corresponding circle, as shown in Figs. 2B and 2C. In Figs. 2B and 2C, the course of the wire spring according to the images from the cameras 150 is illustrated by dotted lines corresponding to the boundaries of the wire spring. In Fig. 2B, it is assumed that convergence of the fitting process is not yet complete, so that the circle defined by the reference sphere 21 deviates from the course of the wire spring. Specifically, the circle defined by the reference sphere 21 is at least partially outside the boundaries of the wire spring. In Fig. 2C, it is assumed that the fitting process has converged, so that the circle defined by the reference sphere 21 no longer deviates from the course of the wire spring. Specifically, the circle defined by the reference sphere 21 is inside the boundaries of the wire spring.

[0036] As can further be seen from Figs. 2B and 2C, the diameter of the reference sphere 21 is preferably set to correspond to the diameter of the wire forming the wire spring. Here, it is however noted that calibration of the cameras 150 and / or calibration processing of the images provided by the cameras 150 may be needed to ensure that the 2D projection of the reference sphere 21 is appropriately sized in the images.

[0037] Fig. 3 shows a diagram for further illustrating an exemplary implementation of the fitting process, e.g., as used for generating the model 20 of Figs. 2A, 2B, and 2C.

[0038] As illustrated, the fitting process is based on a set of model parameters. The model parameters may for example include: global orientation of the wire spring, global position of the wire spring, number of reference points and, for each of the reference points, 3D coordinates of the reference point, spline curve orientation at the reference point, and / or spline curve curvature at the reference point. One or more further model parameters could define a degree of freedom for variation of one or more other model parameters. For example, the variation of the coordinates of the reference points from their starting value could be limited to a selectable tolerance range.

[0039] Each reference point may correspond to a reference sphere, and the diameter of the reference sphere may be selected to correspond to the diameter of the wire of the wire spring to be modelled. Starting values of the model parameters be predefined and / or may be selected depending on the knowledge of the wire spring to be modelled. For example, a certain set of starting values could be predefined for use in modelling a wire spring with substantially cylindrical helix geometry, and another set of starting values could be predefined for use in modelling a wire spring with barrelshaped helix geometry. The appropriate set of starting values could then be selected depending on the knowledge of the wire spring to be modelled. In a similar way, starting values of the model parameters could be selected depending on other known characteristics of the wire spring to be modelled, e.g., size or number of windings. For example, a higher number of reference points could be selected for larger-sized wire springs, and a lower number of reference points could be selected for smaller- sized wire springs. Further, a higher number of reference points could be selected for wire springs with a high number of windings, and a lower number of reference points could be for wire springs with a low number of windings.

[0040] At block 320, a subset of the model parameters 310 may be selected to be used in the fitting process or in a given stage of the fitting process. For example, the selection of block 320 could involve selecting only some of the reference points and / or selecting only the 3D coordinates of the references points to be optimized in the fitting process or in the present stage of the fitting process. As a result, a parameter subset 330 is obtained. It is however noted that in some cases also all model parameters could be selected at block 320 and the fitting process then be based on the complete set of model parameters.

[0041] As mentioned above, images 340 provided by the cameras 150 are used as input to the fitting process. The images 340 capture the wire spring to be modelled from different perspectives. In this way, it can be avoided that parts of the wire spring are not or only poorly visible in the images or that the wire spring needs to be moved in order to capture all parts of the wire spring. The fitting process can thus be based on images which are captured while the wire spring is at rest, so that deformation of the wire spring due to such movement can be avoided. Further, it becomes to utilize images which are simultaneously captured and cover the wire spring from different perspectives. This may also enable time resolved measurements by performing the fitting based on images simultaneously captured at a given time instance.

[0042] At block 350, the images may be subjected to preprocessing. Such preprocessing may for example involve scaling and / or distortion to ensure that features on images captured from different perspectives, i.e. , by different cameras 150, are represented in a comparable manner. Such scaling may for example be based on calibration of the cameras 150. Further, the scaling and / or distortion may ensure consistent translation between features of the images and features of the model.

[0043] Further, in the illustrated examples it is assumed that the preprocessing involves thresholding and / or filtering to sharpen contours of the wire spring in the image and to separate pixels corresponding to the wire spring from background pixels, e.g., by binarization. In such binarized image, a pixel corresponding to the course of the wire spring may have value “1”, while pixels corresponding to the background may have a value “0”. In the illustrated example, such binarized image is then be subjected to a distance transform, e.g., by replacing values of pixels corresponding to the background with a value representing the distance to the nearest pixel corresponding to the course of the wire spring. The outcome of such distance transformed images typically has pixels with a maximum value over the course of the wire spring, with a falloff from this maximum value to zero in an adjacent region.

[0044] At block 360, deviation of the model from the course of the wire spring in the images is estimated. For this purpose, the reference points, in particular the above- mentioned reference spheres may be projected into the preprocessed images from block 350. Based on such projection, a congruence metric can be calculated, which represents the deviation of the reference points from the course of the wire spring. For example, by for each pixel corresponding to the projected reference spheres it can be determined whether this pixel matches a pixel which, in the image, represents the wire spring, e.g., by indexing. For example, if the pixel of the reference sphere matches a pixel above a threshold in the distance transformed image, the indexing could yield a value of “1”, otherwise a value of “0”. As a result, a value can be obtained which represents the degree of overlap of the projection of the reference sphere with the course of the wire spring. Fig. 2B illustrates an example with only partial overlap, and Fig. 2C illustrates an example with full overlap. The congruence metric could then be obtained by averaging over the reference points and by averaging over the multiple images.

[0045] Further, image gradients of the distance transformed images may be calculated. The image gradients may be subjected to indexing or similar pixel matching as explained above. That is to say, if the pixel of the reference sphere matches a pixel above a threshold in the gradient of the distance transformed image, the indexing could yield a value of “1”, otherwise a value of “0”. Here, the outcome corresponds to a gradient metric representing how far the projection of the reference sphere is from the course of the wire spring, the value of the gradient metric decreasing with the distance from the course of the wire spring. Such gradient metrics may be used as further input in iterative optimization at block 370. In some cases, the gradient metrics may be calculated in a directional manner, representing for a certain direction how far the projection of the reference sphere is from the course of the wire spring. For example, two independent directions per 2D image could be considered in this way.

[0046] At block 370, an optimization process is applied, with the aim of minimizing the deviation of deviation of the model from the course of the wire spring. In the illustrated example, it is assumed that the optimization process aims at minimizing the inverse of the congruence metric. For this purpose, the current values of the model parameters are adapted. In particular, for each of the reference points, the coordinates are adapted depending on the gradient metric(s). For larger gradient, the amount of adaptation is reduced. In a similar manner, the global orientation and / or global coordinates of the model may be adapted, by 3D translation and / or 3D rotation of the model. Here, certain model parameters may also act as constraint for other model parameters. For example, spline orientation and / or spline curvature of a certain reference point could limit adaptation of the coordinates of this reference point and / or of neighboring reference points. As a result of the optimization process, the model parameters selected at block 320 are updated, thus providing an updated parametric 3D shape model 380. The updated parametric 3D shape model 380 may then be used as a basis for newly calculating the projections of the reference spheres in a next iteration, and the operations explained in connection with blocks 360 and 370 may be repeated with each iteration. It is however noted that calculation of the image gradients is needed only in the first iteration.

[0047] After executing the fitting process for a subset of the model parameters, selected at block 320, the fitting process may be repeated for another subset of the model parameters. Further, such other model parameters could be estimated by interpolation or extrapolation.

[0048] It is noted that the fitting process in the example of Fig. 3 could be modified in various ways. For example, the degree and way of preprocessing at block 350 may vary depending on the characteristics of the images 340 used as input. For example, in some cases the cameras could provide the images 340 already in a binarized format, so that the preprocessing can be simplified to significant extent. Further, different types of distance transformation could be utilized. In some scenarios, the fitting process could also be assisted by machine learning (ML). For example, the starting values of the model parameters could be determined, at least in part, based on a supervised ML algorithm. Further, parameters of the optimization process in block 370 could be adjusted based on an ML algorithm, e.g., using enforced learning to optimize convergence.

[0049] In some implementations, the determination of the parametric 3D shape model may also be based on first determining a set of intermediate 3D shape models, herein also briefly denoted as “intermediate model”.

[0050] Fig. 4 shows a diagram which illustrates an example of utilizing such intermediate models. As illustrated, in this example a plurality of fitting processes 400 is provided. These fitting processes 400 may be executed in parallel, but non-parallel execution would be possible as well. Each of the fitting processes 400 is based on a corresponding set of starting conditions 410, which are used as input to model fitting 420, e.g., as explained in connection with Fig. 3. The fitting processes 400 may use different starting conditions 410 and / or be based on different subsets of the images provided by the cameras 150. For example, such subset of images could correspond to images captured by a subset of the cameras or to images captured with backlight illumination and images captured with frontlight illumination. Further, such subsets of images could differ with respect to image acquisition parameters.

[0051] As a result, each fitting process 400 provides a corresponding intermediate model 430. At block 440, the intermediate models 430 are combined to obtain a combined model 450. Such combination may involve selection among the intermediate models 430 and / or averaging of at least some of the intermediate models 430 and / or concatenating at least some of the intermediate models 430. In some cases, the intermediate models 430 could correspond to different portions of the wire spring to be modelled, e.g., a center portion and end portions, and the combination could involve concatenating the intermediate models 430 so that the combined model 450 models all portions of the wire spring. The combining may be based on quality level of the intermediate models 430, e.g., in terms of level of the congruence metric achieved in the fitting process.

[0052] In an example, each fitting process 400 could use different starting conditions 410. Then, the intermediate models 430 are obtained by individually maximizing the level of the congruence metric or minimizing the deviation, by iteratively performing the fitting process of Fig. 3. The iterations can be stopped after a given number of iterations, e.g., 100 iterations. Then one or more of the intermediate models 430 are selected based on the level of the congruence metric. For example, intermediate models 430 with congruence metric above a threshold or intermediate models 430 with the highest level of the congruence metric could be selected. If multiple intermediate models 430 are selected, these may then be combined by averaging.

[0053] In some implementations, the determination of the parametric 3D shape model may be based on adapting the number of the model parameters in the course of the fitting process, e.g., in the fitting process of Fig. 3. This may also be referred to as “adaptive resolution” of the fitting process. For example, at the start of the fitting process, a low number of the model parameters could be used or subjected to updating in block 370. Subsequently, the number of the model parameters could be increased, e.g., after a certain number of iterations or if it is found that improvements the level of the congruence metric between iterations is below a threshold. With such adaptive number of model parameters, the number of degrees of freedom of the model can be increased gradually to achieve a high-resolution model while at the same time keeping reasonable speed of the fitting process.

[0054] Further, in some implementations the fitting process could comprise different stages relating to different portions of the wire spring to be modelled. In initial stage, a coarse global parametric 3D shape model of the entire wire spring could be determined, e.g., using a first low number of model parameters and the fitting process of Fig. 3. Then, in a next stage, the global parametric 3D shape model can be locally refined, starting from a center portion of the wire spring and iteratively expanding to the ends of the wire spring. Here, model parameters in the form of additional reference points at the edges of the currently modelled center portion, and the values of the model parameters for the previous iteration used as starting values for the expanded model. The iterative expansion may be terminated if the expanded model reaches the end points of the wire spring.

[0055] As mentioned above, calibration of the cameras 150 or of the process of acquiring the images by the cameras 150 may be beneficial in view of accuracy and / or speed of the fitting process. It is however noted that, in some implementations, the fitting process could also be implemented in an at least partly self-calibrating way, by including one or more calibration parameters into the model parameters subject to optimization. In each case, the calibration or self-calibration may enable usage of less sophisticated cameras, e.g., cameras based on rather simple wide-angle lenses, as compared to cameras based on hardware calibrated optical elements. The calibration process may be based on known camera calibration algorithms allowing estimation of a camera matrix for transformation between object coordinates and image coordinates. Such calibration process may be based on markers provided in the measurement space, e.g., on the table 110.

[0056] The resulting parametric 3D shape model of the wire spring may then be further evaluated in view of quality of the spring, e.g., in terms of deviation from a nominal shape or geometry, and / or in view of particular characteristics, like diameter, length, in particular length without load on wire spring, curvatures, overall diameter, diameter(s) at the end of the wire spring, diameter at the center of the wire spring, or the like.

[0057] Fig. 5 shows a flowchart for illustrating a method of measuring a wire spring, such as the above-mentioned wire spring 10. The method of Fig. 5 may be performed by or using the above-described measurement system 100. In some scenarios, the method may be implemented by execution of program code on a computer-based apparatus. For example, the program code could be executed by the above-mentioned processing and control subsystem 160. The program code could be provided on a storage medium or by download or streaming.

[0058] At block 510, one or more cameras are calibrated. This may involve determining at least a part of a camera matrix of the camera. For example, operations of block 510 may involve calibration of the above-mentioned cameras 150.

[0059] At block 520, images of a wire spring to be measured are acquired from multiple cameras at different positions. These cameras may correspond to or include one or more of the cameras calibrated at block 510. Acquiring the images may involve receiving the images or image data corresponding to the images from the cameras. Further, this may involve triggering and / or otherwise controlling capturing of the images, e.g., with respect to image acquisition parameters and / or illumination. The images may be captured with backlight illumination or with frontlight illumination. It is also conceivable to utilize both images may be captured with backlight illumination and images captured with frontlight illumination. In some scenarios, the images may be captured while the wire spring is at rest. It is however also possible that one or more of the images are captured while the wire spring is moving. The number of the cameras may be three or more. But only two cameras or even more than three cameras could be used as well.

[0060] At block 530, a parametric 3D shape model is fitted to the acquired images. This may be accomplished based on a fitting process as explained in connection with Fig. 3. The above-mentioned models 20; 380; 430, 450 are examples of such parametric 3D shape model. The parametric 3D shape model is defined by a plurality of reference points which. By the fitting, the reference points are aligned with a course of the wire spring in the images, e.g., as explained in connection with Figs. 2B and 2C. Each reference point may define a reference sphere which, by the fitting process, is aligned with the course of the wire spring in the images. The number of the reference points can be 50 or more, 100 or more, or 200 or more. The parametric 3D shape model can be based on a 3D spline curve with knots corresponding to the reference points. The 3D spline curve can be formed of linear spline segments or of higher- order spline segments.

[0061] The fitting at block 530 can be based on minimizing a deviation of the parametric 3D shape model from the course of the wire spring in the images, e.g., as explained in connection with Fig. 3. The fitting may be performed in an iterative manner, based on a gradient of the deviation between the positions of the reference points and wire spring in the images, e.g., based on the above-mentioned gradient metric, which in turn is based on image gradients of distance transforms of the images.

[0062] In some scenarios, the fitting process may involve fitting multiple intermediate parametric 3D shape models to the images and then generating the parametric 3D shape model by combining the intermediate parametric 3D shape models. Here, each of the intermediate parametric 3D shape models may be fitted to different subsets of the images and / or the fitting may be based on a different set of starting conditions for each of the intermediate parametric 3D shape models. Examples of such using such combination of intermediate parametric 3D shape models are described in connection with Fig. 4. When using such combination of intermediate parametric 3D shape models, it is thus possible to use an intermediate parametric 3D shape model per subset of images, an intermediate parametric 3D shape model per set of starting conditions, an intermediate parametric 3D shape model per portion of wire spring, and / or an intermediate parametric 3D shape model per camera.

[0063] In some scenarios, a number of parameters of the parametric 3D shape model may be adapted during the fitting process, e.g., starting from a low number of parameters and then increasing the number of parameters to improve accuracy of the parametric 3D shape model. For example, a number of the reference points could be adapted during the fitting process. In some scenarios, the wire spring may have, or be regarded as being formed of, a central portion and end portions. In such cases, the fitting process could involve first fitting the parametric 3D shape model to the central portion and then expanding the fitting of the parametric 3D shape model to the end portions, e.g., by iteratively adding further reference points at edges of the parametric 3D shape model fitted to the central portion.

[0064] At block 540, quality of the wire spring may be assessed based on the fitted parametric 3D shape model obtained at block 530. Such quality assessment may be part of a manufacturing process of wire springs by a spring winding machine, such as the above-mentioned spring winding machine 200. This may involve subjecting all or a selected sample of the wire springs produced by the spring winding machine to such quality assessment. In such case, the wire springs to be assessed could be automatically fed from the spring winding machine 200 to the measurement system 100. In other scenarios, the quality assessment could be performed independently of manufacturing the wire springs, e.g., on wire spring purchased from a vendor. In the latter case, the quality assessment could for example be performed at a site where the wire springs are assembled to a product, e.g., to a mattress or to some other kind of bedding or seating product.

[0065] At block 550, a spring winding machine for producing wire springs may be controlled based on the fitted parametric 3D shape model obtained at block 530, such as the above-mentioned spring winding machine 200. For example, spring winding parameters of the spring winding machine 200 could be set or adjusted based on the fitted parametric 3D shape model, e.g., with the aim of improving matching of the produced wire springs to a nominal spring shape or with the aim of tuning a characteristic of the produced wire springs.

[0066] It is to be understood that the illustrated methods, apparatuses and systems are susceptible to various modifications, without departing from the illustrated concepts. For example, the illustrated concepts can be applied to measurement and modelling of various kinds of wire springs, without limitation to helical spring geometries. Further, it is noted that the illustrated concepts could be applied in various kinds of deployments, e.g., in a standalone measurement system, in a measurement system integrated in a spring winding machine, in a measurement system co-located with a spring winding machine, or in a measurement system integrated or co-located with a machine for assembly of wire springs to a further product.

Claims

C LA I M S1. A method of measuring shape of a wire spring (10), the method comprising: from multiple cameras (150) mounted at different positions, acquiring images (340) of the wire spring (10); fitting a parametric three-dimensional, 3D, shape model (20; 380; 430, 450) to the acquired images (340), wherein the parametric 3D shape model (20; 380; 430, 450) is defined by a plurality of reference points which, by said fitting, are aligned with a course of the wire spring (10) in the images (340).

2. The method according to claim 1 , wherein each reference point defines a reference sphere (21 ) which, by said fitting, is aligned with the course of the wire spring (10) in the images (340).

3. The method according to claim 1 or 2, wherein the number of the cameras (150) is three or more.

4. The method according to any of the preceding claims, wherein the images (340) are captured while the wire spring (150) is at rest.

5. The method according to any of the preceding claims, wherein the number of the reference points is 50 or more, 100 or more, or 200 or more.

6. The method according to any of the preceding claims, wherein the parametric 3D shape model (20; 380; 430, 450) is based on a three dimensional spline curve with knots corresponding to the reference points.

7. The method according to any of the preceding claims, wherein said fitting is based on minimizing a deviation of parametric 3D shape model from the course of the wire spring (10) in the images (340).

8. The method according to claim 7, wherein said fitting is performed in an iterative manner, based on a gradient of the deviation between the positions of the reference points and wire spring (10) in the images (340).

9. The method according to any of the preceding claims, wherein said fitting comprises fitting multiple intermediate parametric 3D shape models (430) to the images (340); and generating the parametric 3D shape model (450) by combining the intermediate parametric 3D shape models.

10. The method according to claim 9, wherein each of the intermediate parametric 3D shape models (430) is fitted to different subsets of the images (340).11 . The method according to claim 9 or 10, wherein for each of the intermediate parametric 3D shape models (430) said fitting is based on a different set of starting conditions.

12. The method according to any of the preceding claims, wherein a number of parameters of the parametric 3D shape model (20; 380; 430, 450) is adapted during said fitting.

13. The method according to any one of the preceding claims, wherein a number of the reference points is adapted during said fitting.

14. The method according to any of the preceding claims, wherein the wire spring (10) comprises a central portion and end portions, and wherein said fitting comprises first fitting the parametric 3D shape model (20; 380; 430, 450) to the central portion and then expanding the fitting of the parametric 3D shape model (20; 380; 430, 450) to the end portions.

15. The method according to any of the preceding claims, comprising: based on the fitted parametric 3D shape model (20; 380; 430, 450), assessing quality of wire springs (10).

16. The method according to any of the preceding claims, comprising: based on the fitted parametric 3D shape model (20; 380; 430, 450), controlling a spring winding machine (200).

17. An apparatus (100) for measuring shape of a wire spring (10), the apparatus comprising processing circuity configured to:from multiple cameras (150) mounted at different positions, acquire images (340) of the wire spring (10); and fit a parametric three-dimensional, 3D, shape model (20; 380; 430, 450) to the acquired images (340), wherein the parametric 3D shape model is defined by a plurality of reference points which, by said fitting, are aligned with the features of the wire spring (10) in the images (340).

18. The apparatus (100) according to claim 17, wherein the apparatus (100) is configured to perform a method according to any one of claims 1 to 16.

19. The apparatus (100) according to claim 17 or 18, wherein the apparatus (100) further comprises the cameras (150).

20. A computer program or computer program product comprising program code which, when executed by a computer-based apparatus (100), causes the computer- based apparatus (100) to perform a method according to any of claims 1 to 16.

Images