Method of generating a three-dimensional model
The method of applying marker arrangements with fixed positions and filtering images based on geometric properties enhances the accuracy of three-dimensional models using a monocular camera, addressing the challenges of movement and inaccuracy in existing technologies.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- BOOLE LTD
- Filing Date
- 2026-01-23
- Publication Date
- 2026-07-30
AI Technical Summary
Creating accurate three-dimensional models using a monocular camera is challenging due to inherent inaccuracies and movement of the object during scanning, making it unsuitable for applications requiring high accuracy, such as anthropometric measurements or capturing moving subjects or devices.
A method involving the application of marker arrangements with fixed relative positions and orientations, where images are filtered based on geometric properties to generate a three-dimensional model, excluding distorted images and allowing for camera or object movement during scanning.
This approach significantly increases the accuracy of three-dimensional models by excluding distorted images, enabling high-accuracy modeling even with a monocular camera, particularly suitable for moving subjects or devices.
Smart Images
Figure EP2026051761_30072026_PF_FP_ABST
Abstract
Description
[0001] Method of generating a three-dimensional model
[0002] Field of the disclosure
[0003] The disclosure relates to the field of three-dimensional models. Specifically, this disclosure relates to the field of three-dimensional models based on image processing.
[0004] Background
[0005] It is known to create three-dimensional models of objects based on optical scans of the three-dimensional objects. For example, it is possible to create three-dimensional models based on optical scans by stereoscopic cameras, monocular cameras, structured light cameras, and Light Detection and Ranging (LiDAR) devices.
[0006] However, it is known to be difficult to create accurate models of three-dimensional objects when using a monocular camera.
[0007] For example, Luhmann et al 2023 textbook states ‘Highly accurate measurements cannot be expected when using only one camera, according to the configuration for space resection" (see Luhmann, T. et al. (2023) Close-range photogrammetry and 3D imaging. Berlin: De Gruyter, page 540).
[0008] As a second example, Zhang & Maga, 2023 discloses that the highest accuracy deliverable using 3D reconstruction using monocular cameras and six degree-of-freedom (6DOF) markers is 2% (see Zhang, C. and Maga, A.M. (2023) ‘An open-source photogrammetry workflow for reconstructing 3D models’, Integrative Organismal Biology, 5(1). doi:10.1093 / iob / obad024). To achieve 2% accuracy, Zhang and Maga required the object to be still and photographs to be taken very carefully. Specifically, collection of specimen images used in their study took 20-30 minutes using a DSLR camera. The length of time required, the requirement for the object to be still, and the equipment required makes the use of this technology unsuitable in many environments e.g. hospital clinics or consumer self-scan services.The above noted difficulties with creating accurate models of three-dimensional objects when using a monocular camera make the technique unsuitable for certain applications where a high degree of accuracy is required.
[0009] For example, a maximum average measurement error of 2mm is generally considered the maximum acceptable error for anthropometric measurements (see the following four references: (i) Katz D, Friess M. 2014. 3D from standard digital photography of human crania — A preliminary assessment. Am J Phys Anthropol 154: 152-8; (ii) Stull KE, Tise ML, Ali Z, Fowler DR. 2014. Accuracy and reliability of measurements obtained from computed tomography 3D volume rendered images. Forensic Sci Int 238: 133-40; (iii) Morgan B, Ford AL, Smith MJ. 2019. Standard methods for creating digital skeletal models using structure-from-motion photogrammetry. Am J Phys Anthropol 169: 152-60; (iv) Oriola LS, Oller NA, Martinez-Abadias N. 2022. Virtual anthropology: forensic applications to cranial skeletal remains from the Spanish Civil War. Forensic Sci Int 341: 111504. As such, use of monocular cameras with 6DOF markers for objects larger than 100mm is not suitable for applications where accuracy is required e.g. anthropometric measurements (since a 2% error of 100mm would be greater than 2mm). The average male foot length is 270mm. Therefore, a 2% error would equal to + / - 5.4mm, which is greater than the maximum average measurement error of 2mm.
[0010] Yet another problem when creating three-dimensional models is movement of the three-dimensional object during the scan. Movement of the three-dimensional object decreases the accuracy further, which compounds the difficulties noted above for creating three-dimensional models using a monocular camera. Therefore, it has been generally accepted that monocular camera solutions are not suitable for creating a three-dimensional model of a moving device (e.g. a wind turbine) or a moving subject (e.g. a human patient) using a monocular camera.
[0011] Against this background, there is a need for improvements in creating three-dimensional models of three-dimensional objects using a monocular camera.Summary of the disclosure
[0012] Against this background, there is provided:
[0013] a method of generating a three-dimensional model of an object wherein a plurality of marker arrangements is applied to the object, each marker arrangement of the plurality of marker arrangements comprising a plurality of targets arranged such that each target of said marker arrangement has a fixed relative position and orientation with respect to each other target of said marker arrangement, the method comprising:
[0014] receiving a plurality of images of the object from a camera;
[0015] creating a subset of images by, for each marker arrangement of each image of the plurality of images:
[0016] determining, based on said image, a target vector of each target of said marker arrangement with respect to the camera;
[0017] determining a geometric property of said marker arrangement based on the target vector of each target of said marker arrangement with respect to the camera;
[0018] comparing the determined geometric property of said marker arrangement with a reference geometric property of said marker arrangement, the reference geometric property being based on the fixed relative position and orientation of each target of said marker arrangement with respect to each other target of said marker arrangement; and
[0019] in an event that the determined geometric property is within a threshold of the reference geometric property, including said image in the subset of images; and
[0020] generating, based on the subset of images, the three-dimensional model of the object by determining, for each marker arrangement within the subset of images, a three-dimensional marker arrangement coordinate of said marker arrangement based on the target vector of at least one target of said marker arrangement.
[0021] By basing the three-dimensional model on the subset of images, rather than on the plurality of images, accuracy of the three-dimensional model is significantly increased. This is because the method makes it possible to exclude images from the subset of images which may have distortions (e.g. as a consequence of movement of the object) from the generation of the three-dimensional model with increased efficiency. Images which have such distortions may have marker arrangements whose geometric properties asdetermined from the image differ significantly from the reference geometric properties. Therefore, by comparing the determined geometric property of said marker arrangement with a reference geometric property of said marker arrangement it is possible to infer whether the image should be included in the generation of the three-dimensional model of the object.
[0022] The skilled person would appreciate that the method may be carried out whether the camera comprises a monocular camera, a stereoscopic camera, or another suitable type of camera. Because it is generally appreciated to be difficult to generate an accurate three-dimensional model using the monocular camera, a greater increase in accuracy may be obtained for the monocular camera than for the stereoscopic camera.
[0023] Additionally, the method makes it possible for one or both of the camera and the object to move while capturing the plurality of images. This is because the method makes it possible to determine how one image of the subset of images relates to another image of the subset of images based on each marker arrangement within the subset of images. Because it is generally appreciated to be difficult to move the stereoscopic camera without altering a calibration of the stereoscopic camera, this is particularly useful if the camera comprises the stereoscopic camera.
[0024] Brief description of the drawings
[0025] A specific embodiment of the disclosure will now be described, by way of example only, with reference to the accompanying drawings in which:
[0026] Figure 1 shows a system in accordance with the disclosure for generating a three- dimensional model of an object;
[0027] Figure 2 shows a first schematic representation of an object for which a three- dimensional model may be produced in accordance with the disclosure, wherein a plurality of marker arrangements is applied to the object;
[0028] Figure 3 shows a marker arrangement in accordance with the disclosure;
[0029] Figure 4 shows a method in accordance with the disclosure of generating a three- dimensional model of an object; and Figure 5 shows a method in accordance with the disclosure for creating a subset of images;Figure 6 shows a first representation of a marker arrangement, wherein the determined geometric property is within the threshold of the reference geometric property;
[0030] Figure 7 shows a second representation of a marker arrangement wherein the determined geometric property is not within the threshold of the reference geometric property;
[0031] Figure 8 shows a third representation of a marker arrangement wherein the determined geometric property is not within the threshold of the reference geometric property; and
[0032] Figure 9 shows a second schematic representation of the object, showing a first set of axes of a first target and a second set of axes of a second target.
[0033] Detailed description
[0034] According to one or more embodiments of this disclosure, a system 100 for generating a three-dimensional model of an object 106 is provided.
[0035] Figure 1 shows an embodiment of the system 100 for generating the three-dimensional model of the object 106.
[0036] The system 100 comprises a camera 104. The camera 104 is configured to capture a plurality of images of the object 106. The camera 104 may comprise or consist of a monocular camera. The camera 104 may be moved relative to the object 106 (e.g. by a user holding the camera 104 or by a moveable camera mount), such that the camera 104 captures images from different viewpoints relative to the object 106. For example, the camera 104 may capture at least one image from each side of the object 106. The object 106 may also be moved with respect to the camera 104 with the camera 104 fixed (e.g. by a fixed camera mount). The object 106 may be moved with respect to the camera 104 as the camera 104 is moved around the object 106 (e.g. the object 106 and the camera 104 may each be in motion). Because the plurality of images comprises at least one image from each side of the object 106, the plurality of images may be used to create the three-dimensional model of the object 106.
[0037] In alternative arrangements, the camera 104 may comprise or consist of a stereoscopic camera. This disclosure describes embodiments where the camera comprises themonocular camera. Nonetheless, the skilled person will readily appreciate from this disclosure they may use the embodiments of this disclosure with the stereoscopic camera.
[0038] The camera 104 may be calibrated based on a calibration target to provide one or more camera calibration parameters. The camera calibration parameters may comprise a zoom, a focal length, a distortion coefficient, and / or a resolution. The calibration target may comprise a chequerboard pattern. Alternatively, or in addition, the calibration target may comprise a ChArllco pattern. The camera calibration parameters may be determined based on an image captured by the camera of the calibration target.
[0039] The system 100 may be used for producing a three-dimensional model of the object 106. The object 106 is best seen in Figure 2, which will now be described. As shown in Figure 2, the object 106 comprises a three-dimensional object (the three-dimensional object shown in Figure 2 is a cylinder). The object 106 may comprise a limb and / or body part of a human patient or animal (e.g. an arm). The object 106 may comprise a wind turbine. As the skilled person would readily appreciate, the object 106 may comprise any fixed-shape three-dimensional object. In some arrangements (not shown in the Figures), the object 106 may comprise a first portion and a second portion. For example, the first portion and the second portion may be connected via a pivot (e.g. an arm). The object may comprise any other object to which a plurality of markers arrangements 202 may be applied.
[0040] A plurality of marker arrangements 202 is applied to the object 106 (an example marker arrangement of the plurality of marker arrangements is labelled 202 in Figure 2). Each marker arrangement 202 of the plurality of marker arrangements 202 comprises a plurality of targets (an example target of the plurality of targets is labelled 204 in Figure 2). Each target 204 may comprise a six-degree-of-freedom marker. For example, each target 204 may comprise an Arllco marker. Each target 204 may comprise a unique identifier, such as a barcode, a recognisable shape, a letter, a number, and / or the like. In the example shown in Figure 2, the six degree of freedom marker of each target 204 comprises a square and the unique identifier comprises a letter (A, B, C, or D). As shown schematically in Figure 2 (but not drawn to scale), due to the impact of perspective when capturing a two-dimensional image of a three-dimensional object, each of the targets 204 and the marker arrangements 202 have the appearance of being distorted from their actual square shape. In the same way, the cylinder has the appearance of a frustoconical element (wherein its diameter appears larger at the left of the image compared with the right of the image). Thisis the case notwithstanding that the object depicted in Figure 2 is a cylinder and the marker elements depicted in Figure 2 are square.
[0041] Figure 3 shows a front view of one marker arrangement 202 of the plurality of marker arrangements 202. Figure 3 may be representative of an image captured by the camera 104 with a lens of the camera 104 aligned with the marker arrangement 202. Unlike in Figure 2, since Figure 3 shows a front view of the marker arrangement 202, each of the targets 204 and the marker arrangements 202 are not distorted and therefore show their actual square shape. As shown in Figure 3, the plurality of targets 204 is arranged such that each target 204 has a fixed relative position and orientation with respect to each other target 204 of said marker arrangement 202. A reference geometric property of each marker arrangement 202 is based on the fixed relative position and orientation of each target 204 of said marker arrangement 202.
[0042] The reference geometric property may comprise a reference distance between each target 204 of said marker arrangement 202. For example, the reference geometric property may comprise a reference distance between a midpoint of a first target 204a and: a midpoint of a second target 204b, a midpoint of a third target 204c, and / or a midpoint of a fourth target 204d. The determined geometric property may comprise an average Euclidian distance between the midpoint of each target 204. The reference geometric property may comprise a reference distance between a corner of a first target 204a and: a corner of a second target 204b, a corner of a third target 204c, and / or a corner of a fourth target 204d. The corner of the first target 204a, the corner of the second target 204b, the corner of the third target 204c, and the corner of the fourth target 204d may be a like corner (e.g. a bottom left corner, a bottom right corner, a top left corner, or a top right corner. The reference distance may be determined by a measurement of each marker arrangement 202 (e.g. using a ruler, a distance sensor, and / or the like).
[0043] The reference geometric property may comprise a reference rotation between each target 204 of said marker arrangement 202. The reference rotation between each target 204 of said marker arrangement 202 may be about a reference axis of said marker arrangement 202. For example, the reference geometric property may comprise a reference rotation between a first target 204a and: a second target 204b, a third target 204c, and / or a fourth target 204d. The reference rotation between the first target 204a and: the second target 204b, the third target 204c, and / or the fourth target 204d may be defined as an anglebetween a side of the first target 204a and: a side of the second target 204b, a side of the third target 204c, and / or a side of the fourth target 204d respectively. The reference rotation may be determined by a measurement of each marker arrangement (e.g. a protractor, an angle sensor, and / or the like). The reference geometric property of each marker arrangement 202 of the plurality of marker arrangements 202 may be the same as (and / or within a geometric property tolerance value of) each other marker arrangement 202 of the plurality of marker arrangements 202.
[0044] The marker arrangement 202 may be planar and may extend in what are labelled an X axis and a Y axis. Thus, the marker arrangement 202 may extend in an X-Y plane which may also be labelled a marker arrangement plane. An axis orthogonal to the marker arrangement plane may be labelled a Z axis. Each target 204 of said marker arrangement 202 is within the respective marker arrangement plane. The reference axis of said marker arrangement may be parallel to the marker arrangement 202 plane. For example, as shown in Figure 2, the reference axis of the example marker arrangement labelled 202 may comprise its X axis or its Y axis. The reference axis may be orthogonal to the marker arrangement plane. For example, as shown in Figure 2, the reference axis of the example marker arrangement labelled 202 in Figure 2 may comprise the Z axis. The rotation between each target 204 of said marker arrangement 202 may be about the axis orthogonal to the respective marker arrangement plane 202. By providing each marker arrangement 202 such that each target 204 of said marker arrangement 202 is within the respective marker arrangement plane, the reference geometric property may be more precise (e.g. the geometric property tolerance value may be smaller). For example, the reference rotation between each target 204 of the marker arrangement 202 about the Z axis may be made more precise as a consequence of each target 204 being within the respective marker arrangement plane.
[0045] Each marker arrangement 202 may comprise three or more targets 204. The three or more targets 204 may be arranged as a grid. For example, Figure 3 shows four targets arranged as a two-by-two grid. As the skilled person will readily appreciate, the plurality of targets 204 of each marker arrangement 202 may be arranged with any other fixed arrangement. At least three targets 204 may allow a three-dimensional space to be determined from the at least three targets 204. A fourth target may provide a validation target such that the three-dimensional space determined from the at least three targets 204 may be validated.The plurality of marker arrangements 202 are configured to be applied to the object 106. For example, each marker arrangement 202 may be configured to be mounted on a substrate. The substrate may comprise a carbon fibre structure, such as a three-dimensional printed carbon fibre structure. Each marker arrangement 202 may be printed on a sticker comprising a first surface of the sticker comprising an adhesive configured to adhere to the substrate and / or the object 106 and a second surface of the sticker comprising the plurality of targets 204. Each marker arrangement 202 may be configured to be applied to the object 106 via the substrate. Each marker arrangement 202 may be configured to be applied to the object 106 directly (e.g. in an event the object is flat and / or has a fixed topology). The substrate may be inflexible. For example, the substrate may be configured to flex by no more than 0.1%, 1%, 2%, or 5% of a maximum diameter of the substrate. The substrate may be configured to flex by less than a flex threshold of the maximum diameter of the substrate. The flex threshold may be determined such that, in an event the substrate flexes, a distance between each target 204 changes the reference geometric property by no more than 10%, no more than 5%, and / or no more than 1%. The flex threshold may be determined such that, in an event the substrate flexes, the distance between each target 204 changes by more than the threshold distance value. The flex threshold may be determined such that, in an event the substrate flexes, the rotation between each target 204 changes by more than the threshold rotation value.
[0046] The substrate may comprise a flat surface. The flat surface may comprise the marker arrangement 202 (e.g. applied to the flat surface via the sticker). The substrate may comprise a domed surface opposite the flat surface. The domed surface may cause the substrate to have a decreased flex. The domed surface may be concave or convex. The domed surface may be adapted to a topology of the object. The domed surface may be configured to adhere to the object 106 directly.
[0047] The system 100 further comprises the controller 102. The controller is configured to receive the plurality of images of the object 106 from the camera 104. The controller 102 is configured to carry out the method 300 of generating the three-dimensional model of the object 106. In this way, the system 100 may be used to create the three-dimensional model of the object 106. The controller 102 may comprise a processer. In some arrangements (not shown in Figure 1), the controller may comprise one or more processors distributed across a network (e.g. an Application Programming Interface network and / or the like). The controller 102 may be integrated with the camera 104.The controller 102 is configured to receive the plurality of images from the camera 104. For example, the controller 102 may comprise an input terminal configured to receive image data from an output terminal of the camera 104. The input terminal of the controller 102 and the output terminal 104 of the camera 104 may be in communication via a wired connection (e.g. USB) and / or in communication via a wireless connection (e.g. Wi-Fi or Bluetooth™).
[0048] The controller 102 may be further configured to output the three-dimensional model as a three-dimensional data file. For example, the controller 102 may comprise an output terminal configured to output the three-dimensional model as a three-dimensional data file. The three-dimensional data file may comprise a ,3dm data file and / or a . stl data file and / or the like.
[0049] Figure 4 shows the method 300 of generating the three-dimensional model of the object 106. As explained above, the controller 102 may be configured to carry out the method 300. The method 300 will now be explained.
[0050] As shown in Figure 4, the method 300 comprises a step 302 of receiving a plurality of images of the object 106 from the camera 104. Therefore, the camera 204 may be configured to capture the plurality of images of the object 106 and output the plurality of images to the controller 102.
[0051] In some arrangements (not shown in Figure 4), the method 300 may further comprise determining whether a camera field of view contains at least one marker arrangement 202. For example, the controller 102 may be configured to determine whether the camera field of view contains at least one marker arrangement 202. Alternatively, the camera 104 may comprise a camera controller configured to determine whether the camera field of view contains at least one marker arrangement 202. In an event that the camera field of view does contain at least one marker arrangement 202, the method 300 may further comprise capturing an image of the field of view to include within the plurality of images. In this way, whether or not the camera field of view contains at least one marker arrangement 202 may be used as a trigger for determining whether or not to capture an image. In alternative arrangements, the method 300 may comprise determining whether a captured image contains at least one marker arrangement 202.In some arrangements (not shown in Figure 4), the method 300 may further comprise receiving a calibration check image from the camera 104. The calibration check image may be of the object 106. The method 300 may further comprise identifying at least one marker arrangement 202 within the calibration check image. The method 300 may comprise adjusting the reference geometric property based on the at least one marker arrangement 202 within the calibration check image. For example, the reference geometric property may be determined based on the measurement of each marker arrangement 202. In an event that the at least one marker arrangement 202 within the calibration check image is inconsistent with the reference geometric property (e.g. the at least one marker arrangement 202 may appear larger or smaller by a scale factor, such that the distance between each target 204 of the marker arrangement 202 increases or decreases respectively), the reference geometric property may be adjusted by the scale factor. In this way, accuracy may be increased if there is variability in a size of the plurality of marker arrangements 202 (e.g. due to a printing error). In an event that the scale factor is greater than 1%, greater than 0.5%, or greater than 0,1%, the method 300 may further comprise re-determining the camera calibration parameters based on the camera calibration target.
[0052] In some arrangements (not shown in Figure 4), the method 300 may further comprise determining whether each marker arrangement 202 of the plurality of marker arrangements 202 is identified within at least a threshold number of images of the plurality of images. For example, the threshold number of images of the plurality of images may comprise 30 images such that a normal distribution may be assumed about said marker arrangement 202. In an event that each marker arrangement 202 of the plurality of marker arrangements 202 is not identified within the threshold number of images of the plurality of images, the method 300 may further comprise receiving one or more additional images from the camera 104. As such, the camera 104 may capture one or more additional images. The one or more additional images may be captured and outputted to the controller 102 until each marker arrangement 202 of the plurality of marker arrangements 202 has been identified within the threshold number of images of the plurality of images.
[0053] As shown in Figure 4, the method 300 further comprises a step 304 of creating a subset of images. The step 304 of creating the subset of images is shown in more detail in Figure 5.As shown in Figure 5, the step 304 may be carried out for each marker arrangement 202 of each image of the plurality of images. The step 304 will now be described in more detail for a particular marker arrangement 202 of a particular image, referred to hereinafter as said marker arrangement 202 and said image. As shown in Figure 5, the step 304 further comprises a sub-step 402 of determining, based on said image, a target vector of each target 204 of said marker arrangement 202 with respect to the camera 104. The target vector may comprise a target rotation vector. The target vector may comprise a target translation vector. The target vector may comprise a target rotation vector and a target translation vector. The target vector may be determined based on a pixel density of the target as it appears in said image. For example, an increase in pixel density of a portion of the target may correspond with a translation towards and / or a rotation towards the camera 104. Correspondingly, a decrease in pixel density of the portion of the target may correspond with a translation away from and / or a rotation away from the camera 104. As an example, in embodiments where each target may comprise the Arllco, the target vector of each target may be determined based on an Open-Source Computer Vision Library (OpenCV) algorithm. The target vector may be determined based on the one or more camera calibration parameters.
[0054] In some arrangements, not shown in Figure 5, the sub-step 402 may comprise a step of determining one or more two-dimensional coordinates of each target based on said image. The one or more two-dimensional coordinates of each target 204 may comprise a coordinate of each corner of each target within said image. Determining the one or more two-dimensional coordinates based on said image may include a correction based on the one or more camera calibration parameters for a bend of the image (e.g. due to a shaping of a lens of the camera 104). The target vector of each target 204 may be determined based on the one or more two-dimensional coordinates of each target 204.
[0055] As shown in Figure 5, the step 304 further comprises a sub-step 404 of determining a geometric property of said marker arrangement 202 based on the target vector of each target of said marker arrangement 202 with respect to the camera 104. The geometric property determined in sub-step 404 is referred to hereinafter as the determined geometric property.
[0056] Based on the rotation vector and the translation vector of each target 204, it is possible to determine one or more of: a coordinate of a midpoint of each target 204 with respect to thecamera, a coordinate of a corner of each target with respect to the camera 104, an area of each target 204, a length of a side of each target 204. As an example, in embodiments where each target 204 comprises the Arllco, the coordinate of the midpoint of each target 204 with respect to the camera 104, the coordinate of the corner of each target 204 with respect to the camera 104 of each target 204 may be determined based on an Open Source Computer Vision Library (OpenCV) algorithm. The area of each target 204, and the length of a side of each target 204 may be determined based on the coordinate of the corner of each target 204 with respect to the camera 104.
[0057] As will be readily appreciated by those skilled in the art, the determined geometric property may correspond with the reference geometric property. As such, in embodiments where the reference geometric property comprises the distance between each target 204 of said marker arrangement, the determined geometric property also comprises a distance between each target 204 of said marker arrangement 202. Likewise, in embodiments where the reference geometric property comprises a rotation between each target 204 of said marker arrangement 202, the determined geometric property also comprises a rotation between each target 204 of said marker arrangement 202.
[0058] As shown in Figure 5, the step 300 of creating the subset of images further comprises a sub-step 406 of comparing the determined geometric property of said marker arrangement 202 with a reference geometric property of said marker arrangement 202. For example, a difference between the determined geometric property and the reference geometric property may be calculated.
[0059] As shown in Figure 5, the step 300 of creating the subset of images further comprises a sub-step 406 of, in an event that the determined geometric property is within a threshold of the reference geometric property, including said image in the subset of images. For example, the difference between the determined geometric property and the reference geometric property may be compared with a threshold value. In an event that the difference between the determined geometric property and the reference geometric property is smaller in magnitude than the threshold value, then said image may be included in the subset of images. The threshold may comprise a threshold distance value between each target of the marker arrangement 202. The threshold may comprise a threshold rotation value of each target 204 of the marker arrangement 202 with respect to the axis orthogonal to the respective marker arrangement plane.Referring again to Figure 5, the method 300 further comprises a step 306 of generating, based on the subset of images, the three-dimensional model of the object 106. The step 306 may comprise generating the three-dimensional model of the object 106 based on the subset of images. In some embodiments, images of the plurality of images which are not within the subset of images are not used to generate the three-dimensional model of the object 106. As a consequence, only images where the reference geometric property is within the threshold of the determined geometric property are used to generate the three-dimensional model, which in turn causes an increase in accuracy of the three-dimensional model.
[0060] The three-dimensional model may be generated by determining a three-dimensional marker arrangement coordinate of said marker arrangement 202 based on the target vector of at least one target 204 of said marker arrangement 202. For example, the three-dimensional model may be generated by determining the three-dimensional marker arrangement coordinate of said marker arrangement 202 based on an average position of each target 204 of said marker arrangement 202.
[0061] The three-dimensional marker coordinates may be with respect to a first image of the subset of images. The first image may be identified based on a marker arrangement 202 within the first image having the smallest difference between the determine geometric property and the reference geometric property. The first image may be determined arbitrarily (e.g. as a random image of the subset of images). The three-dimensional marker coordinates may be with respect of a first marker arrangement 202 of the first image of the subset of images. The method 300 may further comprise mathematically rotating and translating a second image with respect to the first image of the subset of images. The first marker arrangement 202 may also be within the second image (e.g. the first marker arrangement may be within the camera field of view as the first image and the second image are captured). The second image may be mathematically rotated and translated about the first marker arrangement 202. In some arrangements, more than one marker arrangement 202 may be within both the first image and the second image (e.g. the more than one marker arrangements 202 may be within the camera field of view as the first image and the second image are captured). The second image may be mathematically rotated and translated about a geometric centre, and / or a centre of mass, of the more than one marker arrangements 202. The second image may be mathematically rotated andtranslated with respect to the first image by a Procrustes superimposition algorithm and / or by a Kabsch algorithm. By mathematically rotating and translating the second image with respect to the first image, movements of the camera 104 and the object 106 with respect to each other may be corrected. As a consequence, it is possible for both the camera 104 and the object 106 to move during the capturing of the plurality of images by the camera 104 without decreasing the accuracy of the three-dimensional model.
[0062] The three-dimensional model may comprise a sparse point cloud. Each point of the sparse point cloud may correspond with the three-dimensional marker arrangement coordinate of each marker arrangement 202.
[0063] The three-dimensional model may further comprise a dense point cloud. The dense point cloud may be determined based on surface contrast of the object 106. In some arrangements, surface contrast may be added to the object 106 as a high contrast pattern and / or talcum powder. The sparse point cloud may be used as a point cloud reference for validating the dense point cloud.
[0064] As explained above, in the method 300, including said image in the subset of images depends on whether the determined geometric property is within the threshold of the reference geometric property. To illustrate the difference between a situation where the determined geometric property is within the threshold of the reference geometric property and a situation where the determined geometric property is not within the threshold of the reference geometric property, Figures 6 to 8 are provided. Figures 6 to 8 may be representations of the marker arrangement 202 based on the rotation vector and the translation vector of each target 204, as derived from an image of the marker arrangement 202 applied to the object 106 (e.g. as shown schematically in Figure 2).
[0065] Figure 6 shows a first representation of a marker arrangement 202, wherein the determined geometric property is within the threshold of the reference geometric property. The first representation of the marker arrangement 202 is geometrically similar to the marker arrangement 202 as shown in Figure 3. As such, the determined geometric property may be within the threshold of the reference geometric property.
[0066] Figure 7 shows a second representation of a marker arrangement 202 wherein the determined geometric property is not within the threshold of the reference geometricproperty. As shown in Figure 7, the second target 204b has been translated to the left compared with Figure 6. As such, the determined geometric property may not be within the threshold of the reference geometric property. For example, the difference between: (i) the distance between the midpoint of the first target 204a and the second target 204b, and (ii) the reference distance, may be smaller in magnitude than the threshold distance value.
[0067] Figure 8 shows a third representation of a marker arrangement wherein the determined geometric property is not within the threshold of the reference geometric property. As shown in Figure 8, the second target 204b has been rotated anticlockwise compared with Figure 6. As such, the determined geometric property may not be within the threshold of the reference geometric property. For example, the difference (i) between the rotation between the first target 204a and the second target 204b, and (ii) the reference rotation, may be greater than the threshold rotation value.
[0068] Figure 9 shows a second schematic representation of the object 106, showing a first set of axes of a first target and a second set of axes of a second target. The first set of axes corresponds with a first target 904a and the second set of axes corresponds with a second target 904b. As shown in Figure 9, the first set of axes is aligned with the second set of axes. Therefore, the rotation between the first target 904a and the second target 904b may be zero. As such, the rotation between the first target 904a and the second target 904b may be smaller in magnitude than the threshold rotation value. For simplicity of the diagram, the first set of axes is not labelled X, Y, Z, but it will be apparent to those skilled in the art which of the first set of axes correspond with X, Y, Z.
[0069] In some arrangements (not shown in the Figures) wherein the object 106 comprises the first portion and the second portion, the marker arrangements 202 applied to the first portion may comprise a first marker arrangement identifier (e.g. a first colour and / or a first barcode) and the marker arrangements 202 applied to the second portion may comprise a second marker arrangement identifier (e.g. a second colour and / or a second barcode), the second marker arrangement identifier being different to the first marker arrangement identifier. The method 300 may further comprise a step of generating a first three-dimensional model based on the marker arrangements 202 comprising the first marker arrangement identifier and a second three-dimensional model based on the marker arrangements 202 comprising the second marker arrangement identifier. The first and second three-dimensional models may then be incorporated into a combined three-dimensional model. The step of generating the first three-dimensional model and the second three-dimensional model may be carried out similarly to step 306 of method 300.
[0070] In a scenario where the first portion and the second portion are connected via a pivot, the first and second three-dimensional model may be incorporated into a combined three-dimensional model based on the marker arrangements 202 proximate the pivot. In alternative arrangements, the first and second three-dimensional model may be incorporated into the combined three-dimensional model based on a pivot coordinate, wherein the pivot coordinate is determined based on a marker arrangement coordinate of a marker arrangement 202 applied to the pivot.
[0071] In alternative arrangements, the marker arrangements 202 applied to the first portion may comprise a first unique identifier and the marker arrangements 202 applied to the second portion may comprise a second unique identifier. In alternative arrangements, the marker arrangements 202 applied to the first portion may be differentiated from the marker arrangements 202 applied to the second portion based on the unique identifier of each target 204 of each marker arrangement 202.
[0072] As the skilled person would readily appreciate, the method 300 may further comprise creating (e.g. by three-dimensional printing) a physical model based on the three-dimensional data file. For example, in an event that the object 106 comprises the limb and / or body part of the human patient or animal (e.g. the arm), the method 300 may further comprise creating, by three-dimensional printing, a prosthetic limb and / or body part of the human patient or animal (e.g. a prosthetic arm). In an event that the object 106 comprises the wind turbine blade, the method 300 may further comprise creating a replacement wind turbine blade. Manufacturing processes other than three-dimensional printing will be apparent to those skilled in the art. In alternative arrangements, the method 300 may comprise creating a complementary model shaped to fit the object 106 (e.g. an ankle-foot orthotic).
[0073] The method 300 may further comprise using the three-dimensional model to determine dimensions of the object 106 such as a length of the object 106, a diameter of the object 106, a volume of the object 106, a surface area of the object 106, and / or the like. The dimensions of the object 106 may be used to create the physical model.As will be appreciated by those skilled in the art, the steps of the method 300 of different embodiments may be readily combined or interchanged without departing from the scope of the claims.
Claims
CLAIMS:
1. A method of generating a three-dimensional model of an object wherein a plurality of marker arrangements is applied to the object, each marker arrangement of the plurality of marker arrangements comprising a plurality of targets arranged such that each target of said marker arrangement has a fixed relative position and orientation with respect to each other target of said marker arrangement, the method comprising:receiving a plurality of images of the object from a camera;creating a subset of images by, for each marker arrangement of each image of the plurality of images:determining, based on said image, a target vector of each target of said marker arrangement with respect to the camera;determining a geometric property of said marker arrangement based on the target vector of each target of said marker arrangement with respect to the camera;comparing the determined geometric property of said marker arrangement with a reference geometric property of said marker arrangement, the reference geometric property being based on the fixed relative position and orientation of each target of said marker arrangement with respect to each other target of said marker arrangement; andin an event that the determined geometric property is within a threshold of the reference geometric property, including said image in the subset of images; andgenerating, based on the subset of images, the three-dimensional model of the object by determining, for each marker arrangement within the subset of images, a three-dimensional marker arrangement coordinate of said marker arrangement based on the target vector of at least one target of said marker arrangement.
2. The method of any preceding claim, wherein:each marker arrangement of the plurality of marker arrangements comprises a respective marker arrangement plane such that each target of said marker arrangement is within the respective marker arrangement plane.
3. The method of any preceding claim, wherein:the reference geometric property comprises a reference rotation between each target of said marker arrangement.
4. The method of claim 3, where dependent upon claim 2, wherein:the rotation between each target of said marker arrangement is about an axis orthogonal to the respective marker arrangement plane.
5. The method of any preceding claim, wherein:the reference geometric property comprises a reference distance between each target of said marker arrangement.
6. The method of any preceding claim, wherein:the target vector comprises a target translation vector.
7. The method of any preceding claim, wherein:the target vector comprises target rotation vector.
8. The method of claim 3, wherein:the threshold comprises a threshold rotation value of each target of the marker arrangement with respect to the axis orthogonal to the respective marker arrangement plane.
9. The method of any preceding claim, wherein:the three-dimensional marker arrangement coordinates are with respect to a first image, the first image being identified based on a marker arrangement within the first image having the smallest difference between the determined geometric property and the reference geometric property.
10. The method of any preceding claim, further comprising:mathematically rotating and translating a second image of the subset of images with respect to a or the first image of the subset of images such that a marker arrangement of the second image overlaps with a marker arrangement of the first image.
11. The method of any preceding claim, wherein:the object comprises a first portion and a second portion, and the marker arrangements of the plurality of marker arrangements applied to the first portion comprise a first marker arrangement identifier and the marker arrangements of the plurality of marker arrangements applied to the second portion comprise a second marker arrangement identifier, the second marker arrangement identifier being different to the first marker arrangement identifier; the method further comprising:generating a first three-dimensional model based on the marker arrangements comprising the first marker arrangement identifier and a second three-dimensional model based on the marker arrangements comprising the second marker arrangement identifier.
12. The method of claim 11 , further comprising:incorporating the first and second three-dimensional models into a combined three-dimensional model based on the marker arrangements proximate a pivot.
13. The method of any preceding claim, further comprising:determining whether each marker arrangement of the plurality of marker arrangements is identified within at least a threshold number of images of the plurality of images; andin an event that each marker arrangement of the plurality of marker arrangements is not identified within the threshold number of images of the plurality of images, receiving one or more additional images from the camera.
14. The method of any preceding claim, further comprising:determining whether a camera field of view contains at least one marker arrangement; andin an event that the camera field of view does contain at least one marker arrangement, capturing an image of the field of view to include within the plurality of images.
15. The method of any preceding claim, further comprising:receiving a calibration check image from the camera;identifying at least one marker arrangement within the calibration check image; and adjusting the reference geometric property based on the at least one marker arrangement within the calibration check image.
16. A controller configured to carry out the method of any preceding claim.
17. A system for generating a three-dimensional model of an object, the system comprising:a camera;a plurality of marker arrangements configured to be applied to an object; and the controller of claim 16.
18. The system of claim 17 wherein the controller is further configured to:output the three-dimensional model of the object as a three-dimensional data file.
19. The system of claims 17 or 18, wherein:each marker arrangement of the plurality of marker arrangements is configured to be mounted on a substrate, and said marker arrangement is configured to be applied to the object via the substrate.
20. The system of claim 19, wherein:the substrate comprises a flat surface, the flat surface comprising the marker arrangement.
21. The system of claim 20, wherein:the substrate comprises a domed surface opposite the flat surface.
22. The system of any of claims 17 to 21, wherein:the camera consists of a monocular camera.
23. The system of any of claims 17 to 22, wherein:each marker arrangement of the plurality of marker arrangements comprises three or more targets arranged as a grid.
24. The system of any of claims 17 to 23, wherein:each target of each marker arrangement of the plurality of marker arrangements comprises a six degree of freedom marker.
25. The system of any of claims 17 to 24, wherein:each target of each marker arrangement of the plurality of marker arrangements comprises a unique identifier.