Structural scanner

Abstract

A structural scanning and analysis apparatus comprises a solid-state light-based scanner arranged to capture a plurality of images of a target surface, the target surface being at least substantially flat and having an area, A; a depth-map generation module arranged generate a depth map of the target surface using surface normal information derived from the plurality of images, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy of at least ± √A / 108;an analysis module configured to analyse at least one of the topographical attributes of the target surface using the depth map; and a feedback module arranged to provide an output generated based on a result from the analysis module. The output may comprise a visual output projected onto the target surface itself, or onto a press used to form the target surface. The output may be or comprise updates to manufacturing settings for making the target surface, the updates optionally comprising machine-readable instructions.

Description

[0001] STRUCTURAL SCANNER

[0002] The invention relates to a method and apparatus for scanning a target surface, processing the scan data to form a depth map of the target surface, and analysing that depth map to provide a useful output regarding the three-dimensional (3D) structure (shape) of the target surface. Scanners as described herein have particular utility in scanning relatively large and flat items - such as sheets of material prepared for use in creating packaging- and may be applied more generally to targets in which the size of surface features to be analysed is much smaller than the extent of the surface to be scanned; they may therefore be described as sheet scanners. In addition, a parametric scanner design for a photometric stereo scanner for use in such a system is disclosed, as is an in situ visualisation apparatus arranged to provide feedback on the target surface’s structure on the target surface itself or on a corresponding machine surface. The invention may be of particular utility in the carton / packaging industry, battery production industry, and other industries that produce or consume sheets or rolls of materials.

[0003] It known to provide Digital Inspection Tables (DITs) and Digital Makeready Tools (DMTs) which can analyse artwork and printed text on sheets of packaging and check it for errors, as exemplified by the Digital Inspection Table developed by the applicant. Devices for analysing creases (an example of a 3D feature) are also known, often referred to as crease and fold analysers, however such devices are only able to scan single 2D cross-sections of individual creases, one after another.

[0004] It is an object of the invention to provide a structural scanner which can provide feedback on 3D features of a target surface (this may be referred to as “2.5D” rather than 3D, as, whilst length, width, and height are all measured, only a surface is generally analysed). This may be of particular utility in quality control processes, in which products are to be assessed in comparison to a template or standard, and the feedback provided may be used to adjust manufacturing settings so as to correct or fix already-prepared items, and / or to improve production quality of subsequently produced items. Aspects of the invention may therefore allow for control of one or more sheet processing machines, or other manufacturing machines, to be determined or adjusted based on structural scan data.

[0005] Whilst the invention is primarily discussed herein in its application to the manufacture of packaging (e.g. cardboard and paper boxes and packets), it will be appreciated that the same techniques and apparatuses may be applied to a wide range of products and materials. For example, an approach developed to identify and measure fold lines or creases in cardboard may also be used to identify protruding threads in a fabric weave, or cracks in an aerospace component.

[0006] According to a first aspect of the invention there is provided a structural scanning and analysis apparatus comprising: a solid-state light-based scanner (which may be referred to as an optical scanner) arranged to capture at least one image of a target surface, the target surface being at least substantially flat and having an area, A; a depth-map generation module arranged generate a depth map of the target surface using surface normal information derived from the at least one image, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy of at least + JA / io8);an analysis module configured to identify and measure at least one of the topographical attributes of the target surface using the depth map; and a feedback module arranged to provide an output, the output being generated based on a result from the analysis module.

[0007] The scanner is described as solid-state as the scanning components have no parts intended to move in use (e.g. unlike a bar scanner in which a bar is moved across a target surface) - the apparatus may still have moving parts, e.g. a drawer to move a target surface into position. For a solid-state photometric stereo scanner, the scanner may have no movement of the camera(s) or lights, and optionally also not of the target surface with respect to the camera(s) or lights (it will be appreciated that the individual components, e.g. cameras, may have internal moving parts such as a mechanical shutter, focusing components, or microelectromechanical circuitry). The lack of moving parts for scanning may increase speed of the scanning process and / or improve durability of the apparatus. For example, such an apparatus may capture a target surface of at least 1070 mm x 800 mm, at an accuracy of ± 0.03mm or ± 0.05mm, in no more than 30 seconds. Data capture may take less than 0.9 s, and optionally about 0.5 s for each lighting scenario used, and the processing may take the rest of the time. All scan data may therefore be captured within two to ten seconds, and optionally in less than three seconds. Various different lights and camera may be used. Infra Red (IR) lights or other lights invisible to humans may be used with an IR camera. Coloured lights may be used with a coloured camera. Coloured or monochrome lights may be used with a monochrome camera. For instance, coloured RGB lights that correspond to RGB camera sensitivities can also be used to achieve a system that runs at interactive rates capturing and encoding required lighting information within a single image frame. In some embodiments, the light sources used in the light-based scanner may not require the use of lasers and / or LIDAR. Laser cameras and / or LIDAR are not used at all in some implementations, such that the images generated by the camera are purely 2D, with the 3D information being derived from comparisons of 2D images.

[0008] The term “substantially flat” may be taken to mean that the target surface has a relatively small extent in the z-direction as compared to its extent in an x-y plane, for example having a length and width each at least fifty times its maximum depth perpendicular to that area. Herein, it will be understood that the “depth” of the target surface refers to height variation of the surface, as opposed to a thickness of the item providing the target surface. In some implementations, an average dimension of the area, A, may be at least fifty or one hundred times greater than a maximum depth of the target surface perpendicular to that area. The surface may be described as being globally planar with localised features protruding from that plane. 3D attributes of the target surface may include it, or a specific region of it, being flat - i.e. a lack of 3D features is an example of a 3D attribute of a surface.

[0009] In some implementations, the scanner may be arranged to capture a single image, e.g. with multiple differently-coloured lights to provide independent lighting directions within a single image (e.g. using the RGB channels of a single image to store the three different light directions), however more generally a plurality of images may be used. The plurality of lights for the scanner are generally arranged to provide at least three different lighting directions that are all linearly independent.

[0010] It will be appreciated that a greater accuracy entails a smaller value of the associated error for that accuracy - an accuracy of at therefore covers any accuracy with a larger denominator, e.g. For example, if the scanned area, A, was 2 m2, then the depth measurement error could be no more than ±0.14 mm, i.e. ±140 microns, and may be for example ±30 microns, ±10 microns, or ±2 microns. The accuracy is defined with respect to the area scanned as it is the relative accuracy to area which is important - the apparatus can be scaled as appropriate depending on the area of the target surface. Submillimetre resolution may therefore be provided across areas of greater than 0.5 m2in a single scanning process.

[0011] The light-based scanner may be a photometric stereo scanner, and may comprise a plurality of lights and at least one camera. The photometric stereo scanner may comprise a plurality of lights located above the target surface, and optionally one or more lights below the target surface to backlight it. The plurality of lights and the at least one camera may be arranged such that specular reflections - and in particular mirror points as discussed below - fall outside of the area scanned / the target surface. The plurality of lights and the at least one camera may be arranged such that specular reflections fall at least substantially outside the target surface area. Cross-polarisation may also be used to reduce specular artefacts. It may not be possible to make all specular reflections fall outside the target surface boundary (as there is surface roughness etc.) but “mirror points” - i.e. points on the planar surface where the light from the source is perfectly reflected towards the camera, creating a bright spot or glare - can be determined and the light position(s) adjusted so that these fall outside of the area to be scanned.

[0012] The depth map may show 3D topographical attributes of the target surface to an accuracy of at least

[0013] The target surface may have an area of at least 0.15 m2, 0.2 m2, 0.5 m2, 0.8 m2, 1 m2, or 2 m2(the area of the target surface, A, being the area scanned). The depth map may have micron-level resolution; for example, ±1-5 pm or ±10-50 pm, depending implementation details. The accuracy may be e.g. at least ± 30 pm, ± 20 pm, ± 10 pm, ± 5 pm, ± 2 pm, or ± 1 pm in depth measurements. The position accuracy in the plane (x, y) may be equal to, or in some cases lower than, the accuracy in the depth measurements (z), for example being at least ± 100 pm, ± 50 pm, ± 20 pm, ± 10 pm, or ± 2 pm.

[0014] The analysis module may be arranged to analyse the at least one image in addition to the depth map, and optionally to provide an analysis result comprising visual information as well as depth information.

[0015] The output from the feedback module may comprise data arranged to be used by a manufacturing machine to adjust its settings so as to modify an identified 3D topographical attribute of a surface formed by the manufacturing machine - the output itself may be one or more computer-readable instructions arranged to cause the manufacturing machine to adjust its settings, or may be data which the manufacturing machine, or a processor associated therewith, uses to generate machine-readable instruction. Here, “manufacturing machine” refers to any machine used in forming / manufacturing the scanned surface. Alternatively or additionally, the output may comprise human-readable identification of errors or unexpected features in the target surface.

[0016] The output may be or comprise one or more of: a visual output presented to a user on a screen; a visual output presented to a user by projection onto the target surface; a quality control pass / fail result; a list of discrepancies between a template and the scanned surface; and instructions for a user or machine to correct one or more discrepancies between a template and the scanned surface. The output from the feedback module may comprise one or more of: machine-readable data or instructions arranged to cause an adjustment to settings or parameters of a manufacturing device so as to modify one or more attributes of subsequently manufactured versions of the target object; a human-readable report identifying detected features, attributes and / or errors in the scanned target surface, the report comprising one or more of textual, graphical, and pictorial elements; inference of further information based on the position of said features relative to each other; visualisation of the scanned target surface with detected features highlighted or annotated, the visualization being displayable on a screen and / or projectable onto the physical target surface; quantitative metrics, statistics and / or pass / fail determination results in respect of detected features and attributes of the scanned target surface; and a quality-control assessment of the surface, for instance, packaging.

[0017] The analysis module may comprise a comparison module arranged to compare at least one 3D topographical attribute of the target surface as shown by the depth map to a template for the target surface. The output provided by the feedback module may be generated based on a result of the comparison. The comparison may comprise identifying one or more 3D topographical features of the target surface based on the template. At least one identified feature may be or comprise braille provided on the target surface (the output may comprise an assessment of the legibility of the braille); or a crease on the target surface (the output may comprise an assessment of the shape or location of the crease, a tolerancing profile of the crease, a consistency of the crease, or a foldability of the crease - i.e. whether or not geometry and / or material properties would allow the material to be folded along the crease, or a combination of marked creases). Additionally or alternatively, the at least one identified feature may be or comprise a feature created using a cutting knife (the output optionally comprising an assessment of the impact and / or presence of the feature created using a cutting knife); an embossed or debossed feature (the output optionally comprising an assessment of the presence or consistency of the embossed / debossed feature); a partial cut through the target surface (the output optionally comprising an assessment of the presence of the partial cut based on a comparison to the reverse of the target surface and / or light transmission through the target surface); or a burst or foil (the output optionally comprising an assessment of the presence or absence of a burst or foil in the target surface). The template may be generated manually, or by scanning an approved target surface with the light-based scanner, generating a depth-map of the approved target surface using the depth-map generation module, and storing information extracted from the depth-map (and / or the complete depth-map itself), and optionally associated tolerances, for use as the template. An approved target surface used in template generation may be smaller than a typical target surface in use - for example, the approved target surface may be the surface of a single blank / item, whereas multiple such items may generally be arranged in a sheet for scanning instead of being scanned individually. In such cases, generation of the template may comprise duplicating a template for the approved target surface in a pattern to cover at least a majority of the typical target surface area (based on a known / input arrangement of blanks into the sheet); or the comparison module may be arranged to compare the template to each of a plurality of regions of the target surface area in use. A template not made by scanning a target surface may be referred to as a virtual template. Irrespective of the template generation approach used, the template may be provided as a pre-defined type of data file, for example a PDF.

[0018] The comparison module may assess whether or not one or more of the following properties match the template: (a) presence or absence of a feature included in the template (which may be referred to as a template feature); (b) height; (c) width and / or length; (d) shape; (e) position; and (f) the presence of an unexpected feature (i.e. a feature not present in the template). Optionally, the comparison module may assess how any deviations compare to allowed variations / tolerances.

[0019] Some implementations may not use a template, or may use other comparisons in addition to a template. For example, the analysis module may be arranged to: search the image for a plurality of instances forming a repeated pattern of features and / or attributes in the target surface; and compare the plurality of instances to one another, and / or to pre-defined tolerances for the specific features and / or attributes, to determine if a desired specification is met. Optionally, searching the image for the plurality of instances may comprise searching for areas of graphics or text in the image of the target surface. In this way, a previously-unknown repeating pattern may be identified, and a consistency analysis between repeats may be performed.

[0020] In implementations with or without templates, the feedback module and / or analysis model may be arranged to: analyse a group of adjacent features or attributes in the target surface; and generate an output based on the interaction between the features or attributes, wherein preferably the group of features or attributes include at least two of a different type. For example, creasing card requires applying pressure on both sides of the crease, which may result in flattening braille. The feedback module may therefore suggest re-embossing braille dots after creasing, or moving a set of braille dots further from a crease, due to the interaction between the crease and the braille.

[0021] The apparatus may comprise an image processing module configured to receive the captured image(s) and apply illumination correction to normalise illumination intensity and / or sensor response non-uniformity across the images.

[0022] The light-based scanner may be arranged to scan flattened or pre-assembly packaging, and the output from the feedback module may comprise at least one of: instructions for adjustment of a die-cutter or press used in manufacturing the packaging; and a quality-control assessment of the packaging.

[0023] The feedback module may comprise a projector arranged to project one or more colours or patterns onto a viewing surface, the viewing surface being: the target surface; a shaped surface of a manufacturing machine arranged to produce the target surface (e.g. a die), wherein the shaped surface is shaped to correspond to the target surface so as to form the target surface; or a reverse side of such a shaped surface (e.g. the back of a die). As such, one or more 3D topographical features identified within the depth map and / or one or more generated outputs can be projected onto the viewing surface. A user may therefore view the results of the comparison in situ, on the target surface or on e.g. a press used to form that surface, facilitating immediate identification of problem areas / features, and often revealing details otherwise invisible to the naked eye (e.g. micron-scale size errors). Projecting the information onto the reverse side of the shaped surface of a manufacturing machine may facilitate adjustments such as adding or removing packing material to increase or decrease embossing depth. The light-based scanner and the feedback module may be integrated such that results of the analysis and comparison can be projected onto the target surface whilst the target surface remains in the position in which it was scanned (e.g. in a final position for a scanning apparatus in which the target surface is moved across a scanner light bar, or in the single scanning position in embodiments in which the surface is not moved during scanning). Alternatively, the feedback module may be provided by a separate apparatus.

[0024] The apparatus may therefore be used to capture a 3D profile of a formed sheet, extract features (creases, braille, embossing, etc), and then compare the scan against a pre-defined standard. Testing of prototypes has shown that the apparatus can - in ~20 seconds - make automatic quality judgments that would take experienced operators upwards of 15-40 minutes to get right. Various sheet processing machines may be used to produce a formed sheet - for example, die-cutters, presses, embossers, foil machines, laminators, and the like.

[0025] In the field of manufacturing of packaging, there are two general approaches: Sheet-fed, in which single, individual pages or sheets are fed into a press or other sheet processing machine; and Web press production, in which a continuous roll of material is used (e.g. paper, card, thin polymeric material, or a composite material such as polymer-laminated card). In either case, the result is a plurality of nets, or blanks, for packaging (which may or may not remain interconnected until a later separation step, e.g. by easily- breakable linkages such as perforated lines). A single blank can be laid flat for analysis, or a plurality of these blanks can be arranged at least substantially in the same plane for ease of assessing multiple blanks at once - this generally flat arrangement of blanks is referred to as a “sheet” (due to its general shape) whether the blanks were formed from a single-page sheet or from a web reel. The two processes may therefore be thought of as “sheet-to-sheet” and “web-to-sheet” processes. As well as quality approvals in the carton packaging field, the technology also has implications for the sheet / web forming industry because, for the first time, it enables a die cutter or other sheet processing machine to observe and assess its own output. With this new capability, autonomous die cutting machines that adjust themselves (e.g. pressure, plate position, speed, etc) based on what is observed can be provided, either with a scanner as described herein integrated into the die cutter itself, or with such a scanner in communication with the die cutter (directly or indirectly). Die cutting is a fabrication process that uses specialised machines - generally referred to as die-cutters - to convert stock material by cutting, forming, and shearing. Die-cutting therefore refers to forming creases and embossing (e.g. by pressing with non-sharp parts of a die) as well as cutting of the material. The technology may also be exploited beyond the die cutting / blank-forming industry. For example, to scan printing plates, to scan metal surfaces for defects, to check part conformance and dimensions, to check materials and fabrics during production, to generate textures for graphics, and all manner of semi-flat surfaces that contain 3D features - intentional (e.g. surface texture or embossings) or otherwise (e.g. flaws). The light-based scanner may be arranged to scan flattened, or pre-assembly, packaging, such that the packaging can be arranged as an at least substantially flat sheet. The output from the feedback module in such embodiments may comprise at least one of: instructions for adjustment of a die-cutter or press (or other sheet processing machine) used in manufacturing the packaging; and a qualitycontrol assessment of the packaging.

[0026] The processing (depth-map generation, comparison, and output generation) may be performed locally or remotely with respect to the light-based scanner. For example, at least one of the depth-map generation module and the analysis module may be provided by a server - or multiple servers - remote from the lightbased scanner. At least one of the depth-map generation module and the analysis module may be provided by a computational device integrated with, or physically connected to, the light-based scanner. At least one of the image processing module (where present), the depth map generation module, the analysis module, and the feedback module may therefore be implemented on a separate system communicatively coupled to the light-based scanner over a data communication network, such that at least one of the illumination correction, depth map computation, feature analysis, and feedback generation is performed remotely from the lightbased scanner.

[0027] In implementations using comparison to a template, the template may be generated by scanning an approved target surface with the light-based scanner, generating a depth-map of the approved target surface using the depth-map generation module, and storing that depth-map and associated tolerances for use as the template. The same scanner may therefore be used to make the template for later use in comparisons, or a template generated by another scanner (or indeed manually) may be provided for use. A PDF template may be generated for ease of use by multiple machines / template readers. The specification of surface features such as crease or embossing quality has previously not been digitised - at least in the field of packaging - because no industry standard software packages (i.e. Esko ArtiosCAD, PackZ, Adobe PDF) model the complex interaction a die tool and counter have on a given substrate, and doing this manually is excessively time-consuming. Embodiments described herein may therefore address this by implementing an “examplebased learning” method where customer-approved blanks (an already industry-standard practice) are scanned and saved for future comparison. By digitising the approved crease profiles and embossing characteristics, the data can be used as a reference for producing similar or repeat packaging jobs. The comparison performed by the comparison module may comprise identifying one or more features of the target surface based on the template. At least one identified feature may be or comprise braille provided on the target surface. The output provided by the feedback module may comprise an assessment of the legibility of the braille.

[0028] According to a further aspect of the invention, there is provided a packaging quality control apparatus comprising: a solid-state light-based scanner arranged to capture at least one image of a target surface of a sheet arranged to be used to form packaging, the target surface having an area of at least 0.15 m2and being substantially flat; a depth-map generation module arranged to generate a depth map of the target surface using surface normal information derived from the at least one image, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy of at least ±0.1 mm; an analysis module configured to identify and measure at least one of the topographical attributes of the target surface using the depth map; and a feedback module arranged to provide an output, the output being generated based on a result from the analysis module.

[0029] The target surface may be at least substantially flat, such that the target surface has a maximum depth perpendicular to the area of less than 10 mm, and optionally of less than 5 mm, 2 mm 1 mm, or 0.5 mm. The depth map may show features of the target surface to an accuracy of at least ±0.05 mm. The target surface may have an area of at least 0.25 m2, 0.5 m2or 0.8 m2.

[0030] The sheet may be a paperboard sheet, and may be die-cut. To use the apparatus, an operator may place a sheet of packaging blanks on a bed of, or associated with, the scanner. Within as little as 30 seconds, the operator may be presented with a user-friendly analysis of the surface shape and optionally also suggestions of one or more actions to adjust the machine, and / or a quality-control assessment. The analysis may include one or more of the following:

[0031] • Creases - conformance to width, height, and angle / skew tolerances;

[0032] • Braille - conformance to ISO 17351 :20131, PDF comparison, heights, and legibility;

[0033] • Embossing / Debossing - consistency against a template or tolerance;

[0034] • Sheet Pressure - revealing high and low die stations, and optionally identifying die stations in need of realignment and suggested position or pressure change for each affected die station, or replacement of related tooling / parts due to wear;

[0035] • Dimensions - critical measurements, e.g. packaging box length, flap size and angle, etc.

[0036] The analysis results outputs may be viewed on a Human Machine Interface (HMI) and / or projected onto the target surface itself as described below. For creases, height, width, angle, and / or skew (optionally at multiple locations along the crease) and length, position, and / or global direction (for the crease as a whole) may be determined and assessments that analyse crease specific attributes relative to the position and angle in the wider sheet may be made (e.g. to compute an counterplate offset if all creases in one direction are angled / skewed to one side). For embossing / debossing, depth, consistency, size, and / or position may be assessed. Security markers, cuts, nicks, tapes, and overall dimensions and shape (e.g. including warp) may also be identified. In implementations in which one or more images are used in the analysis as well as the depth map information, feedback on 2D features such as barcodes and other printed symbols, or colour, may also be provided. These outputs may be provided with respect to individual cartons on a sheet, and / or the sheet as a whole.

[0037] The packaging quality control apparatus may be as described with respect to the first aspect. The apparatuses described above are arranged to be used with substantially flat scanned surfaces, and so may be described as Sheet Scanners. These Sheet Scanners can be used to make automatic quality judgments in less than a minute, for example in 30 seconds, whereas the same assessments could take experienced operators upwards of 15-40 minutes to get right. The printing industry has used camera-based monitoring of print quality for some time, but the complex nature of cutting and creasing has kept such a system out of reach for 3D shape. The Sheet Scanners work by capturing a 3D profile of a sheet (e.g. a die-cut sheet), extracting features (creases, braille, embossing, etc), and then comparing the scan result against a template and predefined standards. The template may be provided as a digital file (e.g. a PDF file).

[0038] According to a second aspect, there is provided a method of scanning and analysing the structure of a target surface, where the target surface is substantially flat and has an area, A. The method comprises: capturing at least one image of the target surface using a solid-state light-based scanner; generating a depth-map of the target surface using surface normal information derived from the at least one image, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy of at least identifying and measuring at least one of the topographical attributes of the target surface using the depth map (this may comprise comparing the depth map to a template for the target surface); and providing an output, the output being generated based on a result from the analysis module.

[0039] The method may be performed using the apparatus of the first aspect, and any of the features described with respect to the first aspect may apply. The capturing step may take less than one minute for the whole surface, and optionally less than 20 seconds, 10 seconds, 5 seconds, or less than one second. The light-based scanner may be solid-state, such that no movement is required during the capturing step.

[0040] According to a third aspect, there is provided a method of generating a template for the structure of a target surface where the target surface is substantially flat and has an area, A. The method comprises: capturing at least one image of an approved product having a surface selected to be used as a model for the target surface using a solid-state light-based scanner; generating a depth-map of the surface of the approved product using the at least one image, wherein the depth map shows features of the approved product surface to an accuracy of at least and generating a template based on the depth map.

[0041] The template may be used as the template discussed in either or both of the preceding aspects, and may be used by the apparatus of either or both of the preceding aspects.

[0042] The method may further comprise generating a set of manufacturing settings or instructions for manufacturing the target surface, optionally in the form of computer-readable instructions for one or more manufacturing machines and / or user-readable instructions - based on the depth map and knowledge of machine capabilities. In such embodiments, the method may further comprise updating / revising the instructions / manufacturing settings based on comparison results obtained when target surfaces manufactured according to the instructions are compared to the template. Similarly, even in embodiments in which a template and set of instructions is generated by another method, improvements to the instructions / corrections to manufacturing machine control may be made based on the results of comparisons between the template and a depth-map of a sheet made to the instructions. A sensing technology and digital workflow is therefore provided that allows a new generation of automated sheet processing machines - e.g. die cutters - to be developed.

[0043] Various aspects of the invention may therefore provide faster product preparation for sale (e.g. due to quicker quality control and / or automated machine control updates), reduced wastage (errors identified and corrected more quickly), increased production capacity (speed), simplified operator responsibilities (due to automation taking over some or all of the quality control and / or machine reconfiguration processes), and enhanced quality assurance (due to higher accuracy, improved transparency of the process as human judgement is avoided, and / or more flexibility in outputs due to automated reporting options).

[0044] According to a fourth aspect of the invention, there is provided an analysis and manufacturing system, comprising: a scanner arranged to provide scan data from which 3D topographical attributes of a target surface can be derived, the target surface being at least substantially flat; a depth-map generation module arranged generate a depth map of the target surface based on the scan data; an analysis module configured to identify and measure at least one of the topographical attributes of the target surface using the depth map; a feedback module arranged to provide an output, the output being generated based on a result from the analysis module; and a manufacturing apparatus arranged to be used in manufacturing the target surface, and to receive the output from the feedback module, wherein the output from the feedback module comprises data for one or more adjustments to settings of the manufacturing apparatus, and wherein the manufacturing apparatus is configured to implement the one or more adjustments in response to receipt of the output.

[0045] The analysis and manufacturing system may comprise the structural scanning and analysis apparatus of the first aspect, the structural scanning and analysis apparatus providing the scanner, depth-map generation module, analysis module, and feedback module. The analysis and manufacturing system may be used to implement the method of the second aspect.

[0046] According to a fifth aspect of the invention, there is provided a method of automatically adjusting settings of a manufacturing apparatus arranged to be used in manufacturing a target surface, the method comprising: obtaining scan data from which 3D topographical attributes of a target surface can be derived from a scanner, the target surface being at least substantially flat; generating a depth map of the target surface based on the scan data; identifying and measuring at least one of the topographical attributes of the target surface using the depth map; providing an output to the manufacturing apparatus, the output being generated based on a result from the identifying and measuring of the at least one of the topographical attributes and comprising data for one or more adjustments to settings of the manufacturing apparatus; and at the manufacturing apparatus, receiving the output and implementing the one or more adjustments in response to receipt of the output.

[0047] The method may be performed using the apparatus for the first and / or fourth aspect. The output may be or comprise machine-readable instructions for one or more adjustments to settings of the manufacturing apparatus. The adjustments may affect (e.g. correct, or cause to be correctly formed) one or more 3D topographical attributes of a surface subsequently made by, or corrected by, the manufacturing apparatus.

[0048] According to a sixth aspect, there is provided a photometric stereo scanner comprising: a plurality of lights located spaced from a target surface, each light having a luminous area from which light emanates; and a camera, and wherein the plurality of lights and the at least one camera are in fixed positions with respect to each other, the lights and camera being arranged such that specular reflections fall at least substantially outside the target surface by trigonometrically determining a closest mirror point to a boundary of the target surface for each light-camera pair, each mirror point for each light-camera pair being the point on a plane including the target surface where the angle between the plane and a corresponding point within the luminous area of the light is equal to the angle between the plane and a centre of the camera, and ensuring that every closest mirror point is outside of the boundary of the target surface such that all mirror points fall outside of the boundary.

[0049] The luminous area may comprise both the area of light sources, such as LEDs, and the area of reflectors that form part of the light, such as purpose-built reflectors arranged to direct the light or incidental reflectors such as metal screws. The luminous area therefore includes areas of both original light emission / production, and light reflection. Cross-polarisation of light may be used to further reduce specular effects.

[0050] The photometric stereo scanner may comprise a single camera, and the single camera may be located centrally with respect to the target surface, and arranged to face the target surface. The one or more cameras may be arranged to capture an area of at least 0.25 m2, 0.5 m2, 0.8 m2, 1 m2, or 2m2in each image captured. An image maybe formed from a collage / tiling of images from each camera in implementations with multiple cameras.

[0051] The photometric stereo scanner of the sixth aspect may be used as the light-based scanner of the apparatus of the first aspect, and may be used in performing the method of the second and / or third aspect.

[0052] According to a seventh aspect, there is provided use of the structural scanning and analysis apparatus of the first aspect, and / or of the photometric stereo scanner of the sixth aspect, in one or more of: quality-control of packaging; assessment of the legibility of braille; identification and assessment of creases; sheet pressure analysis; validation of embossing (or equivalently of debossing); surface inspection; and plate and tool validation. The Sheet Scanner described herein is the first technology to be fast enough, accurate enough, and large enough to make meaningful quality judgments about die cutting processes. More generally, previously- developed 3D scanners are generally designed for objects with more similar sizes in all three dimensions, and / or lack the required detail, and have not been designed for substantially flat surfaces. The scanners as described herein remedy this deficiency and fill this niche, with applicability across a range of fields.

[0053] According to an eighth aspect of the invention, there is provided a feedback visualisation apparatus comprising: a viewing surface onto which a Iprojection image is to be projected, the viewing surface being either:

[0054] (i) a target surface; or

[0055] (ii) the front or reverse side of a shaped surface of a manufacturing machine arranged to produce the target surface, wherein the shaped surface is shaped to correspond to the target surface so as to form the target surface; a memory arranged to store the projection image, the projection image being determined based on at least one result of an analysis of at least one 3D topographical attribute of the target surface, wherein the projection image is adjusted based on a calibration mapping to account for the 3D position of the viewing surface; and a projector in a known position relative to the viewing surface, the projector being arranged to project the projection image onto the viewing surface so as to visually show the at least one analysis result for the target surface in a location on the viewing surface corresponding to a location of the topographical attribute of the target surface. This in situ presentation may be described as spatial augmented reality.

[0056] The manufacturing machine may be a press or die-cutter, and the shaped surface of the manufacturing machine may be a press with patching sheets. The feedback visualisation apparatus may comprise a bed on which the target surface is arranged to be positioned for viewing and the projector may be in a known position relative to the bed. Information on the thickness of the target surface (e.g. from the depth map, looking at edges of the target surface) may be used to refine an estimate of the target surface position. This may be unnecessary for thin targets.

[0057] The projection image may be determined based on at least one result of a 3D topographical comparison between the target surface and a template for the target surface. The at least one result may include an identification of a feature of the template which is missing from the target surface, and the projector may be arranged to project an indication of the missing feature onto the target surface in the intended location of the feature, and optionally to mark / highlight it as missing or otherwise specially classified.

[0058] The apparatus may comprise at least one projector with a lens facing - and optionally at least substantially parallel to - the target surface. The or each projector or arrangement of projectors may be a laser projector. The or each projector may be a projector of the type used by cinemas, or a home-cinema projector, for example depending on desired resolution, brightness, and target surface area.

[0059] The feedback visualisation apparatus may comprise a backlight arranged to illuminate the target surface from behind the target surface. Such a light may facilitate detection and / or highlighting of cut-out regions or edges, or indeed thinner regions of material if the material of the target surface is not completely opaque. A target surface, e.g. packaging, may be scanned from the front and / or back faces. The features will look different - e.g. embossings becoming debossings - but either or both sides may be scanned and analysed.

[0060] The feedback visualisation apparatus may comprise a low-angle light located closer to the target surface than the lens of the or each projector and directed across the target surface. The low-angle light may be used to highlight raised features, such as creases. Low-angle lights of different colours may be arranged along different sides of the target surface to provide a visual normal map.

[0061] The at least one result may be an output provided by a feedback module as described above with respect to earlier aspects. The output may be based on comparing a depth map to a template. The shape comparison may be or comprise comparison of a depth-map obtained by scanning the target surface to a template for the target surface, for example as described with respect to earlier aspects.

[0062] The adjustment of the projection image may be performed by processing circuitry of the visualisation apparatus, or in advance of the projection image being provided to the visualisation apparatus such that the visualisation apparatus may only store the adjusted projection image.

[0063] The target surface may be as described above. For example, the target surface may be substantially flat such that the target surface has an area, A, in a first plane with a length and width each at least fifty times (and optionally at least one hundred times) its maximum depth perpendicular to that area. The target surface may have an area of at least 0.15 m2, or 0.25 m2, and optionally of at least 0.5 m2. The output may provide micron-level accuracy in the shape comparison.

[0064] Mapping the projection image to the target surface may comprise using the results of at least two calibration mappings at different known heights relative to the bed. The adjustment of the projection image to the 3D position of the target surface may be an adjustment based on simply bed shape and location relative to the projector and an average thickness of the item of which the target surface is a surface. In other examples, the adjustment of the projection image may be an adjustment based on a depth of the target surface in each specific area, for example using knowledge of material thickness (for average surface height relative to the bed) and also depth-map information of the scanned target surface (for local variations in surface height). A depth-map may be obtained as described above. The same depth-map may be used for the shape comparison and for the projection image adjustment.

[0065] A sequence of images may be projected, so providing a moving graphic.

[0066] The at least one result of the shape comparison may include an identification of a feature of the template which is missing from the target surface. The projector may be arranged to project an image of the missing feature onto the target surface in the intended location of the feature, and / or to highlight the feature as missing (e.g. with a specific colour). The at least one projector may be arranged to implement colourhighlighting of features or areas of the target surface to identify approved features or areas with a first colour (e.g. green) and errors or unexpected features with a second colour (e.g. red), the second colour being different from the first colour.

[0067] The feedback visualisation apparatus may further comprise - or be provided as part of - a lightbased scanner arranged to capture data from which a depth-map of the target surface is generated. The projector may be arranged to project the information onto the target surface whilst the target surface is held in a position in which it is scanned by the light-based scanner. The feedback visualisation apparatus may therefore be integrated with the structural scanning and analysis apparatus of the first aspect, and may provide the feedback module of the first aspect.

[0068] The target surface may be substantially flat, e.g. such that the target surface has an area, A, in a first plane with a length and width each at least fifty times its maximum depth perpendicular to that area. According to a ninth aspect of the invention, there is provided a method of visualising information regarding a target surface in situ, comprising: positioning the target surface on a bed; generating a projection image based on at least one result of an analysis of at least one 3D topographical attribute of the target surface; mapping the projection image to the target surface using the results of a calibration mapping to account for the 3D position of the target surface, so as to adjust the projection image to the target surface; and projecting the adjusted projection image onto the target surface using a projector in a known position relative to the target surface so as to visually display the at least one analysis result for the target surface in a location on the target surface corresponding to a location of the topographical attribute of the target surface.

[0069] The method may be performed using the apparatus of the eighth aspect. The projection image may be produced using data gathered by a structural scanning and analysis apparatus as described for the first aspect, and / or using a photometric stereo scanner of the sixth aspect.

[0070] Intuitive usability and operator user experience may therefore be greatly improved, communicating what is wrong with a target surface and where in a single glance at the illuminated target surface. The visualisation, or other output, may also provide instructions on how to fix the discrepancies on the die cutter or other manufacturing machine(s). Validation of embossing / debossing / brand marking / structural patterns / tactile markers, identification of manufacturing defects, dimensional measurements, and / or plate and tool validation / wear estimation may be provided.

[0071] It will be appreciated that features described with respect to one aspect of the invention may be used in conjunction with any other aspect of the invention, mutatis mutandis. The use of an optical scanning system as described herein in conjunction with algorithms to analyse light interactions with surface features to infer the shape of a sheet’s surface can therefore be utilised in a wide range of scenarios.

[0072] The invention will now be described by way of example only with reference to the following Figures:

[0073] Figure 1 shows one example of a structural scanning and analysis apparatus in line with the invention, comprising a light-based (optical) scanner;

[0074] Figure 2 is a schematic representation of key parts of such a structural scanning and analysis apparatus;

[0075] Figure 3 (Figures 3A and 3B) shows an example depth map of a target surface that has been generated using the apparatus of Figures 1 and / or 2, the example target surface comprising a plurality of shaped regions (blanks) each intended to form a box for packaging a product;

[0076] Figure 4A shows an example of a normal map of an individual blank for forming a box, extracted from a larger surface scan produced using a structural scanning and analysis apparatus as described herein;

[0077] Figure 4B shows a depth map corresponding to the normal map of Figure 4A;

[0078] Figure 5 shows a close-up of part of the normal map of Figure 4, illustrating the identification of structural features from a scan;

[0079] Figure 6 (Figures 6A and 6B) shows a close-up photograph of a part of a net intended to form the corner of a box (6A), alongside a normal map of the same portion overlaid with a template for that portion of the packaging (6B);

[0080] Figure 7 shows part of a depth map of a target surface with identified embossed features highlighted; Figure 8 shows data obtained for a single embossed feature, showing the shape and depth of the feature;

[0081] Figure 9 shows a depth map generated for a target surface comprising a plurality of blanks which are intended to be identical;

[0082] Figure 10 shows an example of scan results for a target surface comprising a plurality of blanks each comprising braille;

[0083] Figure 11 shows a depth map of part of a single blank, highlighting damaged / improperly formed braille;

[0084] Figure 12 shows an example of scan results for a target surface comprising a crease;

[0085] Figure 13 illustrates an example of the use of an apparatus as disclosed herein in refining and adjusting machine settings to obtain a more accurate desired product;

[0086] Figure 14 illustrates an example of the use of an apparatus as disclosed herein in automating machine setting configurations for producing a desired product;

[0087] Figure 15 illustrates a method according to various embodiments;

[0088] Figure 16 illustrates an example of a feedback visualisation apparatus;

[0089] Figure 17 shows use of a feedback visualisation apparatus as described herein in displaying the results of analysis of a target surface;

[0090] Figure 18 shows use of a feedback visualisation apparatus as described herein in highlighting missing features on a target surface;

[0091] Figure 19 shows another example of use of a feedback visualisation apparatus as described herein in highlighting features which are, and which are not, approved;

[0092] Figure 20 illustrates an example of a photometric scanner design according to the invention;

[0093] Figure 21 illustrates another example of a photometric scanner design according to the invention;

[0094] Figure 22 illustrates a further example of a photometric scanner design according to the invention, showing an approach to reducing specular effects;

[0095] Figure 23 is a photograph of a prototype scanner according to the invention;

[0096] Figure 24 shows another example of a structural scanning and analysis apparatus in line with the invention, comprising a light-based scanner with a different scanning bed arrangement from that shown in Figure 1 ; and

[0097] Figure 25 illustrates determination of mirror points for specular avoidance.

[0098] Figure 1 of the appended drawings provides a rendered view of one implementation of a structural scanning and analysis apparatus 1. In this implementation, the apparatus 1 comprises a box-like structure 2 arranged to support and enclose various components of the apparatus 1. A movable panel or door 3 - in this example formed from three slidable panels - can be used to cover and expose a scanning bed 4 therewithin. An article 9 to be scanned can be placed on the scanning bed 4, and the door 3 can then be closed to shield the article from light from outside of the apparatus 1 whilst the article 9 - and more specifically a target surface 10 of the article - is scanned. In other implementations, a drawer design may be used for loading the target surface 10, enabling the scanner to be fully enclosed, as shown in Figure 24.

[0099] In the embodiment shown in Figure 1, a screen 5 is provided on the apparatus 1. This screen provides a graphical user interface - GUI - which a user may use to activate / control the apparatus 1 (e.g. using a touch-screen interface, or one or more other user input devices such as a mouse or keyboard) and to view the results of the scan. It will be appreciated that this interface 5 is simply one example of a possible interface, and that many other options may be used in additional or alternative embodiments. For example, the apparatus 1 may (additionally or alternatively) be controlled remotely from a separate device, or by a manual control interface (e.g. one or more buttons) rather than a GUI. Similarly, the apparatus 1 may (additionally or alternatively) provide its outputs to another device for display, such as by using a WiFi or ethernet connection to send the outputs to a remote user device (e.g. a laptop computer) or directly to a machine for implementation (e.g. adjusting settings of a die-cutter). The outputs provided may therefore be human-readable and / or machine-readable.

[0100] The apparatus 1 comprises a light-based scanner 100. The scanner 100 may be described as a “3D” scanner, as it is arranged to capture data in three dimensions - length, width, and height. In many implementations, only one surface 10 of an article 9 is scanned, so the scanner 100 may be used to produce only a model of the surface structure of the article 9, not of the shape of the entire article. The scanner 100 is arranged to capture images of articles 10 positioned suitably with respect to the scanner 100 (e.g. located on a dedicated scanning bed 4). Figure 2 illustrates the internal workings of an apparatus 1 such as that shown in Figure 1. As the scanner 100 is light-based, the scanner 100 comprises one or more lights 102 and one or more photosensitive devices 104 (cameras) for capturing light reflected from the article 10 being scanned. The scanner 100 may also comprise one or more lenses, mirrors, and / or other components for controlling and directing the light. The one or more photosensitive devices 104 convert the information received into electronic data for further processing. The light-based scanner 100 (or of any other embodiment described herein) may use white light to illuminate the target surface. In some embodiments, the light-based scanner may not require the use of lasers, laser cameras and / or LIDAR. The light-based scanner of the present application may also perform scanning and analysis of the target surface without using structured light and / or without using stereo vision techniques requiring multiple viewpoints (e.g. the approach used herein can be done with a single camera in a single position, for example using different lighting between images).

[0101] In various embodiments including the embodiment being described, the scanner 100 is a photometric stereo scanner 100. Photometric stereo is a computer vision technique used to estimate the surface normals of a target object by observing that object under different lighting conditions (photometry). It uses the fact that the amount of light reflected by a surface is dependent on the orientation of that surface in relation to the light source and an observer (e.g. a camera). By measuring the amount of light reflected into a camera, the possible surface orientation for the region of the surface captured in each pixel is limited. Given enough light sources at different angles, the surface orientation may be constrained to a single orientation such that a complete normal map for the surface (i.e. a map including information of the normal to the surface at each point) can be generated. This normal map, together with information regarding the position of the target object (and / or other calibration information), can be used to generate a depth map showing the 3D structure of at least a target surface of the target object 10. A depth map is a specialised image that stores information about the distance or depth of each pixel from the camera / viewpoint. While it can be displayed as a 2D image, the pixel values represent the third dimension (depth) rather than colour information - depth maps are therefore often referred to as “2.5D”, as they provide information about the 3D structure without themselves being truly 3D. In addition to a depth map, further data, including normal maps, diffuse maps, albedo maps, and / or target masks, may also be generated and optionally presented to a user.

[0102] In embodiments in which the scanner 100 is a photometric stereo scanner 100, the one or more photosensitive devices 104 generally comprise at least one camera 104. In some embodiments, a single camera 104 may be used - optionally positioned centrally with respect to the article 10 to be scanned - each image taken by the single camera 104 may capture an entire target surface of the article 10 to be scanned. In other embodiments, multiple cameras 104 may be used, optionally with their images being tiled together to capture the entire target surface 10 of the article 9 to be scanned. The use of multiple cameras 104 may facilitate obtaining a higher resolution for the same target surface, and the cameras may be synchronised such that they each take a photograph at the same time. Multiple lights 102 are generally provided in the photometric stereo scanner 100, in different positions and / or at different angles (it would be appreciated that a single light could instead be moved between different known positions between photographs being taken, but that use of a plurality of lights is generally easier and cheaper). Different combinations of lights may be turned on and off between capturing photographs of the surface 10 so as to provide different illumination scenarios, changing the orientation of each small part of the surface in relation to the light source(s) between photographs by changing which light source(s) are used, and so providing sufficient data for surface normals to be determined.

[0103] Different forms of light-based scanners 100 may be used in other embodiments; for example, barbased scanners may be used in which the target surface is moved at a known velocity past a stationary light source (or vice versa when used for aspects not requiring a solid-state scanner). Any suitable scanner 100 known in the art may be used. A door 3 or other cover may be used to shield the scanning process from stray, unwanted or unpredictable, light. This may increase scan accuracy / reduce errors, and also improve safety and / or comfort by shielding users from bright lights. Such a cover 3 may not be necessary, or indeed provided, in all embodiments. Similarly, a drawer, web, feeder, or robot can used to position the target surface 10 under a scanner 100 in various implementations.

[0104] In the embodiments being described, the target surface is substantially flat. In particular, the scanned surface of the target object 10 has an area, A, in a first plane of which an average of the length and width (or the diameter, for a circular target surface) is much greater than a maximum depth of features of the surface perpendicular to that plane. For example, the length and width within the plane may each be at least fifty, or one hundred, times the maximum depth of any features of that surface perpendicular to that plane. In some embodiments, such as when the apparatus 100 described herein is arranged to scan packaging made of thin cardboard, the article 9 as a whole may be substantially flat - indeed, the depth of embossed features may be comparable to or greater than a material thickness of the article 9 in some embodiments. In embodiments in which the article 9 as a whole is substantially flat, the thickness of the material may be determined from the depth map (e.g. in the region of cut edges) as well as determining the height / depth of surface features. In other applications, or indeed with packaging made of thicker cardboard, the article 9 may have a thickness much greater than the depth of any features on its target surface 10 - it will be appreciated that the same principles can be applied to detailed surface examination of relatively small features on a flat surface of a relatively thick (not flat) article as well as to relatively flat articles.

[0105] The output from the scanner 100 - which may be a set of photographs / images and information on the lighting scenario for each image - is then processed. The processing may be performed locally (e.g. within a dedicated processor of the apparatus 1 or remotely (e.g. in “the cloud”) or may be split between one or more local and remote processing units 200. Irrespective of the nature and split of the processors 200 used, the following processing modules 200 may generally be provided.

[0106] A depth-map generation module 202 is arranged to receive data generated by the scanner 100 (e.g. the at least one image produced by the scanner 100, and generally also information including the lighting scenario for each image and information relating to a distance between the camera(s) and the scanned surface), and to generate a depth map of the target surface based on that information. A normal map may be generated as a first step towards generating the depth-map. In some embodiments, a local processing unit of the scanner 100 may produce a normal map, and the normal map (and optionally other information) may then be sent to a different processing unit (local or remote) for generation of a depth map. The generation of the depth-map from scanner data may therefore be split between entities - even a “module” 202 may therefore be distributed in some implementations; it will be appreciated that these modules are defined for ease of comprehension of the process rather than to define how computations are performed. Figure 3 shows an example of a generated depth map, with the darkness / lightness of each pixel indicating the associated height (lighter pixels indicate higher regions, in this example). Figure 3A shows the full sheet 10 (colour variation showing a gentle curvature of the sheet as a whole, curling up at the lower and upper edges in the orientation pictured, as well as shape information and creases), and Figure 3B shows a close-up of part of a single blank 11, allowing more detail (in particular, embossed braille 12) to be seen. In this example, the target surface is an upper surface of a set of unassembled boxes - i.e. a tiled set of shaped pieces of card each intended to form a box for packaging a product; these flat shaped pieces each intended to form a box are commonly referred to in the field as blanks (even though they may comprise printed text and / or other features, and so not be “blank” in the standard sense of the word - “blank” is therefore used as a term of art herein, and may be replaced by “net” or similar should the reader find “blank” unclear). Each blank includes fold lines / creases along which the blank is intended to be folded to form the box, and also sets of embossed dots to provide braille on the box, so allowing a visually-impaired user to read information on the box. Whilst some embossing may have a purely aesthetic purpose, braille is required to be legible and serves a technical purpose of communicating information. Similarly, the creases enable the blank to be reliably made into a box of the correct shape. Figure 4 shows a normal map (4A) and corresponding depth map (4B) of a blank for an individual box, which is similar to, but not the same as, the blanks shown in Figure 3. Embossed braille 12 can be seen to be present, as well as multiple fold lines, or creases, 13 to be used to assemble the blank into a box. Figure 5 shows two higher-resolution close-ups of portions of the blank of Figure 4, showing that the depth-map is accurate enough to distinguish between a crease 13 and a cut 14. As demonstrated by these figures, the scanner 100 is designed to allow high-precision depth-maps to be generated by the depth-map generation module 202. In particular, the depth-map is of high enough precision to allow very small surface features - with respect to the size of the surface scanned - to be measured accurately. For example, for a target surface with an area of around 0.15 m2to 4 m2, surface features may be measured to sub-millimetre level accuracy, e.g. to micron-level accuracy. Surface features on surfaces with areas on the order of square metres may have depths or widths on the order of only around 10-100 pm. Prior art techniques and apparatuses (such as 3D scanners arranged to scan in a design to be 3D printed) have not been designed for such a high level of surface precision relative to object dimensions, rendering them unsuitable for the scanning of relatively flat target surfaces 10 with surface features required to be correct within tight tolerances. The development of such a scanner 100 therefore unlocks many options in the realm of packaging quality control and beyond.

[0107] It will be appreciated that the scanner 100 may be scaled as appropriate for different target surface areas (A), and that more generally the depth map may be arranged to show depths of features of the target surface to an accuracy of at least ± JA / 1Q8. It will be appreciated that smaller errors mean a higher accuracy, and that this could equivalently be worded as the depth map being arranged to show features of the target surface with typical errors of no more than ± JA / loe. “Typical” errors is specified here as artefacts in data processing often result in a small number of outliers (e.g. singularities due to erroneous readings) - the presence of a small number of obvious outliers (which can be removed) should not detract from the depth measurements corresponding to the majority of pixels of the images being accurate to the level specified, and correspondingly the ability to measure the vast majority of features identified from the scan data to the level of accuracy specified.

[0108] In some embodiments, the generated depth-map is displayed in a human-readable form. In other embodiments, the generated depth-map is simply stored as pixel-by-pixel values for use in further computation, and may never be displayed - i.e. a visual map / image is not necessarily always generated.

[0109] The depth-map is then passed to an analysis module 204 for further processing. The analysis module 204 of this embodiment is a comparison module 204, and is arranged to compare the depth-map to a template for the target surface 10. The template may be stored in a memory associated with the comparison module 204, or may be sent to the comparison module 204 with the depth-map. Figure 6 illustrates an example of the comparison process, with Figure 6A showing a high-resolution photograph of a portion of a blank for a box for a whisky bottle with one crease 6 of particular interest, and Figure 6B showing an overlay of a portion of a template for that blank corresponding to the crease of interest 6 on a depth-map of that portion of the blank (in this example, one section of the crease appears to be at a different angle in the rendering, but this is simply due to a render perspective selected). The position within the plane of the crease 6 and its width may be judged with respect to the narrow lines of the template shown 6a, 6b, and its curvature out of the plane may also be compared to template information in some embodiments. Lines 6c are simply used to indicate a point in the image around which the image can be rotated in visualisation software in the example pictured. In alternative implementations, the analysis module 204 may not perform a comparison to a template / no template may be used. The analysis may comprise identifying surface features above a certain height, for example, so that feedback to answer specific questions can be provided. The implementations described below focus on the use of a template for quality control, but it will be appreciated that the concepts disclosed herein have much broader applicability.

[0110] The comparison in the embodiments being described comprises identifying at least one attribute or feature of the expected / desired target surface 10 as described by the template with a corresponding feature of the real / scanned target surface from the depth map. For example, the one or more features may be or comprise one or more of: cuts through the target article 9; creases (e.g. fold lines); embossed shapes (e.g. “dots” for braille, or artwork); edges; and smooth, flat areas of particular dimensions. Feature identification may be performed by any suitable method known in the art - for example, a parametric model of a feature may be prepared by a user with a specific parameter set for a specific feature type (e.g. for a crease: height, width, crease angles, extending over at least a minimum length) and the depth-map may be searched for matches, or a machine learning model / artificial intelligence may be used to “learn” what creases (or other selected features) look like based on a training data set of labelled depth maps and then to identify corresponding features in a new / ”unknown” depth map. In the embodiments currently being described, the template provides information on expected positions of features, so facilitating feature identification.

[0111] The comparison module 204 may assess whether or not one or more of the following properties match the template, and how any deviations compare to allowed variations: (a) presence (e.g. if a template feature is present, partially present, or missing completely from the article); (b) height (perpendicular to the plane of the surface); (c) width and length (within the plane of the surface); (d) shape, (e.g. including curvature out of the plane as well as dimensions within the plane); (e) position (e.g. location and / or orientation); and (f) the presence of an unexpected feature (e.g. one that should not be present at all / is not present in the template). Deviations from the template within certain bounds may be allowed, whereas deviations beyond certain bounds may be forbidden - for example triggering a quality control rejection and / or remedial action. For some features, whether or not one property exceeding a particular level is forbidden may depend on levels of other properties. For example, legibility of braille may be influenced by both dot height and dot shape, and a dot height below a preferred minimum may be accepted provided that dot shaping is reliable and distinctive.

[0112] In some embodiments, the depth information used by the comparison module 204 may be combined with non-depth information. The non-depth information may include surface brightness, or absence of surface, or a light transmission through the target surface.

[0113] In some embodiments, rather than a comparison to a template, an internal comparison of instances of the same feature or features in the target surface may be compared. For example, the analysis module may be arranged to search the image for a plurality of instances of a repeated pattern of features in the target surface. The repeated pattern of features may correspond to different instances of a carton in the target surface (e.g. different instances of blank 11 in Figure 3A). The analysis module may then compare the instances to one another, or to pre-defined tolerances (e.g. in a look up table), to determine if a desired specification is met. Searching the image to find repeated instances of a carton (or the like) may comprise searching for areas of graphics or text (or any artwork or other markings) in the image of the target surface. Additionally, or alternatively, search for repeated instances of a carton may comprise searching depth profile for repeated instances of a cut / crease pattern, or emboss / deboss pattern.

[0114] A feedback module 206 is therefore arranged to provide an output, the output being generated based on a result of the comparison (or other analysis) performed by the analysis module 204. The feedback module 206 receives one or more results of the analysis from the analysis module 204 and generates one or more suitable outputs based on that data. The outputs may be human-readable and / or machine-readable. The outputs are comparison results in the example being described.

[0115] In the embodiment shown in Figure 1, the apparatus 1 comprises a screen 5, and the feedback module 206 may provide outputs including a change of display on the screen 5 - e.g. one or more alerts may be presented identifying faults with the scanned article 10, or a quality control approval may be presented if there are no discrepancies from the template exceeding set tolerances. Additionally or alternatively, a more detailed human-readable report may be made available either on the screen 5 or at another device 300 (e.g. a computer of a user). The report may include recommendations for machine adjustments to correct the identified defects and bring the products more in line with the template. A user may then make suitable adjustments to the manufacturing process accordingly. In still further additional or alternative embodiments, the feedback module 206 may generate machine-readable outputs arranged to directly adjust manufacturing parameters when fed to a sheet processing machine (e.g. a die cutter or press for manufacturing cardboard packaging). These outputs may be automatically fed to the relevant sheet processing machine(s) 300 so as to allow for automated correction based on feedback from the scanning and analysis apparatus l .In some embodiments, the analysis module 204 and the feedback module 206 may be arranged to analyse groups of adjacent features or attributes in the target surface. The group of adjacent features and / or attributes may include at least two different types of feature / attribute (e.g. adjacent creases and cuts, or braille and flat areas). This may allow the feedback module to determine whether the presence of one feature has had an effect on another close by. For example, the pressure applied to form a cut in the target surface may have an effect on an adjacent crease. The pressure of the cut may therefore need to be adjusted to correct the properties of the crease. In another example, the relative distance between an artwork location and a cut location may be used to determine a cut-to-print correction. In some embodiments, the functionality in this paragraph may be performed by one or both of the analysis module and / or the feedback module 206. Any suitable device 300 may therefore be used to display and / or act on the outputs provided by the feedback module 206. The output-handling device 300 which receives the outputs(s) from the feedback module 206 may simply display the output(s) in a human-readable format, leaving further action to the discretion of the user, or may directly cause action to be taken in response. The output-handling device 300 may therefore be or comprise e.g. a computer with a monitor arranged to visually present results, and / or a die-cutter arranged to automatically make adjustments to manufacturing settings based on the results.

[0116] Figure 7 illustrates an example display 400 that may be presented by a computational device 300 based on the output from the feedback module 206. The target surface 10 pictured comprises a plurality of blanks 11, each with braille 12 formed thereon. The image 400 shows a depth-map of the target surface 10 with additional information marked thereon. In particular, in this example each braille dot is represented by a coloured peak, with the height and colour of the peak representing the height of the braille dot 12a - the colour coding may be from red (below minimum height threshold) to green (in target height range) to blue (above target height range) so that issues can be quickly identified and corrected for. Providing feedback on the level of individual dots may allow for targeted corrections to be made. As the image 400 includes depth information, the imaged surface can be rendered at different angles - the imaged surface can therefore be rotated around a selected point (as indicated by the semi-circular arcs shown in Figure 7) to provide a user with a desired viewing angle.

[0117] Figure 8 shows a visualisation of individual braille dots 12a extracted from the depth-map, illustrating that the height (z axis), shape in the plane of the surface 10 (x and y axes), and curvature out of that plane, can all be determined and seen clearly. Whilst each braille dot 12a is a small part of the blank 11 on which it is formed, and an even smaller part of the target surface 10 scanned, the high accuracy of the apparatus 1 allows for analysis of its features. From the slight colour variation around the dot 12a it can be seen that, whilst the blank 11 is generally flat, there is a small amount of curvature resulting in a smooth and slow depth variation around the braille dot 12a. This local curvature is however much less than the height of the braille dot 12a. The highest point 8a of the / each braille dot 12a is then identified from the depth-map (ignoring any outlier-pixels likely to be the result of visual artefacts), and the height 8b of the braille dot 12a calculated as the vertical distance between that highest point 8a and the surrounding area of the target surface 10, i.e. a distance between the highest point 8a and a point 8c at the level of the braille dot’s surroundings, as identified from the depth map. The height of a braille dot 12a on packaging is often ideally around 200 pm.

[0118] Figure 9 shows another output visualisation 400 - in particular a colourised depth-map (colour lost in greyscale reproduction) of the whole target surface 10 shows that each individual blank 11 has been raised by stamping (by ~0.1 mm in this example), with the braille dots 12 being further raised above the general card surface. The colourised depth map therefore effectively provides a pressure map for the stamping process. The unusually dark colour on one blank I la indicates that this blank I la underwent too much pressure during the stamping process, being pushed further than it should have been and deforming the intended shape. A correction may be made to a press based on this feedback (for example by addition of one or more patching sheets). The darker colouration on the right-hand side of Figure 9, at the far end of the target surface 10, is a result of the sheet of blanks curling up slightly at the edge - it will be appreciated that this could be avoided by weighting or otherwise pinning edges of the sheet 10 if the sheet 10 has a natural curvature to it.

[0119] Figure 10 illustrates another example of a graphical display 400 presenting outputs from the feedback module 206. The generally white shape is a depth-map of the target surface 10 adjusted to show the edges and creases of each blank 11 clearly. Each blank has its own braille 12 embossed thereon - and in particular its own set 12 of braille dots 12a. A red, yellow, or green dot (colour lost in greyscale reproduction) is used to represent each braille dot, with the colour indicating the quality ('green-good, yellow-borderline. red-bad). Three sets of braille 12b are shown in red, and so would be rejected as inadequate. The other sets of braille are mainly green, with some yellow dots, and would be deemed approved. The table shown in Figure 10 below this visual representation provides further details of the analysis, including an automatic translation of the braille and a note as to whether or not this matches the others and / or the template, a position error of the braille 12 as compared to its expected location on the template, a percentage of dots in the braille which are shorter than a pre-set minimum height threshold (0.12 mm in this instance), and a note of a maximum height of any braille dot 12a in that set 12. Colour coding is used for each listed parameter to indicate its comparison to set thresholds. Figure 11 illustrates another example presentation 400 of data on braille taken from a depth map, again using colourised peaks for each dot 12a to compare its height (and optionally one or more other parameters) to set thresholds. The dots 12a lacking the dark colour at their tips are shorter than desired.

[0120] Listed parameters and / or thresholds for parameters may differ for different features (e.g. braille vs. creases) as well as between different implementations looking at the same features (e.g. depending on customer requirements and whether or not a template was used). For example, Figure 12 illustrates another example of a graphical display 400 presenting outputs from the feedback module 206, this time assessing a crease rather than braille. The background image 401 includes a plurality of marked creases 13, in this case all with the same orientation across the surface, each of which may be selected and more data 402 on that crease 13 displayed - a graph 402 showing the shape (height, width, and skewness) of the crease 13 at a selected point along the crease is shown in the example pictured, with corresponding graph lines for other points along the crease shown in grey alongside the black line of the selected point, to provide an indication of crease variation along its length. The table below this visual representation shown in Figure 12 provides further details of the analysis, including height, width, and skew of the selected crease. Colour coding is used for each listed parameter to indicate its comparison to set thresholds.

[0121] As indicated by the tabs provided in Figures 10 and 12, braille analysis views (“M” and “F” indicating “male” view of the braille - from above in the orientation shown; embossed dots - and “female” view of the braille - from above in the orientation shown; debossed dots), pressure analysis views, crease analysis views, and dimensional information, may be presented for the same depth-map, within the same software. The “Dimensions” tab may provide information on physical spacing between features - image rectification may be used to provide accurate measurements where the blank 11 (or other part of a target surface 10 being analysed) is at an angle to the camera or other scanner component responsible for collecting light. It will be appreciated that the choice of tabs may be made based on the article scanned and the intended usage of the outputs.

[0122] Turning back to the apparatus 1 used, it will be appreciated that the components 2, 100, 200, 300 described above may be arranged very differently between different embodiments - for example with all of the data processing 200 being performed by one or more processors within or physically connected to the scanner 100 in some embodiments, or being performed on one or more remote servers (e.g. using the cloud for data processing) in other embodiments. Similarly, the scan bed 4 may be physically integrated with the scanner 100, or may be e.g. a pallet or conveyer belt on which the article 9 to be scanned is resting, and above which the scanner 100 is placed. The pallet (or equivalent) 4 may be carefully positioned with respect to the scanner 100, or the scanner 100 may be moved into position with respect to the pallet. Further, whilst the scanner 100 is shown above the target surface 10 of the article 9 to be scanned in Figure 2, an underside of the article may be scanned in other embodiments and the relative positions may be adjusted as appropriate. Figures 13 and 14 demonstrate two possible process flows which may be performed using the apparatus 1 described herein.

[0123] In Figure 13, a job specification, or template, 50 for a desired product (e.g. packaging) is provided. The template 50 defines the 3D physical structure of at least a part of the desired article 9, and in particular of the target surface 10 of the article, and may itself be or comprise, or comprise information extracted from, a depth-map, optionally with specified tolerances for a set of features. This template 50 is then used to define a set of manufacturing instructions or settings - for making the product specified in the template 50 or tuning the machine that is producing the product. This instruction creation may be performed automatically by one or more computational processing units 200, optionally with contributions and / or checking from a user. In other embodiments, a user may define the set of instructions and provide that to the computational device 200 for handling, and optionally for conversion from human-readable instructions to machine-readable instructions. The set of instructions is then sent to a manufacturing device 300 (e.g. it may be downloaded to the device 300 from the cloud, or transferred by wire, or by a portable physical data transfer means such as a USB stick, or by human reading and data entry), which in the embodiment shown is a die cutter 300. Settings of the die cutter 300 are adjusted to match the instructions, and articles 9 are then produced following the instructions. These articles 9 - e.g. a sheet of blanks 11 for cartons, as described above - are then placed in a sheet scanner 100, and scanned as described above. The scan data are then provided to the one or more computational processing units 200 and compared to the template 50. Any errors / discrepancies between the scanned article surface 10 and the template 50 may therefore be identified, and the instructions may be updated to correct these errors. The updated instructions may then be sent to the manufacturing device 300 and used in subsequent production runs. The process may be iterative, with repeated updates to the instructions. Machine learning models / artificial intelligence (Al) may be used in this process, using feedback from previously-made adjustments to guide the Al to further refine and correct the instructions.

[0124] In embodiments in which the manufacturing device 300 is currently in use to make the products 10 in question when the updated instruction set is received, minor adjustments may be made to the current machine configuration rather than resetting all parameters. The updated instruction set may therefore be accompanied by a specific identification of the required changes to get from the old instructions to the new instructions. In some cases, only the specific identification of the required changes may be sent to the manufacturing device 300, such that the new instruction set is formed in situ in the manufacturing device. The updates may be automated, or may require user interaction. For example, on receipt of an updated instruction set with the update relating to a height error of 0.2 mm on certain creases the manufacturing device may display a message requesting an operator to add patching on the “red” / identified as incorrect creases for a 0.2 mm correction; or may automatically adjust the matrix used by 0.2 mm where required. “Patching” may simply involve adding tape or another patching sheet or material to an identified area of a press - manually or by means of an automated system. Similarly, if crease analysis identifies creases in a particular region of the target surface 10 as having a certain skew, plate angle of a pressing plate used to form the creases may be adjusted (such a plate may be provided as part of a die-cutter, or in a separate press).

[0125] The output of the comparison may therefore be a set of adjustments which can be sent back to a die cutter 300 (or other sheet processing machine / manufacturing machine). This feedback can be automated, and can therefore enable automatic, and so generally faster, fine-tuning. Example adjustments for a die cutter include: motorised counter position adjustment based on creasing angle; pressure adjustment based on creasing height and profile, guided patching; tool wear analysis over time / throughout a job to flag when a tool may need treatment or replacing. In implementation not using a template, a user may review the analysis outputs and decide on machine adjustments based on those outputs.

[0126] In some implementations, the one or more computational processing units 200 may be provided as a part of the manufacturing device 300, or as a part of the scanner 100. In some implementations, the one or more computational processing units 200, the manufacturing device 300, and the scanner 100 may all be integrated.

[0127] Figure 14 shows a process including generation of a template 50. In this implementation, a manufacturing device 300 (e.g. a die cutter) is used to make products 10. At least one of the products is then approved (generally by a human user or customer). That approved product is then scanned using a scanner 100 as described herein, and a depth-map is generated for use as, or for use in generating, the template 50. The template 50 comprises at least key 3D attributes extracted from the depth map. The template 50 generally also comprises information on tolerances / quality control targets. Tolerances may be set automatically based on pre-defined criteria, or input from the user or customer may be requested to set tolerances for features recorded in the depth map. The template 50 of this implementation is generated by scanning an approved target surface 10 with the light-based 3D scanner 100, generating a depth-map of the approved target surface using a depth-map generation module 202, and storing (all or part of) that depthmap and associated tolerances for use as the template 50. In some implementations, the depth map obtained from the scanner 100 may be simplified or corrected (e.g. for slight curvature of a target sheet 10) before storage as a template 50.

[0128] For packaging, it is common to scan many blanks 11 at once - the blanks 11 often being formed as a sheet 10 of blanks. The approved target surface used in template generation may be a whole sheet 10 of blanks 11 , or may be a single blank 11. If a single blank is approved in isolation, the approved target surface is likely to be smaller than a typical scanned target surface 10. In such cases, generation of the template 50 may comprise duplicating the depth-map of the approved target surface in a pattern to cover at least a majority of the typical target surface area - e.g. the template blank may be tiled in an expected pattern for the full sheet of blanks. Information from the user or customer may be requested as to the expected pattern of blanks 11 to form sheets 10. Alternatively, the comparison module 204 may be arranged to compare the (small, single-blank) template 50 to each of a plurality of regions of the target surface area 10 that is scanned. The matching of regions may be performed automatically based on feature recognition, and / or information from the user or customer may be requested as to the expected pattern of blanks 11 to form sheets 10.

[0129] Once produced, the template 50 is then used to produce a set of instructions for forming the product 10, the instructions including machine settings suitable for producing the product 10. This instruction set can then be provided to a manufacturing device 300 - which may or may not be the same manufacturing device as that used to make the original approved product - and used to control settings of that device 300 in making the product. In various implementations, the manufacturing device 300 can be automatically configured based on the instructions received, without requiring a user to do any set-up or adjustment. In other implementations, the user may be presented with human-readable set-up instructions to implement some or all of the instruction set on the manufacturing device 300. In implementations in which the machine set-up is done manually, the instruction set may be sent to a user device separate from the manufacturing device 300, and / or to a GUI of the manufacturing device 300. In other embodiments, rather than the template being derived from a scan of an approved target surface, an electronic or “virtual” template may be used. The electronic template may be provided to the comparison module in the form of a data file (e.g. a pdf). The electronic template may define various attributes of the target surface that are set during a design process.

[0130] Figure 15 illustrates a method / interlinked set of methods 1500 using the apparatus 1 as described herein in various ways - different sections 1510, 1520, 1530 of the overall method 1500 may be performed independently, optionally by different entities, and in some embodiments a single method section 1510, 1530 may be performed in isolation. The method 1500 comprises a section 1510 for generating a template 50, and optionally also an instruction set (as described above for the process shown in Figure 14). In this template generation method 1510, a target surface 10 of an approved (e.g. signed-off by a customer) product is scanned 1511 using a light-based 3D scanner 100. The product is selected such that its target surface 10 is at least substantially flat (having an area, A, in a first plane with a length and width each at least fifty times its maximum depth perpendicular to that area). A depth-map is then generated 1512 using the data obtained from the scanning step 1511, and optionally also other information relating to the scanner configuration and target surface position. The generation step 1512 is arranged to produce a high-accuracy depth-map relative to the size of the area, A, generally showing features of the target surface to an accuracy of at least

[0131] A template 50 is then generated from 1513 from the depth-map. In some implementations, the template 50 is or comprises the depth-map and tolerance information for features of the depth map (e.g. a range of permitted heights, positions, shapes, and / or widths of each 3D feature). Any outliers in the depthmap which are identified as errors may also be removed. In other implementations, the template 50 may be or comprise a simplified version of the depth map with features (e.g. edges, creases, cuts, and embossed portions) marked or highlighted, and associated shape information for each feature. In some implementations, the template 50 may be the final output of this method 1510. This template 50 may then be provided for use in assessing other products made to the same design, as described below in more detail for the structural scanning and analysis method 1530. In other implementations, an instruction set for manufacturing the products is also produced and implemented, as shown in Figure 13. This manufacturing method 1520 comprises creating 1521 an instruction set using the depth-map - either directly, or using the template 50 formed from the depth map, or both - and information regarding manufacturing processes (e.g. available machines 300 and their capabilities and settings). The manufacturing method 1520 then comprises providing 1522 the instructions to a manufacturing device 300 (and optionally providing some or all of the instructions to a plurality of manufacturing devices 300 forming parts of an assembly line). The manufacturing method 1520 then comprises producing 1523 products using the settings as defined in the instruction set. In some implementations, the product manufacture 1523 may be the final step in this method 1520. In other implementations, the method 1520 further comprises obtaining feedback on the products produced and updating 1524 the instruction set based on that feedback. A feedback loop of improvements to the instructions may therefore be provided. This feedback may be provided by the apparatus 1 described above.

[0132] In particular, a structural scanning and analysis method 1530 may be implemented using such an apparatus 1, the method 1530 comprising scanning 1531 a target surface 10 of a product using a light-based 3D scanner 100. As for the template creation method 1510, the product is selected such that its target surface 10 is at least substantially flat (having an area, A, in a first plane with a length and width each at least ten, fifty, or one hundred times its maximum depth perpendicular to that area). A depth-map is then generated 1532 using the data obtained from the scanning step 1531, and optionally also other information relating to the scanner configuration and target surface position. The depth-map generation step 1532 is arranged to produce a high-accuracy depth-map relative to the size of the area, A, generally showing features of the target surface to an accuracy of at least ± JA / 10B- The scanning 1531 and depth-map generation 1532 steps may be identical to those 1511, 1512 performed for template generation 1510. The methods 1510, 1530 may diverge thereafter.

[0133] The structural scanning and analysis method 1530 comprises comparing 1533 the depth-map generated in the depth-map generation step 1532 to a / the template 50. In the embodiment shown in Figure 15, the template is generated by scanning in an approved sample, as described above with respect to the template generation method 1510. However, in other embodiments the template 50 may be differently- generated, for example being manually created by a user as an idealised model of a product to be produced. The structural scanning and analysis method 1530 may be agnostic to how the template 50 is generated. The method 1530 simply requires that a template 50 be made available for use in the method 1530, by any suitable approach. The template 50 may be stored locally - e.g. in a memory of the scanner 100 or associated computational device 200 - or remotely, e.g. on a cloud server or remote device in communication with a comparison module 204 arranged to perform the comparison 1533. In still other implementations, no template 50 may be used and a different analysis step 1533 may replace the comparison step 1533 - e.g. identifying all creases and measuring their heights and skew angles. In implementations in which a template 50 is used, additional analysis not relative to the template may also be performed.

[0134] The structural scanning and analysis method 1530 then comprises outputting 1534 one or more results of the comparison 1533 (and / or other analysis 1533). The output is generated based on at least one result of the feature comparison step 1533. The outputting step 1534 may take one or more of a wide variety of different forms, ranging from providing a visual or audio output intelligible to a human user (e.g. a display on a screen, a change to the illumination settings of one or more status lights - e.g. green light(s) to represent Quality Control (QC) approval, red light(s) to represent QC failure, and / or an audible alert), to automatically causing a machine (e.g. a manufacturing device 300) to change its settings. In embodiments using an instruction set, the output may be used to update 1524 the instructions based on the output comparison result(s). The outputting step 1534 may be or comprise communicating an updated instruction set, or a proposed update to an instruction set, to a particular computational device or server. The device, computer- implemented code, or dedicated processing unit used to output the result of the comparison may be described as a feedback module 206, as it is a part of the overall apparatus 1 which serves to provide feedback regarding an analysis result for the target surface - e.g. the similarity of the scanned surface 10 to the template 50. The outputting step 1534 may therefore comprise any one or more suitable approach known in the art. Additionally or alternatively, the outputting step 1534 may be or comprise a tailored in situ 3D visualisation approach - the approach described below has been developed for the specific scenario of accurate feature analysis on a generally-flat surface.

[0135] In embodiments such as that shown in Figure 16, the feedback module 206 is arranged to project an output based on the comparison result(s) directly onto the scanned (“target”) surface 10. The feedback module 206 of these embodiments may be referred to as a feedback visualisation apparatus 206. This feedback visualisation apparatus 206 is arranged to project one or more colours, patterns and / or other images onto the target surface 10. The feedback visualisation apparatus 206 is arranged such that one or more features identified from the depth-map and / or the one or more generated outputs can be projected onto the target surface 10; the target surface 10 may therefore be a viewing surface. In other implementations, the image(s) may instead be projected onto a shaped surface of a manufacturing machine arranged to produce the target surface, e.g. a shaped press or die, optionally with one or more patching sheets. The manufacturing machine surface can therefore be used as an alternative viewing surface for displaying analysis results in situ. In some embodiments, the viewing surface may be the reverse side of a shaped surface of a manufacturing machine arranged to produce the target surface. The reverse side may be opposite to the side used to shape or cut the target surface. This may allow information to be projected onto the back of a diecutter used to make the target surface. This may allow packing material to be placed on the back of the diecutter to alter the pressure applied by the die in certain areas of the target surface.

[0136] In various embodiments, such as that shown in Figure 16, the feedback visualisation apparatus 206 comprises at least one projector 216, with the embodiment shown comprising three projectors 216. Standard cinematic projectors may be used in various embodiments, with Epson® cinematic projectors being used in the embodiment pictured. It will be appreciated that the number of projectors 216 to use, and their height(s) above the target surface 10, in a particular implementation can be decided based on projector properties, the area, A, of the target surface 10 and the required resolution (e.g. considering pixels per mm of target surface). The projector(s) 216 are arranged to face the target surface 10, from above in the orientation pictured. A lens of each projector 216 is preferably arranged to lie at least substantially parallel to the target surface 10, to reduce a need for perspective corrections. The image projected onto the target surface 10 by the projector(s) 216 is referred to as a projection image. This projection image is based on at least one result of a shape comparison between a target surface 10 and a template 50 for the target surface in the implementation being described, although other analysis results for the target surface may be used to form the projection image in alternative or additional embodiments.

[0137] The feedback visualisation apparatus 206 of this implementation also comprises a bed 4 on which the target surface 10 is arranged to be positioned for viewing. Herein, reference numeral 4 is used for both the scanner bed 4 and the visualiser bed 4 - it will be appreciated that these beds 4 may be the same where the visualisation is performed with the target surface 10 still on or within the scanning apparatus 1, 100, but will be separate entities in embodiments in which the visualiser 206 is not integrated with the scanning apparatus 1, 100. In each case, the bed 4 may have markers (e.g. printed on the surface) to facilitate alignment of a sheet 10 for scanning and / or output visualisation, as applicable. The projector(s) 216 are located in a known position / in known positions relative to the bed 4, and are arranged to project the projection image onto the target surface 10 so as to visually highlight features such as differences between the template 50 and the target surface on the target surface itself. The projection image may be stored in a memory of, or in communication with, the visualiser 206. The projection image is adjusted based on a calibration mapping to account for the 3D position of the target surface 10 - in particular, based on a spacing between the projector lens(es) and the target surface 10 (z-direction) as described below. An angular position of the target surface 10 in the x-y plane may be set, e.g. using bed markers or a removable calibration target, or may be adjusted for.

[0138] In the embodiment shown in Figure 16, the feedback visualisation apparatus 206 also comprises a backlight 226 arranged to illuminate the target surface 10 from behind the target surface. In the embodiment shown, the bed 4 is transparent and the backlight 226 is located behind the bed 4. This backlight 226 may help to highlight edges and cut-out portions of the sheet 10.

[0139] The 3D shape of the target surface 10 may cause the projection image to look different from how the image would appear if projected onto a truly flat surface. As the target surfaces 10 described herein are substantially flat, this effect may generally be minimal for projectors 216 facing the target surface 10 directly (although e.g. curling-up edges, or over-pressed blanks I la, of the target surface may be corrected for by adjusting the projection image based on depth-map information in some embodiments) - however lights / projectors at lower angles to the target surface 10 may be used to take advantage of this knowledge of the target surface shape. In the embodiment shown in Figure 16, the feedback visualisation apparatus 206 also comprises a low-angle light 236 located closer to the target surface 10 than the lens of the or each projector 216. The low-angle light 236 is located to one side of the target surface 10, and directed across the target surface 10. In some embodiments, such as that shown in Figure 16, the low-angle light 236 (which may be located at the back or front of the scanner bed) is angled downwardly towards the surface 10, at a relatively low angle (e.g. 0°< 0< 45°, and optionally 5°< 0 < 20°) as compared to the approximate 90° angle of the projector(s) 216. The low-angle of the light allows raised features of the surface 10 to cast shadows on the side further from the light 236 whilst relatively brightly illuminating the side closer to the light 236, so highlighting raised features. In alternative embodiments, just one, or neither, of a backlight 226 and a low-angle light 236 may be provided. Multiple low-angle lights 236 may be provided in additional or alternative embodiments - for example, a low-angle light 236 of a different colour may be provided along each side of the target surface 10 and these may be used to show an approximated normal map as a visual aid (a lower angle may be selected for this - e.g. 0°< 0< 5°).

[0140] The feedback visualisation apparatus 206 may offer a variety of different feedback modes, such that a user can select which output(s) to present at a given time. For example, if multiple creases 13 have been identified in the sheet 10 of blanks 11 scanned, and three of these creases fall outside of the set tolerances (QC-failures), the feedback visualisation apparatus 206 may project red lines along the QC-failure creases, so allowing a user to immediately identify which creases 13 are at fault without having to compare the target surface 10 to an image of the same on a screen (for example).

[0141] In some embodiments, e.g. embodiments using a photometric stereo scanner 100 in which the target surface 10 remains stationary throughout scanning, the light-based 3D scanner 100 and the feedback visualisation apparatus 206 are integrated such that results of the analysis and comparison can be projected onto the target surface 10 whilst the target surface remains in the position in which it was scanned, at least with respect to the bed (the bed 4 is the same for both scanning and visualisation in such embodiments, although the bed may slide between different positions for scanning and visualisation, e.g. in drawer-type scanner designs). In embodiments in which the target surface 10 moves during the scanning process - e.g. using a bar scanner with the target surface 10 moving across it, rather than moving a scanner bar across the target surface - the target surface 10 may be retained in a known final position once the scan is complete, to facilitate precise projection of the output onto the target surface, with correct alignment. In other embodiments, the feedback visualisation apparatus 206 may be separate from the scanner 100 and the target surface 100 may be repositioned in, or in a known position relative to, the feedback visualisation apparatus 206 after scanning.

[0142] The feedback visualisation apparatus 206 generally comprises a memory arranged to store at least one result of a shape comparison between a target surface 10 and a template 50 for the target surface. The shape comparison is generally focused on one or more features extending out of (e.g. embossed features or upward creases) or into (e.g. debossed features, cuts, or downward creases) a plane of the target surface 10. The feedback visualisation apparatus 206 generally comprises a projector 216 arranged to project information based on the at least one comparison result onto the target surface 10. In this way, differences between the template 50 and the target surface 10 - and / or other analysis results - are highlighted visually on the target surface itself. The projected information may simply colour-code features of the surface 10 based on their Quality Control (QC) rating (e.g. with at least two different colours, one for QC-passes and one for QC-failures, and optionally more colours to provide a spectrum between the two extremes), and / or may show images of intended features (features of the template) in regions where those features are missing or distorted, generally with a colour-coding or other marking to emphasise that those features are missing. A marking may be a known symbol or projected lettering to form a label, e.g. “MISSING CREASE”. Similarly, labels with explanatory details may be provided alongside QC-failure rated regions or features, e.g. “TOO SHALLOW” or “ADD PRESSURE”. These labels may be automatically generated based on preset options and the results of the comparison to the template 50. Colours may also be used to indicate departures from an expected depth - e.g. using green light for regions of the correct depth, going through yellow-orange-red for regions that are too high (depending on the magnitude of discrepancy) and through blue-purple-dark / ”black” for regions that are too low.

[0143] One or more projectors 216 are arranged such that an image is cast onto the sheet 10 (e.g. from above). Distance between the projector(s) 216 and the target surface 10 must be accounted for in order for the projection image to lie correctly on the surface, with each projected pixel in the desired position on the target surface. The target surface 10 is generally selected such that it is substantially flat - i.e. the target surface has an area, A, in a first plane with an average dimension much less than the depth of surface features perpendicular to that plane. For example, a length and width of the target surface may each be at least fifty times, and optionally one hundred times, the maximum depth perpendicular to that area. Nonetheless, adjustment for 3D surface features may also be made to allow the scanning output to be shown accurately on the target surface. The projection is therefore calibrated to the physical target 10, to ensure that each pixel maps to distance measurements correctly - often in a non-linear way (accounting for surface warp, lens distortion, projector angle relative to the scanner bed or other bed 4 supporting the target surface 10 on which the visualisation is to be projected, etc). The image to be projected (“projection image”) is mapped to the 3D surface and adjusted / distorted as appropriate such that it has the desired appearance when projected onto the 3D surface by the feedback visualisation apparatus 206. Non-linear (non-affine) mapping is nontrivial where (a) the target surface 10 is not flat (e.g. smooth but angled or curved), and (b) where the target surface 10 has height variations of multiple depths. As part of this mapping, a calibration may therefore be performed for the projector(s) 216 and support bed 4 to be used, and an expected thickness of the target object 10 (and optionally also depth variation of the target surface 10 - this may be limited to sheet curvature and / or over-pressing, or may also take account of surface features such as embossings / debossings / creases). In the embodiments being described a "projection surface" is defined as a grid of connected vertices (e.g. using a spline model or similar). At least two calibration mappings are obtained to allow for interpolation between the two levels and potentially also extrapolation beyond those levels. To perform this calibration, a grid of known dimensions is placed on the support bed 4 with its upper surface at a first known height above the bed 4. The grid may be printed onto paper or another - ideally dimensionally stable - substrate, with the thickness of the substrate being selected to provide the first known height. In some embodiments, the grid may be permanently printed onto the bed 4, or otherwise formed as part of the bed surface, and the first known height may be zero or near-zero. The substrate may be selected to be flexible so as to conform at least in part to non-flatness of the bed 4 (e.g. picking up curvature or a change in angle, or any significant protrusions on the bed surface - the substrate should follow the general, global shape of the bed surface to better represent the expected behaviour of the intended targets, e.g. following low-frequency curvature such as gentle changes in angle). The physical grid may be designed to match the projection surface, or at least to have a marked calibration target in the location of each vertex of the projection surface. The projection surface is projected onto the physical grid and the vertices of the projection surface are moved (manually, or by image recognition software) to align with the marked calibration targets. Each projected vertex of the image is therefore moved to its corresponding physical position, and this updated mapping is saved to memory of, or accessible by, a computational device (which may or may not be a part of the visualiser 206). This allows the saved / stored projection surface to be distorted as appropriate to match the physical space. The same process, with the same grid, is then repeated at least once more, at a different known height. The two heights are selected to be (i) at or near a lowest expected point on the target object 10 (which may be height-zero for a template 50 including edges / cut-outs / gaps, and (ii) at or near a highest expected point on the target object 10. This may be determined by reference to knowledge of target thickness (e.g. cardboard thickness), and optionally also considering one or more of likely target sheet curvature (e.g. curling up at edges), and knowledge of standard forming processes (e.g. cutting, which would reduce target height to zero, embossing, which increases target height above the target thickness, and debossing, which decreases target height below the target thickness). Specific template information for a specific target article surface 10 may be used in some embodiments. However, in tests performed, card thickness information was found to be sufficient in many embodiments, with the variation due to embossings / debossings and creases generally being small relative to card thickness, and so within the range of extrapolation with acceptable accuracy - other information may therefore not be considered unless higher accuracy is desired. For example, for cardboard packaging with a thickness of around 2 mm (e.g. between 0.2 mm and 4 mm), heights of 0.2 mm and 3.2 mm may be selected for the two saved mappings. Software is then used to interpolate (generally linearly) between the two mappings (or more mappings, if further calibration mappings are performed) to project the pixels in the right place on a given target surface 10, for a given surface height (which may be determined from the depth-map). The same two (or more) calibration heights may be used for a variety of card thicknesses / different sheet thicknesses, with extrapolation and interpolation being performed as required.

[0144] The projection image to be displayed to a user can therefore be distorted based on the depth-map information and known correspondence of depth-map pixels to projection pixels so as to correctly display the information on the not completely flat surface. The result of this process is a "projection mapping" variant that is able to deal with surfaces of different 3D sizes and shapes, using GPU rendering across multiple outputs. The number of nodes / vertices used may be adjusted based on the scan area and the required visualisation accuracy.

[0145] The projected output is designed to be intuitive - clearly indicating information at a glance that would take time, effort, and careful measurement for a user to obtain otherwise. For example, green / red highlighting for pass / fail of highlighted features, with optionally more subtle variations of colour (e.g. greenish-yellow through to orange for a spectrum between pass and fail), or more discrete colours (e.g. a single amber colour for borderline features, and bright white or purple illumination for missing features).

[0146] Figure 17 shows an example of an output relating to braille quality being projected onto the target surface. Blue / green light shows QC-approved dots, pink / red light shows QC-failed dots, and yellow light shows borderline cases (colour information lost in greyscale transformation). At a glance, a human user can therefore identify particular blanks 11, or regions of the sheet 10, where braille reproduction is weaker. The pointing finger shown in Figure 17 indicates the worst-reproduced braille, with most dots being marked yellow or red, whereas other dots are mostly green. In some implementations, no third colour may be used for borderline cases, and all dots may be classed as either QC-passed or QC-failed. Figure 18 shows a similar example where, in addition to red light for QC-failed dots, green light for QC-passed dots, and yellow light for borderline cases, white light - in particular, bright white circles - are used to indicate locations where dots present in the template 50 are completely missing from the scanned surface 10. The white light circles are arranged around the positions where the missing dots should be. In this embodiment, all of the lights are presented as circles encircling the (intended or actual) location of a braille dot 12a. In this way, the projected output is larger than the individual braille dot 12a, and so more visible, whilst still accurately identifying the dot to which it refers. Projecting light onto the peak of the dot 12a only is an alternative option, but may be less clear especially in implementations in which there is skew of the peaks.

[0147] Figure 19 illustrates use of the in situ visualiser for presenting feedback relating to creases 13 on the target surface. Multiple creases 13 are marked by the projected image, with approved creases e.g. 13a, 13c highlighted in green, and non-approved creases, e.g. 13b, highlighted in red. “Tag” images 402a, 402b providing additional information on two of the creases 13a, 13b are also provided as part of the projection image. These “tags” 402a, 402b each include a graph showing the shape (height, width, and skewness) of the respective crease 13 at a selected point along the crease 13, the selected point being marked by an arrow associated with the tag 402a, 402b. Whilst the approved crease 13a has a symmetrical profile, the additional data presented 402 show that the non-approved crease 13b has a skewed profile, at least in the region indicated - this may be the reason for the non-approval of this crease. The approval may be based on comparison to a template 50 for the target surface 10, or simply based on crease analysis with basis, general rules for creases irrespective of an overall surface shape.

[0148] In addition to presenting comparison results, lights of the visualiser 206 may also be used to show a normal map on the target surface 10 by using two or more (low angle) lights of different colours illuminating the target surface from different angles. For example, RGB light may be used, with the red, blue, and green lights all shining onto the surface from different angles (e.g. red along an x-axis within the plane of the surface, green along a y-axis within the plane of the surface, blue along a z-axis perpendicular to the surface). In some implementations, a colour picture of this normal map may be captured and used to confirm the position / alignment of the target surface (e.g. by identification of distinctive peaks or troughs) prior to projecting comparison results onto the target surface. Further, in some embodiments, RGB lights as described here may also be used as part of the scanning process. Data for three lighting directions can be contained within a single image - one image could be used with R, G and B channels each containing image data from an independent lighting direction.

[0149] Whilst the methods as described herein are not limited to a particular scanner type, a photometric stereo scanner 100 was found to be well-suited to the large but relatively flat targets 10 for which the approaches described herein are optimised. Subject to careful parametric design of the photometric stereo scanner 100, this approach can be scaled to target surfaces of significantly different areas, providing high depth precision relative to the scanned area in all cases. In this scanner design, the scanner 100 comprises a plurality of lights 102 and at least one camera 104. The lights 102 and camera(s) 10 are fixed in position / stationary - the illumination conditions are varied by selectively turning lights on and off between capturing images.

[0150] The lights 102 used operate in the visible spectrum in the embodiments being described, although infrared (IR) lights, and others could be used in additional or alternative embodiments. Cameras 104 generally have lower quantum efficiency in parts of the spectrum outside of the visible range, so providing cost benefits to using the visible light range as specialist sensors are less likely to be needed. The lights 102 can be very bright, necessitating safety features to prevent eye damage in some embodiments - such safety features may include controls preventing the lights from turning on when there is a direct path between the light and a human eye (e.g. when a cover or shutter is open - this may be referred to as an interlock), and limiting the maximum duration of light activation to a certain time period, e.g. a fraction of a second. This time limitation may serve two purposes: (1) preventing eye damage in case of interlock failure, and (2) preventing damage to the polarisers or the lights themselves due to thermal build-up.

[0151] A plurality of lights 102 are mounted so as to illuminate the target surface 10 from above (in the orientation shown in e.g. Figure 20 - it will be appreciated that this may be reversed in other embodiments). These lights 102 are not placed directly above the target surface 10, but rather offset from it and angled towards it. The lights 102 are angled such that a centre of each beam hits the scanner bed 4 outside of the area of the target surface 10, so reducing specular effects (as discussed below). The lights 102 are mounted at angles between 30° and 80° degrees relative to the target surface 10 in the embodiment being described.

[0152] A backlight 102z is also used in the embodiments being described to illuminate the surface 10 from below. This allows edges of the article 9 to be clearly identified, so that the shape of the target surface 10 can be “cut out” in the image obtained (i.e. creating a mask, separating the article 9 from the background). Whilst other methods of background removal may be used in other embodiments, backlighting is preferred because it is fast to image and helps to keep the scan time short, as well as keeping processing requirements lower than otherwise, so decreasing time required for analysis. An adaptive filter may be used to create the target surface mask, based on information obtained from one or more images taken with the backlight 102z illuminated.

[0153] The photometric stereo scanner 100 of various embodiments is as shown in Figure 20, with a number, n, of lights 102a. ..n arranged above a target surface 10 and around a camera 104. A single camera 104 located centrally with respect to the target surface 10 and with respect to the lights 102 is used in this embodiment. In various embodiments, the number of lights, n, may be between 5 and 20, and optionally between 8 and 16, to balance illumination quality with cost and complexity.

[0154] The camera 104 comprises a lens arranged to facilitate acquisition of in-focus images of the target surface 10. A high-quality lens with minimal distortion may generally be preferred to increase the scan accuracy. Unlike previous scanners using telecentric lenses, standard perspective lenses are generally used to provide a view of a wider area. The lens' focal length, and corresponding sensor size determine the target scanning area relative to the camera mounting location.

[0155] The system 100 shown in Figure 20 is suitable for scanning carton sheets 10, capturing the whole area 10 without moving the article 9. A single camera 104 is placed centrally and directly facing the target surface 10 to achieve maximum resolution. In this embodiment, the camera 104 captures images in monochrome to prioritise image sharpness, but colour cameras may be used in additional or alternative embodiments. The system 100 can utilise either a global shutter or a rolling shutter as there is not usually movement in the scene relative to the amount of time for which the lights 102 are activated. The lens is focused, and the aperture of the camera 104 is set to maintain the targets of interest (i.e. the surface of the 106-sized carton sheet, in this example), within the lens's focal plane. The term “106-sized” is taken from an industry-standard naming scheme (including 102-sized, 104-sized, 106-sized, 145-sized, etc.), and a “106-sized machine” denotes a machine able to take a sheet of maximum dimensions 1060 mm x 760 mm. By contrast, a “145-sized” machine can take a sheet of maximum dimensions of 1450 mm x 1050 mm. A scanner 1 as described herein sized for the 106-sized sheets was shown to successfully identify small surface features with a precision of ~30 microns. Systems 100 may be sized for various standard sized sheets - e.g. 106-sized sheets (min. 400 mm by 350 mm, max. 1060 mm by 760 mm), 145-sized sheets (min. 580 mm by 450 mm, max. 1450 mm by 1050 mm), 165-sized sheets (min. 600 mm by 520 mm, max. 1650 mm by 1300 mm), 170-sized sheets (min. 730 mm by 520 mm, max. 1700 mm by 1300 mm). Sheet sizes of 317 mm by 470 mm, 630 mm by 469 mm, 915 mm by 635 mm, 1270 mm by 916 mm, and 1778 mm by 1219 mm may also be used. To manage a larger scan area 10 at the same resolution, multiple (potentially cheaper) cameras and lenses may be configured in an array, as shown in Figure 21. The output of the multiple cameras 104a, 104b may be “stitched” or tiled together. The overlap between the camera visible areas - as shown by the cones (in 3D) / triangles (in the representative image) in Figure 21. The same lighting configuration may be used as for a single camera 104 in some implementations, but scan quality for a different camera position may benefit from the use of additional lights 102 in selected positions relative to each camera 104. The apparatus 100 can therefore be thought of as modular, and offers scalability of design: additional cameras 104 and / or lights 102 may be introduced to do one or more of (i) increasing the size of the area scanned, or (ii) enhancing the level of detail captured (resolution / accuracy). The geometry of the scanner 100 is also therefore adjustable: the scanner's size can be scaled up or down, and the shape of the region covered can be adjusted by changing the relative positions of lights and cameras (e.g. from a more square shape, to a more elongate rectangular shape, depending on the article surface 10 to be scanned) - different target dimensions may therefore be accommodated. In some embodiments, multiple modules each comprising at least one camera 104 and a plurality of lights 102 in fixed positions relative to each other and to that camera 104 may be provided, and these modules may be movable with respect to each other (on installation, and optionally also in use - moving parts may therefore be used in some embodiments to allow reconfiguration of the same apparatus 100 for use in scanning different target geometries). In various embodiments, including those pictured in Figures 20 and 21, the apparatus 100 is designed such that there are no moving parts - the camera and lens positions are pre-set on assembly, and changes in use are not required. Cost and complexity may therefore be reduced as compared to system designs requiring moving parts, and reliability and durability may be correspondingly increased.

[0156] A lighting arrangement 102a,... 102n above the target surface 10 illuminates the target surface - lights 102 may be turned on and off in sequence. Photographs may be captured with each light enabled in turn, or with particular combinations of lights enabled in turn. For a target surface 10 containing gaps / holes, a backlight 102z may be included so that an image can be captured that distinguishes between a target 10 and the background. In addition, a “two-sided” system to enable capture of both the top and bottom surfaces of a target 10 simultaneously has also been developed. A transparent surface - e.g. of acrylic or glass - can be used as a support for the article 9 to be scanned, allowing both sides 10 of an article such as a cardboard sheet to be scanned at the same time. An array of lights 102a...n and one or more cameras 104 may be provided both above and below the target surface 10 in such embodiments. The arrays 102 may be symmetrical, and camera position may also be symmetrical. In one implementation made for performance testing, a camera 104 was placed centrally (x= 0, y= 0) with respect to a plane of the target surface, 1300 mm above the target surface (z=1300). Eight lights 102a..h were then provided, all at the same height (z= 1178), but at different (x,y) positions: 102a at (0, 554), 102b at (-680, 528), 102c at (-709, 0), 102d at (-680, -528), 102e at (0, -554), 102f at (680, -528), 102g at (709, 1178), and 102f at (680, 528). Light (x,y) positions may vary in other implementations, and may not be symmetrical. In other implementations, light heights may also vary. The camera 104 was a 12 bit, Mono, 8192 x 5460 px, with a lens with a 28 mm focal length. The lights 102 were custom-built 200 W LED lights (white light), with custom control circuitry and a crosspolarised configuration. A 16-light version was also tested, adding eight further lights 102 to the arrangement described - the presence of more lights may allow for more even lighting, so enabling processors to reduce the impacts of shadows and speculars. Blacked-out internal panels were used to avoid stray light and reduce internal reflections.

[0157] Scanning surfaces with a specular response component, such as the glossy, coated side of a cardboard sheet, poses challenges because specular highlights and cast shadows are visible; these artefacts are classed as outliers - at least when assuming a Lambertian reflectance and local illumination model, as is often done in processing images provided by such a scanner 100 - this can lead to inaccurate estimates of surface normal, which in turn can lead to depth map errors. The scanners 100 of various embodiments make use of cross-polarized lighting to reduce unwanted specular reflection effects from glossy / shiny surfaces. Specular reflection is a type of surface reflectance in which incident light is reflected into a single outgoing direction, and is often described as a mirror-like reflection. This can lead to a single, very bright, spot on the target surface 10 (or one per light, when multiple lights are on at once), which may occlude structural features of the scanned surface. To reduce the occurrence or severity of such effects, the apparatus 100 being described comprises polarising filters - one on each light source 102 and one on the or each camera lens 104 - to reduce or eliminate unwanted specular responses from the target surface 10 when acquiring the images required for surface construction. In optics, the use of cross-polarisation is known to reduce glare. The light ray, initially unpolarised, passes through the first linear polariser (on or near the light source 102), which allows only light waves oscillating in a specific plane to pass through, while blocking the rest. The polarised light then hits the target surface 10. If the surface 10 is smooth and reflective, it will reflect the light while maintaining its polarisation state. However, if the surface is rough or scatters the light (matte or textured) the reflected light will become partially or fully depolarised. The reflected light then travels back up towards the camera 104. Before reaching the camera sensor, the light passes through a second linear polariser oriented perpendicular to the first one. This second polariser therefore blocks the polarised light that was directly reflected from smooth surfaces, as its polarisation state is still aligned with the first polariser. However, the depolarised light from rough or scattering surfaces will be able to pass through the second polariser to some extent. The use of the two filters provides a cleaner image, without the unwanted bright spot of a specular reflection.

[0158] Ideally, each light 102 should be relatively small (so allowing for the use of multiple lights in precise positions whilst maintaining a reasonably compact apparatus size) and also sufficiently powerful for the intended scanning purposes. For a prototype scanner with a l m2scan area, tailor-made LED packages were designed for use as lights 102 of the scanner 100, with each light 102 providing around 20,000 lumen (200 W). A linear polariser is attached to the front of each light 102 - the linear polariser can be turned, manually or automatically. Each light 102 is equipped with an onboard PCB control system arranged to activate the light upon receiving the appropriate signal (e.g. from a computer or other controller forming a part of the scanner 100). This control system may also be arranged to ensure that the light 102 is automatically turned off after a specific period to prevent damage to the polarising filter due to prolonged exposure to high-energy emissions. In various embodiments, a diffuser is added to each light 102 to reduce the impact of any “hot-spot” from the LED.

[0159] Use of one or more polarisers to remove the impact of specular reflections due to e.g. gloss varnishes also allows the system 100 to be tuned to do the opposite - i.e. specifically look at the glossy, metallic, or otherwise highly-reflective features of a surface 10. This tuning can be done by adapting the polarisation scheme used on the lights 102 and camera(s) 104 to isolate the feature of interest. For example, use of a circular polariser in combination with a linear polariser was shown to be usable to reveal gloss varnish printing over a target surface 10. In some embodiments, other diffusers and / or more complex polarising stacks can therefore be added to some or all of the lights 102; this may allow the system to be tuned to specific surface types and reflectance characteristics, e.g. detection of anisotropic areas (e.g. holograms).

[0160] The specular robustness of the apparatus 100 can also be improved in other ways, as illustrated in Figure 22. In embodiments such as that shown in Figure 22, the lights 102 are positioned such that the specular reflection falls outside of the target scanning area 10, albeit at the cost of increased system size and / or reduced light information in the scene. Figure 22 shows a parametric geometry - with angles and distances between the lights 102, the target surface 10 and scanner bed 4, and the camera 104 marked. Desired light positions can be generated relative to a camera 104 and the target surface 10. The generated light positions can produce specular reflections over the target surface 10, or (as pictured here) off the target surface 10 at the expense of a larger physical geometry (i.e. more widely-spaced lights relative to the target area size). The design space comprises target scan size (area A); camera and lens properties; light number and positions; and illumination characteristics. The parametric system design process therefore considers factors including the following in deciding apparatus geometry: the target surface area / area visible to the camera 104; the camera working distance to the target surface 10; the luminous area of each light 102; and the desired height of each light 102 above the target surface 10 (to get the best exposure). In various embodiments, different lights 102 may be placed at different heights.

[0161] The luminous area of a light 102 includes both the area of one or more light sources providing the light, such as LEDs, and the area of any reflectors built into the light - for example purpose-built reflectors, or incidentally shiny surfaces such as heads of metal screws.

[0162] For a flat plane 10 facing a camera 104, the bright spots of specular effects generally appear at “mirror points” where an angle, 0, between a point light source and the plane is equal to an angle, 0, between a centre of the camera lens and the plane, as marked in Figure 25. This can be determined using basic trigonometry and knowledge of the positions of the light 102 and camera 104 with respect to the target surface 10. In reality, most lights are not point light sources, so instead produce a “mirror area” comprising these mirror points for each position across the luminous area of the light 102. A closest mirror point to a boundary of the target surface 10 can therefore be determined for each light-camera pair (it will be appreciated that the full mirror area may be determined and checked in some implementations, but that geometrical reasoning may be used to avoid the necessity of calculating the full area). The light 102 can then be positioned so as to ensure that its closest mirror point is outside of the boundary of the target surface 10 to be scanned, such that all mirror points (and therefore the complete mirror area) fall outside of the boundary. The same process can be performed for each light 102, and also for each light-camera pair where there are multiple cameras 104. Figure 25 illustrates this process for the example of lights 102 with a circular luminous area. Surface roughness or other 3D surface features may result in specular reflections outside of the mirror area, but the process described above was found sufficient for most substantially flat target surfaces as described herein. A margin may be defined around the target surface boundary - as opposed to allowing the mirror area to be directly adjacent to the target surface boundary - in some implementations.

[0163] In various embodiments, lights 102 may be placed facing different directions, e.g. one or more lights 102 may be placed such that their light is directed straight across the target surface 10. Light angle, as well as or instead of light location, may be adjusted to move the specular reflections in some implementations.

[0164] If a system geometry that places the specular reflection locations from the lights 102 off the sheet 10 is impractical (e.g. too big, and / or unsafe bright lights nearer to user’s lines of sight), then the performance of polarisers (polarizing lenses of the lights 102 and camera(s) 104) may be balanced with light characteristics to at least partially extinguish the specular reflections. One or more of the following can be considered in the design process: polariser extinction ratio, light power and polariser thermal damage, light position / direction relative to scanning surface 10 (and evenness of illumination), and in some cases, adding a diffuser to the light. In some embodiments, complete extinction of the specular reflections may be possible. In embodiments in which specular effects are not fully removed, the number of lights 102 used may be increased to counteract the outlier(s) generated by the specular highlight in the surface generation - more even, brighter, illumination may lessen the impact of the specular bright spot(s).

[0165] The parametric design space approach described herein therefore employs various strategies to reduce the impact of specular response components on surface reconstruction errors: (1) removing speculars from the target area by changing camera position relative to the target surface; (2) balancing illumination, polarisation, and other optics (e.g. diffusers) to find an extinction solution; (3) increasing the bit-depth of the camera / high dynamic range (HDR) imaging, and (4) adding more lights to increase the amount of information available to the solver. The relative importance of these factors guides the parametric design for a given target size, allowing a 3D scanning apparatus 100 of this type suitable for many different kinds and sizes of target 10 to be designed based on the same principles.

[0166] Additionally, outlier-tolerant solvers can be implemented in software - it will be appreciated that these are typically slower than solvers which are less tolerant to outliers, however. For instance, a solver may be arranged to determine if a specularity component exists in any of the images provided to it (for instance, by mapping the specular reflections a priori), and then, for pixels where this condition exists, to choose a surface gradient value that is tolerant to this outlier. Multiple different approaches are known for solving an appropriate system of linear equations; the outlier-tolerance varies between approaches. For example, one or more of the following approaches may be used: Least Squares (L2) Minimisation; Least Absolute Deviations (LI) Minimisation; RANSAC (Random Sample Consensus) of different light combinations; Robust PCA (Principal Component Analysis); Bayesian learning approaches; Specular-Aware Photometric Stereo Models; and / or Multi-View Photometric Stereo (in scanners 100 with multiple cameras). In addition, if one side of the target surface 10 is glossy and the other matte, a decision may be made to scan the matte side to reduce issues with speculars.

[0167] The apparatus 100 may therefore offer specular robustness: the system may maintain its performance even in the presence of intense specular reflections from light sources, ensuring reliable and consistent results. Multiple different approaches can be traded off in the parametric design of a particular system 100, bearing in mind size and cost considerations as well as data quality.

[0168] As discussed above, careful control of light and camera positions, amongst other considerations, may therefore avoid or at least minimise glare and unwanted reflections during the scanning process, so allowing for the collection of cleaner data. Reducing the specular reflections is one example of robustness to target surfaces (in particular, glossy / shiny surfaces vs. matte surfaces, for that example). The scanner 100 may be designed to balance the optimisation of lighting performance, such as ensuring even illumination across the scanning area, with maximising the information captured by the camera(s) 104, including bit depth. This balance enables the system 1 to handle a wide range of target surfaces effectively - including those with darker / more absorbative areas such as dark printed inks.

[0169] In general for a photometric stereo scanner 100 as described herein, at least 100, and optionally 150 or more photographs may be taken with the same camera position but different lighting conditions, as use of a solid-state scanner allows this to be done rapidly and conveniently. However, many fewer photographs may be taken in other implementations - with use of a single photograph alone being possible provided that colour information is used to provide at least three independent lighting directions within the same image. Some duplicate photographs (same camera position, same illumination scenario) may also be taken for errorchecking purposes. Once sufficient photographs have been acquired by the scanner 100 for a normal map (i.e. a map showing the normal to the surface at each point / each pixel of the photograph) with an approved accuracy / noise level to be generated, the scanning process may be deemed complete. Once the scanning process is complete, the data captured are analysed. Software is used to infer the shape of the target surface 10 by analysing how light interacts with different surface features (e.g. creases on a sheet of pre-formed boxes). Calibration information - e.g. based on knowledge of the distance between the camera 104 and the scanner bed 4 and the average thickness of the article 9 resting on the scanner bed - can be used in conjunction with the normal map to produce a depth map (i.e. a map showing the height / depth perpendicular at each point / each pixel of the photograph). It will be appreciated that the generated maps may never be visually presented, but instead simply stored as a set of data points such as: normal map: location (e.g. pixel reference / x,y coordinates), angle of surface normal (e.g. in degrees or radians); depth map: location (e.g. pixel reference / x,y coordinates), depth (e.g. in pm).

[0170] The scanning apparatus 100 of the examples being described comprises one or more connections to various output systems 5, 300, 216 that can share the data with human users and / or other machines. Figure 23 provides a photograph of a prototype apparatus 1 as described herein, in which the scanner 100 is a photometric stereo scanner 100 as described above, and is combined with a feedback visualisation apparatus 206 built into the same structure 2 as supports the scanner components. The feedback module 206 also provides one or more outputs for display on a screen 5 of a user computer 300. The computer 300 is wired to the scanner 100 and feedback visualisation apparatus 206 in the embodiment shown, but wireless communications may be used in other embodiments. The scanner bed 4 is used to receive the article 9 (a sheet of blanks 11 for packaging containers, in this example), with the article’s target surface 10 upwards. The scanner 100 and feedback visualisation apparatus 206 are both located above the scanner bed 4, facing the scanner bed. In other embodiments, the scanner 100 and / or feedback visualisation apparatus 206 may be located below the scanner bed 4 - the scanner bed may be transparent in such embodiments.

[0171] The data processing 200 (depth map generation and analysis, and preparation of outputs in appropriate formats) may be performed entirely by the computer 300, entirely remotely (e.g. in the cloud / on one or more remote servers), or may be split between local and remote processing.

[0172] Using the techniques described herein, this apparatus 1 was found to be capable of scanning larger areas quickly and with a high level of detail - for example, all images required for the analysis may be gathered within 20 seconds, and optionally within 10 seconds, 7 seconds, or 5 seconds (with precise times being adjusted as appropriate based on light intensity, camera properties, and / or desired de-noising approaches). The approach and apparatus 1 described herein was found to allow a relatively high level of detail to be acquired across a relatively large (semi-flat) scan area. Prototype scanners 1 were able to detect 10 micron-scale features with high precision across scanning areas of up to 1 square metre. Across a range of test systems and areas, an accuracy of at least ±0.05 mm was demonstrated, with precision of at least ±0.01 mm. Prior apparatuses for 3D scanning have generally focused on obtaining a full 3D view of a more compact object, rather than focusing on thin articles or simply surface analysis. This prejudice has led to modifications suitable for high-quality surface scanning being overlooked, which is remedied by the inventions described herein.

[0173] It will be appreciated that the embodiments described in detail herein are given by way of illustrative example only, and not intended to be limiting. The scope of the invention is to be limited only by the appended claims. For example, whilst the scanner 1 shown in Figures 1 and 23 may be described as having a “cupboard-type” arrangement with a large open volume covered by a door or shutter 3, the structure 2 may be quite different. As shown in Figure 24, the scanner 1 may have a more “drawer-type” arrangement with a relatively narrow (vertically) slot into which a target surface 10 for scanning can be inserted. The structure 2 may comprise a pull-out drawer including a scanner bed 4 and optionally also a cover for the slot (e.g. in the style of a front face for a drawer) or the item 10 may be slid into an open slot, and a cover optionally then pulled across. In such embodiments, the scanner bed 4 may be transparent (e.g. made of acrylic) and the lights 102 and camera 104 may be located below the scanner bed 4 (such that an underside of the item 9 is scanned / is the target surface 10) so making use of the volume which is below a user’s comfortable working height. A screen 5 is provided to display outputs to the user in the embodiment pictured, mounted on an upper surface of the scanning apparatus 1. In some embodiments, a flocking material may be applied to part or all of the internal surface of the scanner to reduce internal reflections.

[0174] Although the 3D topographical features of the target surface discussed primarily herein are braille or creases in the target surface, other 3D topographical features may be considered. For example, the 3D topographical features may include any one or more of:

[0175] (i) braille provided on the target surface, and optionally wherein an assessment of the legibility of the braille is made;

[0176] (ii) a crease on the target surface, and wherein optionally an assessment of the shape of the crease, a location of the crease, a tolerancing profile of the crease, a consistency of the crease or a foldability of the crease is made.

[0177] (iii) a feature created using a cutting knife, and optionally wherein an assessment of the impact and / or presence of a feature created using a cutting knife on the target surface is made;

[0178] (iv) an embossed or debossed feature, and optionally wherein an assessment of the presence or consistency of the embossed or debossed feature is made;

[0179] (v) a partial cut through the target surface (e.g. a cut part way through the thickness of the target surface), and optionally wherein an assessment of the presence of a partial cut compared to the reverse of the target surface is made (e.g. by performing scans of both sides of the target surface) and / or light transmission through the target surface; or

[0180] (vi) a burst or foil, and optionally wherein an assessment of the presence or absence of a burst or foil in the target surface is made (in packaging, a "burst" refers to the point at which a package fails due to excessive pressure or load, typically during a burst test but this may happen during manufacture on occasion. For example, a braille die might press through the other side of a carton / material, or a male creasing rule might not align well with a female counter tool and cause the material to break).

[0181] Further, in implementations of some aspects the scanner used may not be a photometric stereo scanner, and in implementations of other aspects, the scanner may be a photometric scanner but may not have the accuracy described for other embodiments. Similarly, the visualisation apparatus of some aspects may be used with a non-solid-state scanner (e.g. a moving bar scanner) instead of with a solid-state scanner as described herein.

Claims

CLAIMS1. A structural scanning and analysis apparatus comprising: a solid-state light-based scanner arranged to capture at least one image of a target surface, the target surface being at least substantially flat and having an area, A; a depth-map generation module arranged generate a depth map of the target surface using surface normal information derived from the at least one image, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy of at least ± / . / 108’ an analysis module configured to identify and measure at least one of the topographical attributes of the target surface using the depth map; and a feedback module arranged to provide an output, the output being generated based on a result from the analysis module.

2. The structural scanning and analysis apparatus of Claim 1, wherein the light-based scanner is a photometric stereo scanner comprising a plurality of lights and at least one camera, and wherein optionally the plurality of lights and the at least one camera are arranged such that specular reflections fall at least substantially outside the target surface area.

3. The structural scanning and analysis apparatus of Claim 1 or Claim 2, wherein the depth map shows 3D topographical attributes oftarget surface to an accuracy of at least ± / / 2xand optionally of at least4. The structural scanning and analysis apparatus of any preceding claim, wherein the target surface has an area of at least 0.15 m2, 0.2 m2, 0.5 m2, 0.8 m2, 1 m2, or 2 m2, and wherein the depth map has micron- level resolution.

5. The structural scanning and analysis apparatus of any preceding claim, wherein the analysis module is arranged to analyse the at least one image in addition to the depth map, and to provide an analysis result comprising visual information as well as depth information.

6. The structural scanning and analysis apparatus of any preceding claim, wherein the output from the feedback module comprises one or more of: a. data arranged to be used by a manufacturing machine to adjust its settings so as to modify an identified 3D topographical attribute of a surface formed by the manufacturing machine; and b. human-readable identification of errors or unexpected features in the target surface.

7. The structural scanning and analysis apparatus of any preceding claim, wherein the analysis module comprises a comparison module arranged to compare at least one 3D topographical attribute of the target surface as shown by the depth map to a template for the target surface, and wherein the output provided by the feedback module is generated based on a result of the comparison, and optionally wherein: a) the template is generated by scanning an approved target surface with the light-based scanner, generating a depth-map of the approved target surface using the depth-map generation module, and storing information extracted from the depth-map, and optionally associated tolerances, for use as the template; orb) the template is a virtual template that is not derived from scanning an approved target surface; and wherein optionally the template is a data file such as a pdf.

8. The structural scanning and analysis apparatus of Claim 7, wherein the approved target surface used in template generation is smaller than a typical target surface in use, and wherein either: a. generation of the template comprises duplicating a template for the approved target surface in a pattern to cover at least a majority of the typical target surface area; or b. the comparison module is arranged to compare the template to each of a plurality of regions of the target surface area in use.

9. The structural scanning and analysis apparatus of any preceding claim, wherein the analysis module is arranged to: search the image for a plurality of instances forming a repeated pattern of features and / or attributes in the target surface; and compare the plurality of instances to one another, and / or to pre-defined tolerances, to determine if a desired specification is met, and wherein optionally searching the image for the plurality of instances comprises searching for areas of graphics or text in the image of the target surface.

10. The structural scanning and analysis apparatus of any of Claims 6 to 9, wherein the comparison comprises identifying one or more 3D topographical features of the target surface based on the template, and wherein at least one identified feature comprises:(i) braille provided on the target surface, and wherein optionally the output provided by the feedback module comprises an assessment of the legibility of the braille;(ii) a crease on the target surface, and wherein optionally the output provided by the feedback module comprises an assessment of the shape of the crease, a location of the crease, a tolerancing profile of the crease, a consistency of the crease, or a foldability of the crease;(iii) a feature created using a cutting knife, and optionally wherein the output provided by the feedback module comprises an assessment of the impact and / or presence of the feature created using a cutting knife;(iv) an embossed feature, and optionally wherein the output provided by the feedback module comprises an assessment of the presence or consistency of the embossed feature;(v) a debossed feature, and optionally wherein the output provided by the feedback module comprises an assessment of the presence or consistency of the debossed feature;(vi) a partial cut through the target surface, and optionally wherein the output provided by the feedback module comprises an assessment of the presence of the partial cut based on a comparison to the reverse of the target surface and / or light transmission through the target surface; or(vii) a burst or foil, and optionally wherein the output provided by the feedback module comprises an assessment of the presence or absence of a burst or foil in the target surface.

11. The structural scanning and analysis apparatus of any of claims 7 to 10, wherein the comparison module assesses whether or not one or more of the following properties match the template, and optionally how any deviations compare to allowed variations: (a) presence or absence of a feature included in thetemplate; (b) height; (c) width and length; (d) shape; (e) position; and (f) the presence of an unexpected feature.

12. The structural scanning and analysis apparatus of any preceding claim, wherein the feedback module and / or analysis model are arranged to: analyse a group of adjacent features or attributes in the target surface; and generate an output based on the interaction between the features or attributes, wherein preferably the group of features or attributes include at least two of a different type.

13. The structural scanning and analysis apparatus of any preceding claim, wherein the light-based scanner is arranged to scan flattened or pre-assembly packaging, and wherein the output from the feedback module comprises at least one of: a. instructions for adjustment of a die-cutter or press used in manufacturing the packaging; and b. a quality-control assessment of the packaging.

14. The structural scanning and analysis apparatus of any preceding claim, wherein the feedback module comprises a projector arranged to project one or more colours or patterns onto a viewing surface, the viewing surface being:(i) the target surface; (ii) a shaped surface of a manufacturing machine arranged to produce the target surface, wherein the shaped surface is shaped to correspond to the target surface so as to form the target surface; or (iii) the reverse side of a shaped surface of a manufacturing machine arranged to produce the target surface, wherein the shaped surface is shaped to correspond to the target surface so as to form the target surface; such that one or more 3D topographical features identified within the depth map and / or the one or more generated outputs can be projected onto the viewing surface, and wherein optionally the light-based scanner and the feedback module are integrated such that results of the analysis and comparison can be projected onto the target surface whilst the target surface remains in the position in which it was scanned.

15. A packaging quality control apparatus comprising: a solid-state light-based scanner arranged to capture at least one image of a target surface of a sheet arranged to be used to form packaging, the target surface having an area of at least 0.15 m2and being substantially flat; a depth-map generation module arranged to generate a depth map of the target surface using surface normal information derived from the at least one image, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy of at least ±0.1 mm; an analysis module configured to identify and measure at least one of the topographical attributes of the target surface using the depth map; and a feedback module arranged to provide an output, the output being generated based on a result from the analysis module.

16. The apparatus of Claim 15, wherein at least one of the following applies: a. the depth map shows features of the target surface to an accuracy of at least ±0.05 mm; and b. the target surface has an area of at least 0.25 m2, 0.5 m2or 0.8 m2.

17. An analysis and manufacturing system, comprising: a scanner arranged to provide scan data from which 3D topographical attributes of a target surface can be derived, the target surface being at least substantially flat;a depth-map generation module arranged generate a depth map of the target surface based on the scan data; an analysis module configured to identify and measure at least one of the topographical attributes of the target surface using the depth map; a feedback module arranged to provide an output, the output being generated based on a result from the analysis module; and a manufacturing apparatus arranged to be used in manufacturing the target surface, and to receive the output from the feedback module, wherein the output from the feedback module comprises data for one or more adjustments to settings of the manufacturing apparatus, and wherein the manufacturing apparatus is configured to implement the one or more adjustments in response to receipt of the output.

18. The analysis and manufacturing system of Claim 17, comprising the structural scanning and analysis apparatus of any preceding claim, the structural scanning and analysis apparatus providing the scanner, depth-map generation module, analysis module, and feedback module.

19. A method of automatically adjusting settings of a manufacturing apparatus arranged to be used in manufacturing a target surface, the method comprising: obtaining scan data from which 3D topographical attributes of a target surface can be derived from a scanner, the target surface being at least substantially flat; generating a depth map of the target surface based on the scan data; identifying and measuring at least one of the topographical attributes of the target surface using the depth map; providing an output to the manufacturing apparatus, the output being generated based on a result from the identifying and measuring of the at least one of the topographical attributes and comprising data for one or more adjustments to settings of the manufacturing apparatus; and at the manufacturing apparatus, receiving the output and implementing the one or more adjustments in response to receipt of the output.

20. A photometric stereo scanner comprising: a plurality of lights located spaced from a target surface, each light having a luminous area from which light emanates; and at least one camera, and wherein the plurality of lights and the at least one camera are in fixed positions with respect to each other, the lights and camera being arranged such that specular reflections fall at least substantially outside the target surface by trigonometrically determining a closest mirror point to a boundary of the target surface for each light-camera pair, each mirror point for each light-camera pair being the point on a plane including the target surface where the angle between the plane and a corresponding point within the luminous area of the light is equal to the angle between the plane and a centre of the camera, and ensuring that every closest mirror point is outside of the boundary of the target surface such that all mirror points fall outside of the boundary.

21. The photometric stereo scanner of Claim 20, wherein there is a single camera, and the single camera is located centrally with respect to the target surface, and arranged to face the target surface.

22. The photometric stereo scanner of Claim 20 or Claim 21, wherein the camera is arranged to capture an area of at least 0.25 m2, 0.5 m2, 0.8 m2, 1 m2, or 2m2in each image captured.

23. Use of the photometric stereo scanner of any of Claims 20 to 22 and / or of the structural scanning and analysis apparatus of any of Claims 1 to 16 in one or more of: a. quality-control of packaging; b. assessment of the legibility of braille; c. identification and assessment of creases; d. sheet pressure analysis; e. validation of embossing; f. surface inspection; and g. plate and tool validation.

24. A feedback visualisation apparatus comprising: a viewing surface onto which a projection image is to be projected, the viewing surface being one of:(i) a target surface;(ii) a shaped surface of a manufacturing machine arranged to produce the target surface, wherein the shaped surface is shaped to correspond to the target surface so as to form the target surface; or(iii) the reverse side of a shaped surface of a manufacturing machine arranged to produce the target surface, wherein the shaped surface is shaped to correspond to the target surface so as to form the target surface; a memory arranged to store the projection image, the projection image being determined based on at least one result of an analysis of at least one 3D topographical attribute of the target surface, wherein the projection image is adjusted based on a calibration mapping to account for the 3D position of the viewing surface; and a projector in a known position relative to the viewing surface, the projector being arranged to project the projection image onto the viewing surface so as to visually show the at least one analysis result for the target surface in a location on the viewing surface corresponding to a location of the topographical attribute of the target surface.

25. The feedback visualisation apparatus of Claim 24, wherein the projection image is determined based on at least one result of a 3D topographical comparison between the target surface and a template for the target surface, and wherein the at least one result includes an identification of a feature of the template which is missing from the target surface, and wherein the projector is arranged to project an indication of the missing feature onto the target surface in the intended location of the feature.

26. The feedback visualisation apparatus of Claim 24 or Claim 25, wherein the projector is arranged to implement colour-highlighting of features or areas of the target surface to identify approved features or areas with a first colour and unexpected features with a second colour, the second colour being different from the first colour.

27. The feedback visualisation apparatus of any of Claims 23 to 26, wherein the apparatus comprises at least one projector with a lens facing - and optionally at least substantially parallel to - the target surface, the apparatus optionally further comprising one or more of: a. a backlight arranged to illuminate the target surface from behind the target surface; and b. a low-angle light located closer to the target surface than the lens of the or each projector and directed across the target surface.

28. A method of visualising information regarding a target surface in situ, comprises: positioning the target surface on a bed;generating a projection image based on at least one result of an analysis of at least one 3D topographical attribute of the target surface; mapping the projection image to the target surface using the results of a calibration mapping to account for the 3D position of the target surface, so as to adjust the projection image to the target surface; and projecting the adjusted projection image onto the target surface using a projector in a known position relative to the target surface so as to visually display the at least one analysis result for the target surface in a location on the target surface corresponding to a location of the topographical attribute of the target surface.

Images