Tag data processing system

Abstract

A product label printing and inspection system comprising data processing means configured to control the printing of product labels, to control the acquisition and reception of images of the printed labels by image acquisition means such as an optical scanner, and to inspect the printed labels for defects, wherein each printed label conforms to a label format specification for that label, whereby the labels have a common layout and comprise information relating to the printed product located in one or more areas on the label, wherein the data processing means comprise a label inspection module in which a reference image of the label is provided and the inspection module is configured to cause the acquired image to be sequentially compared to the reference image in accordance with predetermined quality control indicators relating to the expected information content and location in the label areas, and wherein if the label does not conform, the label is flagged for review or rejection. The label format specification can be determined and / or stored in the form of an accessible file prior to the printing process. The label format specification can then be retrieved by the label printing and inspection system prior to the printing process using that label format. In one aspect of the invention, the label format specification can be used to provide instructions to be sent to the printer for printing of each label. Thereby, by assembling the specification for label area locations, area information and verification tools and optionally verification tool parameters, the reference image and its associated verification mask can be configured in real time.

Description

TECHNICAL FIELD

[0001] The present invention relates to the field of product labeling, and in particular to a system for inspecting or examining labels after they have been produced to confirm that they are correct and substantially free of defects. Labels can display information such as product identifiers, manufacture dates, batch numbers, and item numbers, which can be necessary for regulatory compliance or to provide provenance information. This is particularly important for high value products such as pharmaceuticals (or candidate pharmaceuticals for tracking), aerospace parts, and medical devices, where an "audit trail" must be established and verified from manufacture, through distribution, and to the point of use or installation of the final product. BACKGROUND

[0002] Labels are typically printed in batches or by the number of impressions using a printer and appropriate print media, such as self-adhesive paper labels on a backing web. Typical printers used are inkjet printers, laser printers, dye-sublimation printers, dot-matrix printers, and wax thermal transfer printers. Wax thermal printers tend to be preferred for label applications due to cost and print speed.

[0003] Defects in printed labels can include a number of possibilities, such as scratches, smudges, missing features or dots or clusters of dots, edge bleed, streaks, bars, misregistration, off-axis rotation, and distortion. These defects can arise due to printer roller misregistration, printer needle misregistration, ink problems, paper surface contamination, or raster mismatch causing blank or skew.

[0004] In the past, label inspection has been performed "manually" by trained operators who typically take samples of labels on a periodic basis by visual inspection. This work is repetitive and tedious, which leads to operator performance degradation or other human errors. One study (Bill Smith, "Making war on defects", IEEE Spectrum, Vol. 30, No. 9, pp. 43-47, 1993) revealed an accuracy of only 80%. Human intervention limits the speed at which printed labels can be inspected, and increases the cost of the process. For processes where every individual label must be inspected (such as pharmaceutical labeling), manual inspection is impractical unless only small batches of labels are printed.

[0005] Machine-based visual inspection can use a variety of underlying techniques, which typically involve producing a reference image against which the actual printed image can be compared to identify defective images. Decisions must be made as to which parts of the printed label are critical, how much deviation from "ideal" is tolerable, and how to measure this.

[0006] US patent 4589141 (1984) discloses an automatic label inspection apparatus in which a television camera takes an image of the label which is then sent to pattern and image recognition circuitry for verification by comparison with stored reference label pictures. An operator is required to undertake a teaching process in which the relevant parts of the label are identified and highlighted using a joystick controller.

[0007] US patent 5755335 (1995) discloses a machine based method for inspecting labels after they have been applied to containers for pharmaceutical packaging. The containers are placed on a rotatable carousel and the labels are identified by detecting the edges of the labels during rotation. The labels are then assessed for compliance with predetermined criteria or specifications for the labels and rejected if there is non-compliance.

[0008] More recently, US 2002 / 0087574 discloses a method for automatically inspecting hard copy labels used in clinical trials in which a scanning device reads the labels after they have been printed. Separate inspection machines are used to scan the labels which are compared to a master label image which is loaded as a master data file. Different areas of the label are assigned different print quality tolerances depending on the sensitivity of the information involved. If the label does not meet the required quality, an automatic stop is activated.

[0009] WO 2011 / 090475 discloses an automatic inspection system for detecting defects in a printed image. A raster image is sent to a printer and printed on a print medium such as paper. A scanner captures a target image of the printed image on the medium at a lower resolution than the printed image. The raster image is converted to a reference image and the reference image is compared to the target image in order to detect defects in the target image. The system is adapted for colour images and the reference image is composed of cyan, magenta, yellow and black components. A structural similarity measurement method is used on each colour channel of the target image.

[0010] WO2014 / 108460 (the contents of which are hereby incorporated by reference) discloses a label inspection system in which a scanner is incorporated into the printer housing so as to allow the label to be inspected on-line as it is wound from the print head. Prior to each label print run, an associated "training image" is created, this image including an inspection mask which configures the inspection process to be carried out on the scanned label image by defining the label areas which include critical data and assigning appropriate inspection tools to each area in a series of layers according to the nature and quality of the image data expected to be found in the scanned image of that zone. By dividing the scanned image into zones and selecting inspection tools optimised for the contents of the zone, the required image processing will be less onerous and time consuming. In prior art label inspection systems such as this system, the development of a reference or training image for each label is a laborious task, usually carried out manually by a skilled operator for each label print run. A key feature of the training image is that it has been tested on printed labels and manually optimised (i.e. trained) so as to apply the appropriate tools and tool parameters to produce an accurate inspection for the given label format. The training image and mathematical model are stored for subsequent print runs of that label. In this way, a library of training images can be created and stored and these training images can be accessed and downloaded remotely when a label print run is carried out at a particular geographical location.

[0011] This can be achieved to some extent by providing a GUI based label creation software which will allow the design of the label format and the specification of the label contents. A separate step takes an image of the final label (scanned or photographed) and this is evaluated and the inspection tools and default parameters are assigned to the zones of the label. The operator identifies the parts of the label to be inspected and the nature of the information such as text or pictures which will be static or expected to vary from batch to batch or from label to label. There will be zones which should not vary such as frames or borders. There can be text such as brand names, regulatory markings, barcodes, colours or shades, dots, patterns. There can also be information such as usage or storage instructions. The various zones can involve different line thicknesses, font sizes, lettering, colours, shades or contrasts. Once the zones have been defined and the inspection tools specified for the zones, the tools must be tested to ensure that they are capable of adequate inspection. This can be done well on pre-printed dummy labels before a batch of actual labels is printed. In order to optimise the inspection process, the inspection parameters such as sensitivity, dimensional tolerance and inner margin can be varied, the algorithms adjusted and the tests re-run. The establishment of a training image (and the system which creates a mathematical model) can therefore require many hours or even days of skilled work and iterative testing and adjustment.

[0012] This is a particular problem in industries where product labels do not conform to any set of format, layout or content standards, so that "off-the-shelf" training images are rarely available. One prior art approach requires the user to adopt a specific labelling standard, which has predetermined training images with established regions and inspection tools and algorithms. These standards would be centrally maintained on a remote server, or accessible for download from a website prior to the labelling print process. However, this binds the user to a specific supplier, and inhibits flexibility in label design.

[0013] Another problem that has been found is that the data processing requirements of label inspection are severe, requiring complex and time-consuming image analysis routines and methods. Computer vision software is available both commercially and open source. One commercial supplier is Cognex, with VisionPro and Vision Library machine vision software. An open source supplier is Open CV (Open Source Computer Vision Library). Both of these provide libraries of image inspection tools. Inevitably, these tools are configured to provide general visual utility, and are not necessarily adapted to the specific content needs of, for example, printed label inspection.

[0014] Scanned images of labels, typically at 300 or 600 dpi resolution, contain a large amount of information that must be decoded by image processing techniques and algorithms to allow interpretation. Typically, in commercial implementations, the data processing is carried out on a dedicated processing system remote from the printer / scanner combination. Thus, for each scanned label, a significant amount of time can be required to scan, send the data packet to the processor for decoding, and pass or fail the label before scanning the next label or evaluating its image. Thus, the label inspection process can slow the label printing process, and ultimately impact overall productivity and cost. SUMMARY

[0015] According to one aspect of the application, there is provided a product label printing and inspection system, the product label printing and inspection system comprising data processing apparatus configured to:

[0016] control printing of product labels;

[0017] control acquisition and receipt of images of the printed labels by image acquisition apparatus, such as an optical scanner; and

[0018] inspect the printed labels for defects,

[0019] wherein each of the printed labels conforms to a label format specification for that label, whereby the labels have a common layout and include information relating to the printed product located in one or more regions on the label,

[0020] wherein the data processing apparatus comprises a label checking module in which a reference image of the label is provided and the checking module is configured so that the acquired image is sequentially compared with the reference image according to predetermined quality control indicators relating to the expected information content and location in the label area, and wherein if the label does not comply, the label is flagged for review or rejection.

[0021] The label format specification can be determined and / or stored in the form of an accessible file prior to the printing process. It can then be retrieved by the data processing system prior to the printing process using the label format.

[0022] The system is typically configured to output control signals for controlling the operation of the printer. The system is typically configured to output control signals for controlling the operation of the scanner. The system can be configured to receive and process input signals acquired from the image acquisition device.

[0023] In an aspect of the application, the label format specification can be used to provide instructions to be sent to the printer for printing the individual labels. Preferably, the reference image is an e-image constructed according to the label format specification. In a preferred aspect, the reference e-image is constructed using the instructions to be sent to the printer. In operation, the reference e-image is prepared in parallel with the printing of the associated label.

[0024] The reference image is typically provided with an inspection mask which associates regions of the reference image with inspection tools stored in the system which are assigned as being of a type suitable for the information to be inspected at the corresponding region on the acquired image. In a preferred arrangement, the one or more inspection tools are assigned to the associated region using the label format specification and a ruleset (or wizard) to select the best inspection tool and parameter settings.

[0025] In a preferred aspect, the reference image is an e-image constructed according to the information used to instruct the printer when printing the label. By constructing (or using the same process to construct) the reference e-image using the image sent to the printer, the extent to which the expected printed image can vary from the acquired scanned image is greatly limited. However, the reference image can instead be obtained as an import image of a sample compliant label.

[0026] The reference image is typically provided with an inspection mask which associates regions of the reference image with inspection tools stored in the system which are suitable for the type of information to be inspected at the corresponding region on the acquired image. At least one inspection tool can have a parameter setting and in forming the inspection mask the parameter is set to a value which optimises the performance of the tool. Typically most or all of the tools will have one or more parameter settings.

[0027] In a particular aspect of the application, the data processing device is configured such that at the start of a print run of a batch of labels, a reference image and its associated inspection mask are established using the label format specification, and optionally the inspection tool parameters are assigned. Thereby, by assembling the specification for label area locations, area information and inspection tools, and optionally inspection tool parameters, a reference image and its associated inspection mask can be configured in real time. This can be done by the inspection mask wizard as outlined below. The specification can be assembled at least partly by importing a label format specification predetermined when the label is generated prior to the print run.

[0028] In building the inspection mask, an iterative process is typically started in case the inspection tool (using a tool appropriate for the task) fails to extract the expected features from the area, whereby the tool parameters are adjusted and the inspection is repeated until the inspection is successful. This testing and iterative optimization can be referred to as a training process.

[0029] In prior art processes, these inspection mask creation, stored electronic images and mathematical models, testing and optimization steps are performed as separate processes very early on in the label printing step process. By automating the mask creation and optimization process, and selecting appropriate inspection tools, the inventors are able to generate the reference image and mathematical model on the fly at the start of the print run, rather than relying on pre-prepared and remotely stored electronic images and mathematical models.

[0030] Once the inspection mask has been created and tested / optimized, the process of checking the labels in a production print run can be started. The checking module is therefore preferably configured such that each inspection area is compared to the corresponding area of the reference image in order to assess the conformity of the label as a whole.

[0031] Preferably, a golden template comparison (GTC) is used to compare one or more areas on the reference image and the captured image. A golden template comparison is well suited for one or more areas associated with a text or picture inspection tool, and is preferably used for all areas associated with a text or picture inspection tool.

[0032] In yet another aspect of the application, the reference image is an electronic image constructed by using the information used when sending print instructions to the printer when preparing to print the first label image. This represents the ideal version of the printed content embodied in the label image. By updating, the electronic image allows for serialization data specific to the label.

[0033] To ensure efficient processing, the electronic image and its associated inspection mask should be constructed in parallel to the sending of print instructions to the printer and the rendering of the label image to be printed. Thereby, the mask can be constructed as the printer renders the image.

[0034] The labels can be printed in batches, with the data constant for that batch. In this case, the individual labels should be identical in the printed content. In many cases, serialisation data (such as product serial numbers) or randomisation data (which can be required in pharmaceutical trials) will be required. In these cases, one or more areas of the printed label contain serialisation or randomisation information that varies from label to label. In this case, the corresponding area of the electronic image can be amended after each label print, so that the area of the image where the information varies is populated with the varied information in response to the corresponding variation in the label print instructions.

[0035] To facilitate the image comparison step, various measures can be taken to adjust the reference image to provide a closer match, so that only unintentional defects or errors are highlighted. Thus, the reference image can be adjusted to have a resolution (typically measured in dots per inch (DPI)) that approximates or preferably matches that of the scanner (and hence its images).

[0036] The reference image can be passed through a blur filter that blurs the reference image to more closely resemble the appearance of the scanned image. The scanning process inevitably introduces blur due to scanning system optics and image sensor characteristics. The reference image representation as an electronic image is of an ideal image with sharp edges. However, a better comparison with the scanned label image takes into account the inevitable blur (and preferably at a matching resolution) in order to compare like with like.

[0037] The system can be configured so that the scanned image is subjected to a geometric transformation in order to reduce or correct any distortion present in the scanned image, and wherein the transformation is applied to the scanned image prior to comparing the reference image with the scanned image. Preferably, the transformation is derived by performing a preliminary comparative analysis of the scanned image relative to the reference image. This again ensures that we are comparing like with like in the label inspection comparison step. Given the quite tight mechanical tolerances present in such machines, it is unlikely that the scanned image will ever exhibit unacceptably extreme distortion. This provides the opportunity to simplify the transformation interrogation process by setting boundary limits on the amount of distortion. The transformation can be derived from a matching multi-keypoint analysis of the scanned image relative to the reference image.

[0038] According to yet another aspect of the application, the system can comprise a text or picture inspection tool that relies on matching key points extracted from a reference image with corresponding key points of a scanned image, wherein the translation of the scanned image is computed by recording the displacement of the key points on both the x and y axes. A histogram can be derived from the recorded data, which represents the frequency of key point pairs as a function of the relative displacement, and wherein the peak of the histogram is taken to represent the best estimate of the accurate key point displacement.

[0039] In order to optimize and simplify the method, repeated or spurious key point pairs, which inevitably occur, are binned from the data set by binning key point pairs that do not have a translation displacement within a predetermined distance of the peak of the histogram, such as within 10 pixels, thereby producing a reduced data set. The peak of the histogram can be recomputed using the reduced data set.

[0040] The data processing device comprises a single computer or a network of computers. At least the acquisition or scanning control and label inspection module functions can be performed by a common computer. The common computer is preferably located in the vicinity so as to allow a direct cable connection with the scanner.

[0041] The system preferably comprises an image acquisition device. This image acquisition device can be a digital camera, but is typically a scanner. The scanner can be integrated with the printer, or be a detachable addition to the printer, or be a standalone device arranged separately from the printer. The scanner can be adapted to receive a printed medium output from the printer in the form of a printed product label, and to scan it in turn.

[0042] In yet another aspect of the application, the computer hardware is loaded with software and / or provided with firmware that is adapted, when connected to a printer and an image acquisition device such as a scanner, to function as a data processing system as described above.

[0043] In yet another aspect of the application, a physical machine-readable data carrier product or digital media storage device is provided, the product or device being loaded with software that is adapted to function, when run on computer hardware, as a data processing system as described above. In operation, the system is connected to a printer and an image acquisition device such as a scanner.

[0044] A printer adapted for use with or in the present application comprises a print head, an ink source which can be liquid or wax-based, a paper feed and / or paper transport mechanism, a physical or wireless connector for a server computer or networked computer.

[0045] The image capture device can be a camera, but more typically will be a scanner having a scanning head and sensor, as is known in the art. The scanner can have internal memory, and / or will be configured to send scan data directly to the data processing unit, as described above.

[0046] The computer will typically have a processing structure and memory (which can be internal to the computer, or external and / or remotely located). There can be a visual display unit and data entry device, such as a keyboard, mouse or other such device. To optimize operation and minimize sending data over long distances, there can be a single computer for the operating system and to control the printer and scanner (or other capture device). BRIEF DESCRIPTION OF DRAWINGS

[0047] The modes for carrying out various aspects of the application are described below by way of example only.

[0048] In the drawings:

[0049] Figure 1 is a schematic flow chart representing components of a label printing and inspection system according to an embodiment of the application.

[0050] Figure 2 is a schematic flow chart representing operations performed by a system according to the application.

[0051] Figure 3 is a flow chart for generating a reference electronic image.

[0052] Figure 4 is a screen shot of an example of a concept simple label design.

[0053] Figure 5 is the same image with information blocks applied. DETAILED DESCRIPTION

[0054] Figure 1A label printing and inspection system including associated peripherals for use in the present application is shown schematically. In this example, the data processing unit is a computer which is configured by software to enable the present application as described below and as set out in the claims. The data processing is loaded with software for executing a label design wizard and for conducting a printing process in which each printed label is checked for compliance with minimum standards. As shown, the data processing unit is connected to a printer (typically a wax thermal printer) which will be loaded with a roll of labels for sequential printing. As each label is printed, it is fed to a scanner which has a scanning head for scanning each label. The scanner produces a scanned image of each label which is transmitted to the data processing unit for analysis and comparison with a reference image of the label as intended to be printed. The present application resides in the configuration of the data processing unit which will be discussed in more detail below. It will be seen that the present application is potentially independent of the printer and scanner which can be any such suitable devices known in the art, although in some cases it will include these peripherals.

[0055] In Figure 2 The processing by the data processing unit is shown schematically. It is assumed that a label format specification will have been prepared. The printing process for the label is configured and in a first step, a verification mask is prepared, which can be done in advance. The operator can choose to use a pre-existing label design (with an associated label format specification to be retrieved).

[0056] Once the label format specification has been retrieved from storage, the preparation of the verification mask can begin. This can be done in advance of or at the start of the printing batch process. As a first step, the verification mask is created. The verification mask wizard analyses the label content (information and layout) and selects appropriate image analysis tools for the type of information to be presented (e.g. text, image, logo, line, etc.).

[0057] At the start of the printing batch process, the commands for label printing are generated which are sent to the printer. The electronic image of the first label to be printed is generated using the same print instructions. The electronic image is adjusted by applying a blur mask and by resizing to match the scanner resolution so that the electronic image is as close as possible.

[0058] The verification tools specified in the verification mask are then trained using the electronic image, the information about the label specification and the tool parameter settings.

[0059] Thus, the system can be configured to conduct a preliminary inspection tool training process, which is conducted on a reference image (or electronic image), and the system is operated such that in the event that the inspection tool fails to extract the expected features from the area, an iterative process is initiated whereby the tool parameters are adjusted and the inspection is repeated until the inspection is successful. Once trained, the system is ready to conduct label inspection.

[0060] According to a new aspect of the invention, the labels are printed while being trained. A scanner is activated and acquires a scanned image of the printed label. This label scanned image is then compared to the electronic image by using the inspection tools defined in the inspection mask. For serialized data, which varies from label to label, the electronic image can be reconfigured to reflect the varying label information sent to the printer. The scanning and comparing process continues until the batch has been printed and all labels have been inspected. In the event of a label that does not conform, various responses can be followed, such as interrupting the printing or sounding an alarm. If the content does not vary from label to label, the electronic image and the trained inspection tools are not regenerated for the remaining labels.

[0061] In the following pages, we describe in detail the various process steps involved in the implementation of the invention in a particular embodiment.

[0062] Label creation - label wizard

[0063] The operator uses the label generation software to create a label for a particular print run or multiple print runs of the same product or data set.

[0064] The label format comprises a list of areas, in this case blocks. The label format can be linked to a schema which will specify exactly all the data needed to produce the label. One schema can reference another to form a hierarchy. All the data available for the label will form a tree structure which will be presented to the user as a variable pane docked to the side of the label editor.

[0065] The user can use drag / drop to populate the label with content from the variable pane:

[0066] A string variable can be dragged from the pane and dropped into the label to create a new text block.

[0067] A reference variable which appears as a container above a set of related variables can be dragged into the label to insert a sub-label.

[0068] A multi-language string variable has its own specific representation in the variable pane.

[0069] An image variable can be dragged onto the label in order to add a variable image to the label.

[0070] Variables can be dragged into cells within tables to add default content to that cell of the table.

[0071] Block positioning

[0072] Blocks on the label have rectangular bounding boxes. Available blocks will include the following:

[0073] Text - Rich text formatting, automatic wrapping, and insertion of multiple variables. The width of a block of text can be stretched or shrunk. The application will allow text to automatically fit to the block it is being rendered in. This is controlled by a fit space parameter on the text block. This will then allow a maximum and minimum point size for the entered text. Optionally, width stretching can be implemented, which will allow the font width to increase if the optimal point size fits, or decrease if the minimum point size overflows. Width stretching has maximum and minimum percentage settings.

[0074] Barcode - A range of barcode symbologies will be supported according to standard practice in the field. In order to properly preview barcode blocks on screen, the interface will feature a control that allows the user to set the resolution of the printer that the label will be printed on. This will calculate the space that the printed barcode will take up on the label. The application will also allow the user to automatically fit a barcode to a specific block size by calculating the optimal fit of the unit size and bar height.

[0075] Picture - A picture-only entity located within the label design system will be supported for addition to label formats.

[0076] Line - A horizontal or vertical line between two points can be specified, with standard options for how the line is drawn.

[0077] Shape - A rectangle (including rounded rectangle) or ellipse, optionally a shape that is stroked / filled.

[0078] Table - A specific number of rows and columns can be specified. Other blocks can be placed in the table so that their position and size do not have to be manually adjusted to align them; they automatically get their position from the table's layout.

[0079] Sublabel - A reference to another label format, so that the referenced label is rendered on the parent label (see below).

[0080] Rich text content - When adding a text block to a label, the user can add rich text content using either single character-level controls or block-level controls for formatting. Character-level formatting controls are available when a text block is opened for editing.

[0081] The following capabilities will be available to the user:

[0082] 1. Selecting a font for selected text

[0083] 2. Select font size for selected text

[0084] 3. Toggle bold for selected text

[0085] 4. Toggle italic for selected text

[0086] 5. Toggle underline for selected text

[0087] 6. Toggle strikeout for selected text

[0088] 7. Left align selected text

[0089] 8. Center selected text

[0090] 9. Right align selected text

[0091] 10. Justify selected text

[0092] 11. Superscript selected text

[0093] 12. Subscript selected text

[0094] 13. Font color and background color selection

[0095] When the rich text variable type is specified within a schema, this gives the user the ability to specify the text content as well as the formatting associated with the text. This can be used to specify chemical formulas using subscript or superscript text, and to select colors for specific text within the data, but not within the label format.

[0096] Sub-labels

[0097] Any label format itself can be used as a block within another label format. A label that is located within the layout of another label acts as a sub-label. Just as a text block can be configured to use a variable to provide it with text data, a sub-label block must be configured to use a variable to provide it with the data it needs. The use of sub-labels can be extended to any desired depth. Thus a particular batch can use custom labels that are handled primarily by a generic product label by customizing it as a sub-label that covers the entire space. Then they can insert additional variables into the blank space in the center.

[0098] Label variables

[0099] Label variables allow the definition of label specific data (i.e. information that varies from label to label) and placement of this data on the layout. This data can be a serial number, date time or a list. Once defined, these label variables are specific to the label format in which they are defined. A serial number will have a starting value, an increment and a value indicating the number of times the sequence is iterated. The maximum length of the serial number will be specified and it can also be configured to have leading zeros. These values can be constant, calculated fields within the available modes or prompt variables.

[0100] Inspection masks and training

[0101] With respect to prior art standalone visual inspection systems, the operator is required to scan the actual label and then perform a laborious task to manually mark the various areas of the scanned label and define how the area is to be inspected. In accordance with the present invention, the wizard incorporates the knowledge held by the label system of the design / content of the label to enable automation of this process; this reduces the amount of time required to manually perform this operation by a considerable amount and is designed to eliminate the need for involvement of a visual engineer.

[0102] Labels and printing

[0103] A typical system for label inspection includes, on the one hand, a printer that receives label printing instructions from label printing software loaded onto a printer service computer. In a preferred aspect, the printer is a wax thermal printer, which provides lower per-print cost and fast printing. However, other printers can be used, including inkjet printers, dye sublimation printers, dot matrix printers or laser printers. For applications such as medical or clinical trial labeling, it is often necessary to perform defect testing on each printed label in a batch.

[0104] Labels are typically printed in batches on self-adhesive paper labels that are attached to a backing web, from which the labels can be peeled off for attachment to containers or packages. Of course, the present invention is not limited to such labels, and any suitable printing medium can be used, such as plain paper, polymeric film, packaging cut-outs (for forming containers), paperboard, provided that the medium can be transported past the print head to allow printing thereon.

[0105] The simplest sense of a printed label can be a product code expressed in alphanumeric text. More commonly, there will be a mix of text such as brand, instructions for use, batch number, product number, certification logo, manufacture date, use-by date, weight value, etc. The text can be presented in the same or different font, font size, and in different orientation (e.g. horizontal or vertical). The text can be in different languages, or can be non-Latin text such as Chinese and Japanese symbols or Arabic script. There can also be pictures such as product images, trademarks, logos, schematic illustrations. The pictures can be in monochrome, grayscale or dithered. The label can have pre-printed borders, frames or other images and text.

[0106] The printed information can be product-specific, batch-specific or serial (in the sense that each label has different information such as item number or unique product serial number). The data can also be non-sequential, and for clinical trials, the data can be randomised. The label can be of different shape, square, rectangular, circular, oval or any other shape specified by the designer. The label can be in any format, layout, content and have static, batch-specific or product / item-specific information. In practice, many labels are typically rectilinear but have rounded corners. The rounded corners are a useful visual feature for identifying the location and orientation of the label in an image.

[0107] In some embodiments, the printed labels can be printed in batches and the stack of labels is conveyed to a separate scanner for scanning of the labels. In this case, the scanner will typically comprise a pull feed (or similar) for pulling a roll or concertina of printed media (on any associated backing layer) through the scanner. Instead of a scanner to capture the label images, a digital image of each label captured on a photographic sensor can be used. However, the preferred arrangement of the present invention is to mount the scanner immediately downstream of the print head of the printer so that as soon as a label is printed, it is scanned to capture a sequence of scanned images of the printed label. The printer and scanner can be integrated in a single machine or can be two separate items combined together. Typically, it is the latter in the case where the label printing system is modified for a printing process where label accuracy is critical and must be checked.

[0108] In order to address any printing problems that arise, it is important in most embodiments that the label inspection takes place "on the fly" and in synchronisation with the label printing process so that if a defect is detected, the printing process can be paused or slowed down, or the printing parameters adjusted, to address the problem causing the defect. In a batched operation where the defect detection is separate from the batch printing, any defective labels will still be detected but there will be a considerable waste of time and material if any defects prove to be systematic over a portion of the labels.

[0109] Potential defects include information drift from its intended location, scratches, spots or "black spots," missing portions or dots, streaks, bands, and variations in intensity. Defects can be isolated or may form a trend that could worsen over time. Some defects may require label rejection, while others are less critical and can be permitted, or may indicate the need for press servicing. However, for critical applications, such as medical, pharmaceutical, or drug testing applications, our primary focus is on detecting and responding to labels that must be rejected. It must also be recognized that errors can be introduced not only during printing but also during the scanning process; therefore, scanner alignment, stability, and tolerances relative to the delivered printing media (e.g., label arrays) are crucial for avoiding erroneous results and unnecessary rejections. However, press errors tend to be more prevalent, as they are inherently more prone to producing defects (through various modes) than scanners involving optical interactions.

[0110] The scanned images are then compared, typically one by one, with reference images representing the intended or target label content. In this invention, the reference image is a digital or electronic "electronic image" derived from the actual image of the sample label, or, in a preferred embodiment, a derived printing instruction to be sent to the label printing press.

[0111] Preparation for testing the mask

[0112] The inventors have confirmed that many steps involved in setting up and inspecting an inspection mask to ensure its reliable operation can be defined by simple rules. These rules may need to be used multiple times for different labels that have potential similarities in the nature and format of the information presented. Therefore, a wizard was created to automate the establishment of these rules. The wizard attempts to create a good inspection for the label by identifying appropriate inspection tools and sensitivities for the various blocks on the label, including different product datasets printed on the label.

[0113] In a preferred aspect, the system of the present application is able to automatically mark the labels for inspection by using an inspection mask wizard and script. The wizard can be run against a previously obtained image of the label to obtain a provisional mask. The provisional mask can then be used in the inspection process. In the possible case that fine tuning of the provisional mask is required due to substandard initial performance, the wizard can be set to automatically make changes to the image processing or can mark suggestions to the operator to change the mask, which the operator can then invoke. The provisional mask is made by the wizard pre-analysing the label format and content using the inspection wizard to select the correct image processing inspection tools (e.g. text or picture) and sensitivity options for each tool. The wizard can also be adapted to check for areas of the label where content overlaps and provide masking protocols to correctly process these areas. Once the provisional inspection mask is formed, a test label inspection is carried out to see if any training errors or warnings occur. Preferably, however, no training images or converted training images are stored in the data processing means (or any associated data storage facility).

[0114] For each printable block on the label format, the wizard will select the appropriate inspection tool, inspection sensitivity and search area and will proceed to test: when presented with an ideal representation of the block, the visual inspection run-time component is able to successfully identify each block. If any block fails on the initial attempt, the wizard will follow a predetermined protocol on how to adjust the inspection parameters in order to iteratively achieve successful identification.

[0115] For example, the wizard processes a block on the label which initially starts by setting a "high quality" inspection tool with high sensitivity and tight search area. If the subsequent inspection carried out is not successfully identified by the visual run-time component in ideal circumstances, the wizard can be programmed to select a "high quality" inspection tool with medium sensitivity (rather than high sensitivity) and tight search area in order to achieve identification pass. Once a pass is obtained for all label areas, the inspection mask can be used for the label printing and inspection process.

[0116] During the training / test time, the wizard can be instructed to use the results of the scan to allow manual modification of the inspection mask. The wizard will provide suggestions to change the inspection mask which can be accepted or rejected by the user. Alternatively, the user can manually modify the mask settings to adjust as necessary to meet the inspection requirements. The suggestions will typically be presented to the user in textual form with the option to apply the change or view more details. If this is the case, the user is preferably shown a graphical example of the error or issue relating to the suggestion made.

[0117] To simplify the process of creating an inspection mask, an inspection mask wizard can be used to automatically create a candidate inspection mask and then train the inspection mask for the target label format. The inspection mask wizard script defines how the image analysis process will associate various types of designed and printed label blocks with each of the various inspection tools, their sensitivities, and defined search areas implemented within the inspection mask configuration. The script will also define how the wizard should act if it is unable to "train" or recognize a particular block.

[0118] In one more detailed example of the invention, some of the steps the wizard takes when training are:

[0119] 1. If the label has a label sheet with rounded corners, create a fixed layer of 4 corners.

[0120] 2. For each block, the default inspection tool, sensitivity, and search area based on predetermined settings in the inspection wizard configuration file will be used. For certain types of blocks, there can be additional decisions.

[0121] a. If the block is text, select the accuracy of the inspection based on the text size (and if some complex characters such as Chinese are used).

[0122] b. If the block is a picture, determine if the image is dithered and use the appropriate settings.

[0123] c. If the block is a barcode and the size of the individual barcode column is small, just set to read the barcode without grading it.

[0124] 3. Add a default inner margin to the block taking into account how far the block is from the edge of the label.

[0125] 4. Check if there are any blocks that overlap. If so, try and reduce the block size by using the inner margin. If the inner margin cannot be used, cut out the exception on blocks with no printed content. There can be some blocks where the final content can not be known, in which case the wizard can generate a suggestion for the user to make a decision on the exception cutout.

[0126] When testing the scanned image, the wizard will try and resolve common inspection errors with alternative settings. For example, if no corners are found, it will try detecting dark corner settings (i.e. if the label sheet is darker than the background).

[0127] Printing process

[0128] Once the mask is prepared for a batch of labels to be printed, the printing process can take place. The flowchart in the figure illustrates schematically the printing process. The previously formed inspection mask is retrieved (see above). The label generation software provides the image information for the first label in the form of print instructions to the printer. The same image information is used to generate an electronic or digital image of the label to be printed (the electronic image). This includes any pictures, text, borders to be printed. The entire electronic image is modified by rescaling the resolution to match that of the label scanner used in the inspection process (this is usually true when there is a resolution difference, as the resolution of the scanner will typically be higher than that of the printer image) and applying a blur layer to mimic the inevitable image blur that would occur due to the performance limitations of the scanner.

[0129] Inspection tool training

[0130] The electronic image is used as a test image to test / train the inspection tools. The process goes through each region / block in the label and does the following:

[0131] 1. Create an instance of the inspection tool specified for that region / block.

[0132] 2. Set the parameters of the inspection tool based on the sensitivity (and any previous user specified overrides).

[0133] 3. Run the training phase on the tool. This will be given the electronic image, the location of the region and the content of the region. If the tool needs it, this will generate a mathematical model of the region. If needed, the tool can also produce different images of the region such as by extracting blur.

[0134] At this stage, different tools do different work. Some tools use the electronic image for image data, others do not need the electronic image and use only the description of the region to indicate the nature of the content.

[0135] One important tool is the text and picture tool. It extracts key points from the electronic image and generates various blurred images used in the image subtraction / comparison stage used when actually inspecting a scanned label.

[0136] At the same time, the label image is printed onto a label provided on a releasable backing layer. The printed image is scanned by a scanner provided immediately downstream of the printer print head. The scanned image will have a resolution dictated by the scanner characteristics or settings and can be subject to scanning artefacts caused by the scanning process and artefacts caused by printing defects present in the printed image.

[0137] Label inspection process

[0138] The scanned label image is then subjected to a process of inspection called inspecting the label. This runs different inspection tools to inspect the scanned label against the electronic image. A first set of tools is run to find a rough location of the label content (starting from the label corners in most cases). Once these tools have been run, the remaining tools are run.

[0139] For each region, the inspection step:

[0140] 1. An instance of the tool created in the "training inspection tools" phase is used.

[0141] 2. The module is given the scanned label image.

[0142] 3. The module is given a transformation from the scanned image to the location on the electronic image. This is where the rough location of the content can be expected to be found.

[0143] Each tool uses the information provided and any models it has built, or information already provided during the training phase, to perform the inspection.

[0144] The text and picture tools extract regions of interest from the scanned label image. It will then extract key points. It will attempt to match these to corresponding key points on the electronic image, as described below.

[0145] Feature extraction

[0146] An important part of the label comparison process between the reference electronic image and the scanned image is the feature matching step, in which the characterising features act as a reference (or key point) against which the location of other label content or regions can be defined. In this context, such a feature can be an edge or, preferably, a corner, or the result of a mathematical function such as the partial derivative in both the (x and y) directions. A filter finds all the potential features in the image. Because there can be many similar features, a "descriptor" is defined to help match the sought characterising features. The descriptor is an abstract way of describing the image region around the feature.

[0147] It is customary practice to look for angular objects. The process involves a large and complex number of calculations. Custom-made computational methods are tailored to be able to match a wide range of images, regardless of any translation, rotation, skew or scaling of the image. Descriptors from the electronic image are compared to the most similar descriptors from the scanned image. Among the many possible candidate features, the best matching descriptors are selected as between the electronic image and the scanned image. Position information cannot be relied upon because a quantification of the amount of mismatch due to rotation or translation or scaling will occur. To deal with scaling mismatches, the source image is scaled to a number of levels and features / descriptors are extracted for each of these levels. In this way, the matching process becomes scale invariant. To deal with rotation, a form of rotation invariant descriptor is selected. The use of multi-scale features and their descriptors can add significantly to the processing time. Similarly, the use of rotation invariant descriptors will add additional processing time. In practice, it can take several seconds to extract features such as corners and their descriptors in a single label. This means that the training of the inspection mask for a label can take a significant amount of time, which is alleviated by carrying out this step much earlier in the prior art batch printing process.

[0148] The present inventors have recognised that for the analysis of printed and then scanned labels, the ability to use scale and rotation invariant features in the feature extraction and descriptor process is not required. In practice, the scanned image will have a known resolution and will typically show only very slight rotation and scaling variations compared to the reference electronic image. Slight local scaling and rotation variations can be expected to be seen in parts of the label. However, neither multi-scale features nor the extraction of a multi-scale set or descriptors is required. Similarly, rotation invariant descriptors are not required. The computational process can therefore be simplified and the processing time can be reduced radically.

[0149] Furthermore, the present inventors have identified that the matching of the extracted features (keypoints) represents a limiting step in the image comparison process. The aforementioned "best match" step can be time consuming due to the discovery of false positives (false matches), particularly due to the presence of "outliers" such as similar but not target keypoints. It is important to match keypoints from the scanned image to keypoints of the electronic image in order to form a transformation to be applied to the scanned image in order to ensure that consistent registration of the scanned image and the electronic image is potentially possible during the comparison step. The formula for creating such a transformation involves solving or approximating a system of equations which can be time consuming and wasteful for false positives, even potentially resulting in an erroneous transformation which will break the entire label comparison process.

[0150] The prior art method of rejecting outliers is known as Random Sampling and Consensus (RANSAC). This is a "trial and error" process in which 3 or 4 sets of keypoint pairs are selected from the image. It is checked whether these keypoint pairs lie on a straight line. A transformation is computed for these points (i.e. assuming they are the correct target features). The transformation is applied to all other points. A census is taken of these points to confirm whether they are within a reasonable end distance of each other. If not (i.e. beyond a certain threshold) the trial is abandoned and alternative random points are picked and the process starts again. The points that provide the closest end distance tolerance are used to develop the transformation. This process can take a long time and can generate bad transformations when there are a large number of outlier key points.

[0151] When inspecting a label, the presence of text can create multiple non-unique key points whenever the letters repeat in a given zone. Again, large rotations and scaling of the image do not occur due to the constraints of the printing and scanning process which limit it to tiny local effects. The former is a problem and can make matching more difficult. The latter offers the possibility of simplifying the calculations in the image processing. Because the image size of the electronic image and the scanned image is known and can be matched, the positional information can be used when matching key points to each other. The positional information extracted is relative rather than absolute.

[0152] Accordingly, in accordance with another aspect of the present application, there is provided a method of matching key points extracted from an electronic image to key points of a scanned image, wherein a translation of the images is computed in both x and y displacement axes. A histogram is then created representing the frequency of key points as a function of relative displacement.

[0153] A list of actual key point pairs is formed by binning pairs that do not have a translation displacement within a predetermined small distance of the peak of the histogram (in one example within 10 pixels). Thereby, duplicate key point pairs are removed as are most outlier key points. Accordingly, the computational processing required by the RANSAC stage is significantly reduced, thus running faster and giving more reliable transformations.

[0154] Accordingly, the inspection tool defined in the inspection mask is applied to the previously identified features, blocks and zones that occur in the mask and are associated with the tool. As part of the inspection process, the label is identified on the scanned image. This can be done by identifying certain fixed features (fixed parts). Since most labels have curved corner features, the corners can be used for this purpose and a "fixed layer" is defined. Any image offset or rotation or distortion of the image during printing will be reflected in a mismatch or distortion of the expected location of the corners. Thereby, a transformation is computed and applied to the scanned image in order to provide a better match and allow an effective comparison of the scanned image and the electronic image label content.

[0155] The scanned image is then analyzed block by block or feature by feature by the previously selected inspection tool and according to associated parameters such as sensitivity, inner padding, etc. Because the appropriate tool is selected and applied to the local area, the image processing is more efficient and less time consuming than analyzing the whole image as a whole.

[0156] The text and picture blocks from the scanned image are compared to the corresponding text and picture blocks separated from the electronic image. A golden template comparison (GTC) is performed whereby the images are overlaid, transformed and, if necessary, scaled to give the best fit, then the images are subtracted from each other to give a difference image. An example of the GTC process is described in more detail in US patent 5640200.

[0157] Based on whether the difference (or mismatch) exceeds a certain threshold, the label is passed or rejected. For example, missing parts of the text or image beyond a certain size can be considered a defect.

[0158] Serialised data

[0159] In the case where there is serialised data (i.e. information specific to the label such as a serial number or patient number) on the label, the electronic image derived from the printer instructions is updated to reflect the individual incremental changes so that the electronic image information will match the printed and scanned images of the label for sequential printing. Embodiments

[0161] Figure 3 A simple example of a concept printed medical device label 10 is shown in the screenshot of Figure 1. The label comprises a generally rectangular planar sheet of paper 11 with rounded corners 12. The label has a self-adhesive back (not visible) which is attached to a backing layer 14. In the upper end region of the label there is an area of text 13 showing alphanumeric representation of product batch data. The text has a specific font design, font size and character spacing. Below the batch data there is also text 15 showing product description information. In this case the font design, font size and character spacing match that of the batch data text. In the middle region of the label there is a simple picture 16 showing a European Conformity (CE) mark. This comprises a mix of two fonts. One large font for CE and a smaller font for the number 0473 on the right hand side. In the lower region of the label is a single tone dithered grey scale picture 17 of the medical device to be labelled.

[0162] To generate an inspection mask from this label, rectangular inspection blocks 18, 19, 20, 21 are applied to each of the four texts, and corner blocks 22, 23 (Figure 2) are applied to the corners of the label. Figure 4 ) For each block, an inspection wizard script is run. Thus in Figure 5In this case, the wizard can be seen before it is applied to the label. Once the wizard has run, a series of dialogues can be viewed or modified. There is a switch to invoke the use of the wizard. The wizard uses the label format specification to identify the nature of the information in each block (i.e. text, or picture, dithered picture, barcode (if present), non-Latin script (e.g. Arabic or Chinese)). Appropriate verification tools are then applied to each block (or area). Some predetermined default parameter values are then applied to each block.

[0163] The wizard steps through each block of the label and assigns appropriate "first attempts" for the settings that will be used during the actual label verification process. This includes selecting the appropriate verification tool from the image recognition toolkit (label corner tool, text tool, graphic tool, dithered graphic tool, etc.). Various levels of sensitivity are available for each tool - low, medium and high. The search area associated with the block usually starts as "tight" extending slightly beyond the block perimeter as a rectangle. The inner margin is usually set to 0.5mm on all block sides.

[0164] Thus, the wizard initially identifies the rounded corners 22, 23 of the label, and then they are used as reference points from which the expected positions of other label features can be derived based on pre-existing knowledge of the label format. The label blocks are then evaluated and for each block the tools and sensitivity to be tried are populated. Thus for this example, the following settings are specified for the mask wizard Figure 5

[0165] • corners - top left, top right, bottom left, bottom right

[0166] o fixed part: label sheet

[0167] o fixed points

[0168] o verification tool: label corner

[0169] o sensitivity: medium

[0170] o search area: fine and long

[0171] o rectangle: 4mm x 4mm

[0172] The batch data text is then identified and the appropriate verification tool "text" is specified. The sensitivity is set to the default "medium". The search area is tight and an inner margin of 0.5mm is applied on all sides of the block:

[0173] • text - batch data

[0174] o fixed part: label sheet

[0175] o verification tool: text

[0176] o sensitivity: medium​

[0177] o Search Area: Tight

[0178] o Inner Margin: 0.5mm on all sides

[0179] Fill in similar set of settings for Product Description:

[0180] • Text - Product Description

[0181] o Fixed Part: Label Sticker

[0182] o Inspection Tool: Text

[0183] o Sensitivity: Medium

[0184] o Search Area: Tight

[0185] o Inner Margin: 0.5mm on all sides

[0186] Certification Mark Picture Invocation Details:

[0187] • Picture - CE Mark

[0188] o Fixed Part: Label Sticker

[0189] o Inspection Tool: Graphic

[0190] o Sensitivity: High

[0191] o Search Area: Tight

[0192] o Inner Margin: 0.5mm on all sides

[0193] • Picture - Product Image

[0194] o Fixed Part: Label Sticker

[0195] o Inspection Tool: Shaking Graphic

[0196] o Sensitivity: High

[0197] o Search Area: Tight

[0198] o Inner Margin: 0.5mm on all sides

[0199] In summary, the invention provides in one aspect a product label printing and inspection system comprising data processing means configured to: control printing of product labels; control acquisition and reception of images of the printed labels by image acquisition means such as an optical scanner; and inspect the printed labels for defects, wherein each printed label conforms to a label format specification for that label, whereby the labels have a common layout and comprise information relating to the printed product located in one or more areas on the label, wherein the data processing means comprises a label inspection module in which a reference image of the label is provided and the inspection module is configured such that: the acquired image is sequentially compared to the reference image in accordance with predetermined quality control indicators relating to the expected information content and location in the label areas, and wherein if the label does not conform, the label is flagged for review or rejection. The label format specification can be determined and / or stored in the form of an accessible file prior to the printing process. This can then be retrieved by the system prior to the printing process using the label format. In one aspect of the invention, the label format specification can be used to provide instructions to be sent to the printer for printing of each label. Thereby, by assembling the specification for label area locations, area information and verification tools and optionally verification tool parameters, the reference image and its associated verification mask can be configured in real time.

Claims

1. A product label printing and checking system comprising data processing means configured to: control the printing of product labels; control the acquisition and reception of images of the printed labels by means of image acquisition means such as an optical scanner; and check the printed labels for the presence of defects, wherein each printed label complying with a label format specification for the label, whereby the label has a common layout and comprises information relating to the printed product located in one or more areas on the label, wherein the data processing means comprise a label checking module in which a reference image of the label is provided and which is configured so that the acquired image is sequentially compared with the reference image according to predetermined quality control indicators relating to the expected information content and position in the label areas, and wherein if the label does not comply, the label is flagged for review or rejection, wherein the reference image is provided with a verification mask which associates areas of the reference image with verification tools stored in the system, the tools being assigned as the type of information suitable to be verified at the corresponding area on the acquired image; and wherein the data processing means are configured so that at the start of a printing run of a batch of labels, the reference image and its associated verification mask are established using the label format specification and, optionally, verification tool parameters are assigned, wherein the system is configured to perform a preliminary verification tool training process, the process being performed on the reference image and the process operating so that in the event that a verification tool fails to extract the expected features from an area, an iterative process is initiated, whereby the tool parameters are adjusted and the verification is repeated until the verification is successful, wherein the label format specification is used to provide instructions to be sent to a printer for printing each label, and wherein the reference image is an electronic image constructed according to the label format specification.

2. The system of claim 1, wherein, the system is configured to output control signals for controlling the operation of the printer.

3. The system of claim 1 or 2, wherein, the system is configured to output control signals for controlling the operation of the scanner and to receive and process input signals acquired from the image acquisition means.

4. The system of any of the preceding claims, wherein, the label format specification is determined and / or stored in the form of an accessible file prior to the printing process.

5. The system of claim 1, wherein, the reference electronic image is constructed using the instructions to be sent to the printer.

6. The system of claim 1, wherein, one or more of the verification tools are assigned to the associated areas using the label format specification.

7. The system of claim 1 or 6, wherein, at least one of the verification tools has parameter settings and, when the verification mask is formed, the parameters are set to values which optimise the performance of the tool.

8. The system of claim 1, wherein, the reference image and the trained verification tools are established in parallel with the printing of the first label.

9. The system of any one of claims 1, 6-8, wherein, the checking module is configured so that each verification area is compared with the corresponding area of the reference image in order to assess the compliance of the label as a whole. the system is configured to output control signals for controlling the operation of the printer. the system is configured to output control signals for controlling the operation of the scanner and to receive and process input signals acquired from the image acquisition means. the label format specification is determined and / or stored in the form of an accessible file prior to the printing process. the reference electronic image is constructed using the instructions to be sent to the printer. one or more of the verification tools are assigned to the associated areas using the label format specification. at least one of the verification tools has parameter settings and, when the verification mask is formed, the parameters are set to values which optimise the performance of the tool. the reference image and the trained verification tools are established in parallel with the printing of the first label. the checking module is configured so that each verification area is compared with the corresponding area of the reference image in order to assess the compliance of the label as a whole.

10. The system of claim 9, wherein, The golden template comparison is used for comparing one or more areas on the reference image and the acquired image, and preferably wherein the golden template comparison is used for one or more areas associated with a text or picture verification tool.

11. The system of claim 10, wherein, The golden template comparison is used for all areas associated with a text or picture verification tool.

12. The system of any of the preceding claims, wherein, The reference image is an electronic image structured by using information used when sending printing instructions to the printer when preparing to print the first label image.

13. The system of claim 12, wherein, The electronic image and its associated verification mask are structured to be sent in parallel with the printing instructions to the printer and to render the label image to be printed.

14. The system of claim 12 or 13, wherein, The area of the printed label contains serialization or randomization information specific to the label, and the corresponding area of the electronic image is corrected after each label printing so that, in response to a corresponding change in the label printing instructions, the changed information is used to fill the area of which the information has changed.

15. The system of any one of claims 12 to 14, wherein, The area of the electronic image of which the information has changed is associated with an optical character recognition tool or an optical character visualization tool.

16. System according to any one of the preceding claims, configured so that the reference image is adjusted to have a resolution that is approximately or preferably matched to the resolution of the scanner.

17. System according to any one of the preceding claims, configured so that, before the comparison, the reference image is subjected to a blur filter that blurs the reference image to more closely resemble the appearance of the scanned image.

18. The system of any of the preceding claims, configured such that the scan image is subjected to a geometric transformation in order to reduce or correct any distortions present in the scan image, and wherein, The transformation is applied to the scanned image before comparing the reference image with the scanned image.

19. The system of claim 18, wherein, The transformation is derived by performing a preliminary comparative analysis of the scanned image with respect to the reference image.

20. The system of any one of claims 4 to 19, configured with a text or picture verification tool, the verification tool relying on matching key points extracted from the reference image with corresponding key points of the scan image, wherein, The translation of the scanned image is calculated by recording the displacement of the key points in both the x and y axes.

21. The system of claim 20, wherein, From the recorded data, a histogram is derived that represents the frequency of key point pairs as a function of relative displacement, and wherein the peak of the histogram is taken to represent the best estimate of the accurate key point displacement.

22. The system of claim 21, wherein, From the reduced data set, repeated or spurious key point pairs are merged by merging key point pairs that do not have a translational displacement within a predetermined distance, such as within 10 pixels, of the peak of the histogram, thereby producing a reduced data set.

23. The system of claim 22, wherein, The histogram peak is recalculated using the reduced data set.

24. The system of any of the preceding claims, wherein, The data processing device comprises a single computer or a network of computers.

25. The system of any of the preceding claims, wherein, At least the acquisition or scanning control and label checking module functions are performed by a common computer.

26. The system of claim 25, wherein, The common computer is positioned adjacent to the scanner.

27. The system of any of the preceding claims, comprising: The image acquisition device is a scanner, which is integrated with the printer, or which is a detachable addition to the printer, or which is an independent device arranged separately from the printer.

28. The system of claim 27, wherein, The scanner is adapted to receive the printed medium in the form of printed product labels output from the printer and to scan it in turn.

29. A method of printing and checking product labels, comprising: Operating a system according to any one of the preceding claims.

30. Computer hardware, loaded with software and / or provided with firmware, said software and firmware being adapted to function as a product label printing and inspection system according to any one of claims 1 to 28.

31. Hardware according to claim 30, connected to a printer and to an image acquisition device such as a scanner.

32. A physical machine-readable data carrier product, loaded with software, said software being adapted to function as a product label printing and inspection system according to any one of the preceding claims when run on a computer.

Images