Method, reading device, and reading system for capturing computer readable codes - Patents.com

Abstract

The present disclosure provides a method for capturing a plurality of computer readable codes, preferably barcodes and / or two-dimensional codes, on a medical product or packaging of a medical product to assist a user, comprising the steps of capturing an image of the medical product or its packaging having said codes by a capture device (2), preferably a smartphone with a camera; extracting at least first and second computer readable codes in the captured image; interpreting the code data by an interpretation device (4) and obtaining from the code data at least information on product ID and further product information, preferably information on expiration date; and based on the obtained information, a predetermined set of related information about the medical product is extracted from the captured codes and / or linked database. selecting the second code as the associated code if the acquired information of the first code is completely included in the acquired information of the second code, or selecting the first code and the second code as the associated codes if each code contains at least part of the information of the predetermined associated information set, transmitting a code marker for each code to a display device (6) based on the results of the identifying step and the selecting step, and displaying an overlay image comprising the captured image and markings of the code according to the code markers for adding auxiliary visual information to the code to provide the user with an improved overview of the computer-readable code. Further, the present disclosure relates to a reading device, a reading system, and a computer-readable storage medium according to independent claims.

Description

[Technical field]

[0001] The present disclosure relates to a method for capturing multiple computer readable codes, preferably bar codes and / or two-dimensional codes, of a medical product or a package of a medical product to assist a user. Furthermore, the present disclosure relates to a reading device, a reading system and a computer readable storage medium according to the preamble of the independent claims. In particular, the present disclosure can be used to process information scanned from a medical product having multiple computer readable codes, preferably multiple bar codes, and from a package (article) and to identify information contained therein. Preferably, the scanner can identify relevant information depending on the intended application. One of the fields of application is the processing and documentation of medical devices. [Background technology]

[0002] Tracking and documenting medical devices as medical products used with respect to a patient or a patient's medical treatment is essential to be able to trace the device in case of an infection or device recall. The device can be any type of medical product. It is necessary to be able to determine which specific item (medical product) was utilized, e.g., implanted or used, with respect to the patient, including lot or serial number information. The lot number may also be referred to as the batch number.

[0003] Currently, manual documentation is common. It involves removing removable stickers from the equipment. Often these stickers are collected by personnel in some way. For example, they are known to be attached to surgical clothing worn during the procedure.

[0004] The seal is then removed and affixed to a form or document in a workstation utilized for documentation purposes, which is then archived or digitized.

[0005] Disadvantageously, some devices are not sealed. In this case, information such as the lot may be printed on the item, the item package (medical product package) or the package of the item set. These items are often forgotten and subsequently not documented.

[0006] The use of medical items on patients can be digitally documented in systems currently used in hospitals, such as hospital information systems (HIS), hospital management software or hospital management systems (HMS), laboratory information systems (LIS), policy and procedure management systems, radiology information systems (RIS), picture archiving and communication systems (PACS), enterprise resource planning (ERP) systems, etc. In many cases, barcode scanners are used to collect the information from the collected stickers.

[0007] There are two main standards for medical devices: global standards 1 (GS1), e.g. global trade item number (GTIN) and the health industry barcode (HIBC) system. For certain devices, the Pharmaceutical central number (PZN, in German "Pharmazentralnummer") is used. A database containing the master data of all possible materials used must be maintained in order to link the right material to the barcode that is scanned. Therefore, a materials management block is required in the information system used, or an interface is required between the information system and the materials management software.

[0008] An illustrative study showed that about 27 percent of the datasets of scanned implants in hospitals were not properly maintained. Specifically, the GTIN was missing. As a result, the barcode could be scanned but not properly processed. In such cases, users were simply shown an error message and had no option to sanitize the data other than manually entering the missing data.

[0009] This is a significant disadvantage since implants such as knee and hip replacement implants must be documented to comply with legislation, at least in some countries. Manual entry of implant data after scanning can lead to human error leading to corrupted data in the system. Incorrect data can also lead to damage claims, so having a proper database of all items in use is highly desirable.

[0010] Furthermore, some devices or batches are provided with multiple barcodes. Medical device manufacturers have some freedom in how they code the relevant information for their devices. In particular, the GS1 standards are open to different interpretations. Thus, the information presented on devices by different manufacturers is inconsistent. It is desirable to have all the relevant information in one scannable element, preferably a Data Matrix code or a similar code. A Data Matrix is ​​a two-dimensional code as shown in FIG. 2a, which requires less space than a conventional barcode. The principle of Data Matrix is ​​partly described in DE 4107020 A1. A Data Matrix with a typical size of 48 rows and 48 columns can display 348 numbers or 259 alphanumeric characters. Depending on the printing resolution, a 48x48 Data Matrix has a size of 10x10 mm to 20x20 mm. According to the definition of the currently used standard ECC200, the commonly used Data Matrix has a maximum size of 144x144 and can hold a maximum of 3116 numeric characters or 2355 alphanumeric characters.

[0011] By comparison, a traditional barcode on a book is a 13-digit number that represents the book's International Standard Book Number (ISBN), as shown in Figure 2b.

[0012] Relevant information for medical device documentation includes GTIN, expiry date, lot or serial number. However, in practice, not only this information but also further information, possibly not related to documentation, is often placed on the medical device. An example is shown in Fig. 2c to 2g. In Fig. 2c, the lowest of the three barcodes contains all the relevant information. In Fig. 2d, it is the lower of the two data matrices that should be scanned for all the relevant information. Users are often confused and embarrassed, and perform unnecessary scanning procedures. As a result, the probability of errors leading to an inaccurate data set increases. In other devices, all the relevant information is contained in several barcodes or data matrices, rather than in a single code item. In this case, conventional scanning techniques result in incomplete documentation. In Fig. 2e, the lower barcode contains the GTIN (i.e. the value after the indicator field "(01)") and expiry date data (i.e. the value after the indicator field "(17)"), while the upper barcode contains the lot number (i.e. the value after the indicator field "(10)").

[0013] When scanning a bar code, the user must adhere to a specific scanning sequence in order for the information system to properly process the scanned data. The current procedure requires the lot number to be scanned first to indicate to the information system that more information follows in addition to the GTIN. When the GTIN is scanned first, it is combined with subsequent GTINs, resulting in documentation errors with no warning or other indication to the user. Current scanning equipment requires trained, knowledgeable personnel who are aware of these issues and can scan the codes in the proper sequence. However, errors are likely to occur with the current system.

[0014] Many scanners (as capture devices) are designed to detect Data Matrix codes by scanning a rectangular area, rather than using a single target beam that highlights the computer-readable code. If multiple codes are present in this area, the scanner must identify which code should be scanned and evaluated first. A skilled operator knows that to increase scanning efficiency, the user covers the codes that should not be scanned or scanned first. Specifically, the user covers the code containing the GTIN and scans the lot number first. However, the indicator fields in the GS1 standard are not universally recognized, and different fields can be used to indicate similar information. More than 500 different so-called application identifiers are known for the GS1 standard. Figure 3 shows an excerpt of the GS1 application identifiers. Also, some users always make the mistake of scanning the long code first, or scanning the codes from top to bottom. But in reality, the length and position have no bearing on the content of the code, and the user needs to look for the application identifier "(10)" in the upper code in Figure 2f and in the lower code in Figure 2g.

[0015] (Related Technology) For example, U.S. Patent Application Publication No. 2016 / 0171258 discloses a system for detecting and processing codes on medical products. The codes are scanned and linked to a central module having a database. Product information for the medical product is retrieved from the code or the database. The user receives feedback when the required information has been read / captured or if the user needs to scan more codes. Only one code can be scanned at a time and it is only possible to identify whether that one code contains the required information.

[0016] A barcode scanner and recognition device are known from JP 2016-212603 A. The barcode scanner scans one or more codes and the recognition device interprets the codes. The recognition device links the code data to a database and measures how often a code has been read. Thus, the recognition device can prevent the same code from being scanned multiple times. The recognition device can further identify whether the code is in a two-stage format and, if so, can read the two codes in the right order / sequence.

[0017] Thus, it is known in the prior art to scan computer readable codes and determine whether required information is present. However, existing techniques do not provide a way to indicate to a user which of multiple scanned codes is the best code. Furthermore, existing techniques do not provide a way to combine product information from multiple codes to obtain required information.

[0018] BRIEF DESCRIPTION OF THE DISCLOSURE It is therefore an object of the present disclosure to be able to capture multiple computer readable codes and provide feedback to a user regarding the availability of required product data / information in the codes and the relevance of the captured codes.

[0019] The object of the present disclosure is achieved by a method for capturing one or more computer readable codes as claimed in claim 1, a reading device as claimed in claim 8, a reading system as claimed in claim 14 and a computer readable storage medium as claimed in claim 15.

[0020] The present disclosure relates to a method for capturing, preferably scanning, multiple computer readable codes, preferably bar codes and / or two-dimensional codes, on a medical product or its packaging to assist a user. The method includes the steps of capturing, preferably scanning, an image of the medical product or its packaging having the codes by a capture device, preferably a smartphone with a camera, and extracting / identifying at least a first and a second computer readable code in the captured image. The code data of the captured codes is interpreted / decoded by an interpretation device, and at least product ID / product identifier information and further product information, preferably expiry date and / or lot number, are obtained from the code data.

[0021] It will be understood that capturing or scanning an image means that a user or device takes a picture of the medical product or medical product packaging, preferably with a camera such as a smartphone camera, or scans the product or packaging with a capture device such as a camera.

[0022] It should further be understood that the code data of at least the first and second computer-readable codes may be, for example, a (digital) photograph of the code. If the capture device is capable of interpreting the computer-readable code, the capture device may also transmit code data that is interpreted data of the code.

[0023] It should be understood that product information is information about a medical product, including, for example, product ID, expiry date, lot number, etc. Product information may further include more information about the medical product, such as, for example, maximum shipping or storage period for a vaccine.

[0024] It should be further understood that interpreting the code data refers to obtaining information from the computer readable code. For example, the code may be interpreted to be identified as a GS1 standard code that includes a product identifier, which is a number or combination of numbers and letters that is read from the code.

[0025] The method of the present disclosure further includes identifying whether a predefined set of related information for the medical product is included in the captured code and / or information in the linked (master data) database based on the acquired information, and selecting the second code as a related code if the acquired information of the first code is completely included in the acquired information of the second code, or selecting the first code and the second code as related codes if each code includes at least a portion of information of the predefined set of related information.

[0026] It should be understood that the predefined relevant information is information about the medical product that needs to be scanned and digitized. The predefined relevant information may be different from the type of medical product. For example, an implant needs to be documented with a lot number, otherwise it does not meet the mandatory documentation requirements. The predefined relevant information may be defined, for example, by legal requirements, by the user, or by other requirements, such as intern requirements of a hospital.

[0027] A computer readable code(s) is selected as the relevant code(s) if it contains a predetermined set of relevant information. The relevant code may be a single code containing all the relevant information or may be a set of codes containing the relevant information in combination.

[0028] The method further includes the steps of transmitting a code marker for each code to a display device based on the results of the identifying step and the selecting step, and displaying an overlay image having the captured image and markings of the code according to the code markers for adding auxiliary visual information to the code to provide a user with an improved overview of the computer-readable code.

[0029] A code marker is additional information that is "added" to computer readable code, such as adding a description to the code indicating whether the code is a related code, an invalid code, or a code that is valid but does not contain additional product information.

[0030] An overlay image is an image that includes a captured image with a computer readable code and a marking of the code according to a code marker, the marking providing visual feedback to the user about the code, which may be, for example, a predefined colored box around the code.

[0031] In other words, the capture device may be adapted to scan one or more computer-readable codes, such as bar codes or two-dimensional codes. The capture device identifies the computer-readable codes in the captured image and transmits code data of the multiple codes to the interpretation device. The interpretation device interprets the code data. The interpretation device thereby obtains product information from the code data. The product information is information about a medical product, such as a product identifier, a lot number, and an expiration date of the medical product. The interpretation device determines whether the product information of the related code includes predetermined related information. The interpretation device further checks whether the missing product information is stored in the database. The interpretation device selects one code as a related code if the one code also includes all the product information of another code. Multiple codes can be related codes if there is no one code that includes all the product information of another code. A related code is any code that adds new product information. Thus, if multiple codes are related codes, the information included in the codes combines a predetermined related information set. The obtained product information is stored in a (master data) database. That is, the interpretation device determines whether the product information of the captured object contains the predetermined related information or whether the predetermined related information is obtainable from the database. The predetermined related information can also be obtained by combining the product information from the code data with the obtained product information from the database.

[0032] The interpreting device transmits a code marker of the relevant code to the display device. The code marker adds additional information to the code. The display device displays an overlay image to the user. The overlay image includes the captured image and the marked one or more relevant codes. The marking highlights the relevant code to the user, thereby providing the user with additional information. Depending on the identification of the interpreting device, the relevant code is marked differently. If the identification of the interpreting device is positive, the relevant code may simply be marked. According to one embodiment, the color of the marking may preferably be green. Optionally, the user receives feedback that the scan is complete. On the other hand, if the identification of the interpreting device is negative, the overlay image includes a warning or message informing the user that the relevant information is incomplete. In this case, the relevant code is also marked / highlighted since it contains product information. However, here the relevant code may be highlighted in red or orange.

[0033] In other words, the present disclosure particularly relates to a method for collecting data about the consumption of articles of medical supplies (medical products) in a procedure (such as a surgery) from computer readable codes, preferably barcodes, in an image that includes one or more barcodes. The method includes the steps of collecting barcode information of at least one computer-readable code, preferably a barcode, each computer-readable code including equipment identifier information (product ID), manufacturing identifier information (e.g., lot number or expiration date), and computer-readable code type information, preferably barcode type information; obtaining additional item information from a database based on the equipment identifier information (product ID), manufacturing identifier information (e.g., lot number or expiration date), and barcode type information; determining whether a predetermined required information set (a predetermined associated information set) regarding the item is present in the computer-readable code based on the equipment identifier information, manufacturing identifier information, barcode type information, and additional item information, and if the determination is yes, determining a minimum set of computer-readable codes, preferably a minimum set of barcodes, including the predetermined required information set regarding the item according to a predetermined rule; and transmitting marking information (code marker) for displaying an image and for providing a marking for the barcode included in the image (overlay image) based on the determination result; and storing the predetermined required information set regarding the item in a memory in association with a procedure.

[0034] The method according to the present disclosure has the following advantages: Simply, the method selects the best computer readable code from a set of multiple computer readable codes; Preferably, the best code is highlighted to the user and additional information is provided; Thus, the user knows exactly which code among the multiple codes is the best; If there is no single code on the medical product or package that contains all the relevant information, the product information from the multiple codes can be combined to gather the relevant information; A major problem of a code containing a product identifier being scanned before a code containing a lot number, linking the lot number to the wrong product, is avoided; In the present disclosure, multiple computer readable codes can be scanned simultaneously, which can ensure that the scanned codes belong to the same medical product; If information is missing in the scanned code, the user is alerted or warned; It is also possible to scan a single code; The user is given feedback whether the single code contains all the relevant information or not; Furthermore, if it is guaranteed that the information system has the appropriate usage information in real time, reordering of medical products when replenishing a certain number of items can be easily automated.

[0035] Advantageous aspects of the present disclosure are set out in the dependent claims and / or specifically described below.

[0036] Preferably, the code marker may include information indicating one or more of the color and / or shape of the overlay around the barcode and / or the text and / or symbols on or near each code. The code marker may be, for example, a box around the code. The code marker may have different colors for different codes depending on the identification result. For example, relevant codes may be colored green. Codes that are readable but do not contain further product information may be colored orange or yellow. Invalid codes may be colored red. The code marker with various information about the code assists the user. Based on the marking of the code, the user can easily determine which codes are relevant. The user is further notified if the captured code set contains a predetermined set of relevant information and is alerted if the captured code set does not contain a predetermined set of relevant information. The user can learn which codes are relevant for the future from the marking of the code.

[0037] In other words, the marking information (code markers) may in particular indicate the color and / or shape of an overlay around a computer readable code, preferably a barcode, and / or one or more of text, numbers and / or symbols on or near each code. Additionally / alternatively, different markings may be used for one or more codes identified as including a predetermined required set of information about the item, for one or more codes identified as not having the predetermined required set of information about the item, for one or more codes identified as invalid, for one or more codes identified as valid but for which no associated additional item information can be obtained from the database, and / or for one or more codes not used in the minimum code set.

[0038] Preferably, the method further comprises the steps of adding, by the interpretation device, the product information to a (master data) database, and identifying, by the interpretation device, whether the database contains predefined required information, preferably product ID, expiry date and lot number, and, if the identification is negative, assigning a warning or message to the code marker, and displaying, by the display device, to the user an overlay image with the warning or message assigned to the code.

[0039] In other words, the product information obtained from the code data may be stored in a (master data) database. After the information is added to the database, the interpretation device identifies whether the database contains the predefined required information or not. Depending on the identification result, the code marker sent to the display device is different. If the identification is positive, the code marker contains information about which code is the relevant one. The relevant code is highlighted in an overlay image. On the other hand, if the identification is negative, the code marker contains a warning or message. The warning or message is shown to the user in the overlay image. This informs the user that the code does not contain all the predefined required information.

[0040] Preferably, the interpretation device stores the acquired product information in a database and determines whether the database contains the predetermined required information. If the acquired product information contains the predetermined required information or if the required information is already stored in the database, the database is completed after the product information is stored. If the interpretation device determines that the predetermined required information is not included in either the product information or the database, the user may receive a warning or message via the display device. The interpretation device may check whether the captured code contains relevant information. If the relevant information is not included in the captured code, the interpretation device checks whether the information can be extracted from the database. If the information cannot be extracted from either the code or the database, the user receives feedback that the required information does not exist.

[0041] Preferably, the method further comprises the step of assigning a green box to each of the code markers, surrounding each of the codes, if one or more of the codes are identified as including a predetermined set of related information about the medical product. In a further step, the display device displays to the user an overlay image in which each of the codes is surrounded by a green box. If one or more of the captured codes include the predetermined set of related information, the codes are identified as related codes. The one or more related codes are marked with a green box in the overlay image. The user can easily understand that one or more related codes are present and that the scan is complete.

[0042] Preferably, the method further includes the steps of: assigning a hidden flag to hide each of the codes if one or more of the codes are identified as invalid and not compliant with a corresponding standard for computer-readable codes; and displaying a covered / hidden / dimmed overlay image of each of the codes. Non-associated codes or codes that are not standard codes may be hidden in the overlay image. A user may focus on relevant codes without being distracted by non-associated codes. A user may learn which codes are relevant for future reference.

[0043] Preferably, the method further comprises the step of linking the captured product ID to the (master data) database by the interpretation device by storing the product ID in the database or by linking the product ID of the code data of the associated code to an existing associated product ID in the database. The method further comprises the steps of adding further product information data to the database of product IDs by the interpretation device and identifying by the interpretation device whether the linked database of product IDs contains predefined required information, preferably expiry date and / or lot number. After scanning the code, the medical product is linked to the (master data) database. The interpretation device identifies whether information is available from the database. Information about which product information is required for the predefined associated information set is also available from the database. By retrieving product information from the database, the predefined associated information set can be completed even if the captured code does not contain all the required information.

[0044] Preferably, the method further comprises the steps of assigning at least a portion of the acquired information of at least the first and / or second code to a respective code marker of the code, and displaying the portion of the acquired information next to the code in an overlay image to assist the user with further information about the medical product. The code marker may include information about the information stored in the code. This may be information that the code includes a product identifier. The displayed overlay image may include a marker next to the code indicating that the code includes a product identifier. The product identifier, which is a number, may also be displayed next to the code. The user immediately understands the information each code contains. The user can also learn for the future what information is stored in a single code. Using the additional information, the user can learn the identifier of the code.

[0045] The objects of the present disclosure are further achieved by a reading device for capturing a plurality of computer readable codes on a medical product or packaging of the medical product to assist a user, the reading device comprising a capture device configured to capture an image of the medical product or its packaging having the codes and extract at least a first and a second computer readable code in the captured image.

[0046] The reading device further comprises an interpretation device. The interpretation device is configured to interpret the code data and obtain at least information of the product ID and further product information from the code data. The interpretation device is further configured to determine whether a predefined set of related information about the medical product is included in the captured code and / or in the information of a linked (master data) database stored in the memory of the reading device or in the memory of the server based on the obtained information. The interpretation device is further configured to select the second code as a related code if the information of the code data of the first code is completely included in the information of the code data of the second code, or to select the first code and the second code as related codes if each code includes at least a part of the information of the predefined set of related information.

[0047] A code marker for each code is transmitted to a display device based on the identified result and the result of the selection. The reading device further comprises a display device configured to display an overlay image having the captured image and a marking of the code responsive to the code marker for adding auxiliary visual information to the code to provide a user with an improved overview of the computer readable code.

[0048] The reading device comprises a capture device, an interpretation device and a display device. The capture device is preferably a smartphone with a camera and is configured to take a picture of the medical product or scan the code. The interpretation device is configured to identify whether the required associated information is present in the captured code and / or in a database. The interpretation device further selects the associated code and transmits at least one code marker for each code(s) to the display device. The display device displays to the user an overlay image consisting of the captured image and the code marker information embedded in the overlay image.

[0049] In other words, the present disclosure preferably provides a data evaluator (e.g., a reading device) that collects data about the consumption of an article of medical material (medical product) in a procedure (such as a surgery) from a computer readable code, preferably a barcode, in an image that includes one or more barcodes. The data evaluator comprises a collection unit (capture device) configured to collect code information of at least one computer-readable code, preferably a barcode, including equipment identifier information (product ID), production identifier information, and code type information; an acquisition unit (interpretation device) configured to acquire additional item information from a database based on the equipment identifier information, production identifier information, and code type information; an identification unit (interpretation device) configured to determine whether a predetermined required information set (predetermined relevant information set) regarding the item is present in the computer-readable code, preferably a barcode, based on the equipment identifier information, production identifier information, code type information, and additional item information, and if the determination is yes, to identify a minimum set of computer-readable codes, preferably a minimum set of barcodes, including the predetermined required information set regarding the item, according to predetermined rules; a transmission unit configured to transmit marking information (code marker) for displaying an image (overlay image) and for providing marking for the computer-readable code, preferably a barcode, included in the image, based on the determination result; and a storage unit (database) configured to store the predetermined required information set regarding the item in a memory in association with a procedure.

[0050] The reading device has the following advantages: it can scan multiple codes and identify whether the codes contain at least a part of the predefined relevant information; it can further identify the relevant code sets that contain (at least a part of) the predefined relevant information set (especially in combination with the remaining information stored in the database) and highlight them in the overlay image; it can thus select the best code from among the multiple codes or combine the multiple codes to obtain the required relevant data set.

[0051] Preferably, the interpretation device is configured to identify that the first captured code is GS1 compliant and another GS1 standard barcode could also be captured, and if the identification is positive, the display device displays feedback to the user to capture another computer readable code. Some medical products have multiple GS1 standard codes. If the product is known to have multiple codes but there is only one code in the captured image, the user is given feedback on the display device to scan another code. Preferably, the user is prompted to change the camera angle to capture multiple codes.

[0052] Preferably, the interpretation device has character recognition capabilities and is configured to link the computer readable code to a description that is not defined in the computer readable code. Some products have computer readable codes that do not indicate what information is stored. The description is rather placed next to the code and written in text. It is advantageous for the interpretation device to be able to read text. The interpretation device can link the code to the text next to it. The interpretation device can read the text, for example, by optical character recognition (OCR) or an artificial intelligence network.

[0053] Preferably, the interpretation device is configured to identify whether the captured code belongs to a sterile material or not and, if the identification is positive, to retrieve additional information about the sterile material, preferably an expiry date, from a database. It is essential that a sterile medical product or material has an expiry date. If the interpretation device identifies that the captured code belongs to a sterile medical product or material, it retrieves additional information, preferably an expiry date, from the captured code or from the database. The interpretation device identifies whether the material is expired and provides feedback to the user via the display device.

[0054] Preferably, the interpretation device is adapted to identify that all information in the predetermined relevant information set is not available and that an electrically powered medical product is provided / used, the interpretation device stops functioning, preferably powering, of the medical product and the display device displays a warning to the user. If the interpretation device identifies that all of the predetermined relevant information is not available for the electrically powered medical product, it can power down the electrically powered medical product. Alternatively, the interpretation device can display a warning to the user via the display device.

[0055] Preferably, the interpretation device has a predetermined set of selectable code markers stored in a memory unit, the code markers including information indicating one or more of the color and / or shape of an overlay around the barcode, and / or text and / or symbols on or adjacent to each code.

[0056] The present disclosure further relates to a reading system for capturing multiple computer readable codes of medical products or medical product packages to assist a user. The reading system comprises at least one reading device as a client device according to any one of the above aspects of the present disclosure, and a central server device having a storage with a (master) database and connected to the at least one reading device for transmitting and receiving data to and from the reading devices. The reading system comprises one central module for data processing and multiple reading systems connected to the central module. The central module provides a combined database of product information. For example, if one of the multiple capture devices is not operational, another capture device can be used providing the same functionality and having access to the same database and stored product information.

[0057] Preferably, the identifying step may further include identifying whether the item is associated with a procedure based on the device identifier information, the manufacturing identifier information, the barcode type information, and the additional item information. The procedure is preferably a surgery or other medical procedure.

[0058] Preferably, a separate marking may be used for the bar code or bar codes indicating that the article is not treatment related.

[0059] Preferably, the predetermined set of required information about the item (the predetermined set of relevant information) includes one or more of a unique item number, a manufacturer identifier, an item number at the manufacturer, a serial number, a lot number, and / or an expiration date.

[0060] Preferably, the additional item (medical product) information includes a predetermined set of required information about the item, and / or the predetermined set of required information about the item includes one or more of a serial number, a lot number, and / or an expiration date.

[0061] Preferably, the method further comprises one or more of the steps of acquiring an image, detecting a barcode in the image, decoding the detected barcode based on a barcode encoding table, interpreting the decoded barcode based on an interpretation table, and / or displaying the acquired image together with the marking.

[0062] Preferably, the method further includes the step of prompting the user and receiving additional input from the user if at least one invalid (bar) code is identified, if a predetermined required set of information regarding the item is identified as not being present in the (bar) code, and / or the method is properly terminated.

[0063] Preferably, the displaying step further comprises the step of displaying an interactive element, preferably based on the barcode information and / or the additional item information.

[0064] Preferably, the method repeats the steps of processing a number of images, preferably after a predetermined time and / or user confirmation.

[0065] Preferably, the marking information (code markers) may indicate one or more of the color and / or shape of an overlay around the barcode and / or text, numbers and / or symbols on or near each code. Additionally / alternatively, different markings may be used for one or more codes identified as including a predetermined required set of information about the item, for one or more codes identified as not having the predetermined required set of information about the item, for one or more codes identified as invalid, for one or more codes identified as valid but for which no associated additional item information can be obtained from the database, and / or for one or more codes not used in the minimum code set.

[0066] Preferably, the identifying unit (interpretation device) is further configured to identify whether the item is relevant for the procedure based on the device identifier information, the manufacturing identifier information, the barcode type information and the additional item information.

[0067] Preferably, a separate marking may be used in the bar code or bar codes to indicate that the article is not treatment related.

[0068] Preferably, the predetermined set of required information about the item includes one or more of a unique item number, a manufacturer identifier, an item number at the manufacturer, a serial number, a lot number, and / or an expiration date.

[0069] Preferably, the additional item information includes a predetermined set of required information about the item and / or the predetermined set of required information about the item includes one or more of a serial number, a lot number, and / or an expiry date.

[0070] Preferably, there is provided a data evaluator comprising one or more of a collection unit configured to collect an image, a detection unit configured to detect a barcode in the image, a decoding unit configured to decode the detected barcode based on a barcode encoding table, an interpretation unit configured to interpret the decoded barcode based on an interpretation table, and / or a display unit configured to display the collected image together with the marking.

[0071] Preferably, the data evaluator further comprises an interaction unit configured to prompt the user and receive additional input from the user if at least one invalid (bar) code is identified, if a predetermined required set of information regarding the item is identified not to be present in the (bar) code, and / or if data collection from the image has been properly terminated.

[0072] Preferably, the data evaluator further comprises a display configured to display the interactive element, preferably based on the barcode information and / or the additional item information.

[0073] Preferably, the data evaluator is arranged to process a number of images sequentially, preferably proceeding to the next image after a predetermined time and / or user confirmation.

[0074] Furthermore, the present disclosure relates to a computer program product comprising a program for a processing device, the program comprising software code portions for carrying out the method according to the above aspects when the program is executed on the processing device.

[0075] Preferably, the computer program product comprises a computer readable medium on which the software code portions are stored, the program being directly loadable into the internal memory of a processing device.

[0076] The present disclosure further relates to a computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform a method according to any of the above aspects.

[0077] Any discussion of the method of capturing computer readable code according to this disclosure equally applies to any discussion of the reading device according to this disclosure, and vice versa. [Brief description of the drawings]

[0078] The present disclosure is explained in more detail below with reference to preferred embodiments using the figures. [Figure 1] 1 is a flow chart illustrating a method of the present disclosure according to one embodiment. [Figure 2a] FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Figure 2b] FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Figure 2c]FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Figure 2d] FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Figure 2e] FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Figure 2f] FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Figure 2g] FIG. 13 is a diagram showing an example of code on a device to facilitate understanding. [Diagram 3] To facilitate understanding, here is a list of GS1 Standard Application Identifiers. [Figure 4a] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4b] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4c] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4d] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4e] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4f] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4g] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4h] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4i] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4j] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4k] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4l] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4m] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4n] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4o] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4p] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4q] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4r] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4s] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4t] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4u] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4v] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4w] FIG. 1 illustrates different product coding formats to aid understanding. [Figure 4x] FIG. 1 illustrates different product coding formats to aid understanding. [Diagram 5] 1 is an exemplary barcode decoding table for ease of understanding. [Figure 6] FIG. 1 is a schematic diagram illustrating a reading device according to a preferred embodiment of the present disclosure.

[0079] The drawings are schematic and are intended only to aid in the understanding of the invention. Features of the various embodiments are interchangeable.

[0080] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIGURE In the following, examples of the present disclosure will be described in detail with reference to the accompanying drawings.

[0081] 1 shows a flow chart of a method 1000 for capturing multiple computer readable codes on a medical product or a package of a medical product to assist a user. In this embodiment, the method is applied to capturing barcodes and QR codes. The method includes the following steps:

[0082] In a first step 1010, the method captures the medical product or its packaging having the code by a camera as a capture device 2.

[0083] In a second step 1020, the method extracts at least a first barcode or QR code and a second barcode or QR code in the captured image.

[0084] The method then proceeds to step 1030, where it interprets the code data by the interpretation device 4 and obtains from the code data at least the information of the product ID and further product information in the form of an expiry date.

[0085] In step 1040, based on the obtained information, it is determined whether the captured code and / or information in the linked database 8 includes a predefined set of relevant information about the medical product.

[0086] Step 1050 includes selecting the second code as the related code if the obtained information of the first code is completely contained in the obtained information of the second code, or selecting the first code and the second code as the related codes if each code contains at least a portion of a predetermined set of related information.

[0087] Once the identification and selection are complete, the process proceeds to step 1060 where a code marker for each code is sent to the display device 6 based on the identification and selection results.

[0088] Finally, the process proceeds to step 1070, which displays an overlay image having the captured image and markings of the code according to the code markers to add auxiliary visual information to the code to provide the user with an improved overview of the computer readable code.

[0089] Such a method allows to add further information to each code and to assist the user. The method is in particular an augmented reality method. In this method, real world information is supplemented by digital world information, so that relevant information can be effectively added where needed. The method is capable of capturing a number of computer readable codes, automatically checks and interprets each computer readable code without the user having the necessary knowledge, and presents the results of the data analysis to the user in the form of additional information displayed to facilitate daily work, in particular in a hospital. Furthermore, the automatic selection performed by the method increases patient safety and workflow speed in the hospital. In particular, the method may be applied to automatically add information contained in the code or at least part of it, for example an expiry date, to an overlay image (real world / digital world) in order to intuitively indicate the required information to the user. This reduces the user's workload, since the user can focus on his / her specific task and perform it better. Furthermore, since the method is performed automatically by the machine and does not require user assistance or manual input, the possibility of errors is minimized.

[0090] Primarily, the particular QR code or barcode that is the best code that conveys the best information is presented to the user. This allows, for example, the user to intuitively understand which barcode he should pay attention to. Furthermore, if multiple computer-readable codes contain a minimum set of relevant information, those barcodes are accurately shown to the user. For example, if one of seven barcodes contains expiration date information, that code among the seven codes is highlighted and displayed to the user. This allows the user to identify whether the expiration date is reached at that time. Alternatively, a warning message may be displayed, highlighting the barcode in a special color, such as red, to inform the user that the expiration date has passed. This allows the user to select the medical product. The method provides the user with an improved overview of relevant information in each specific situation where it is applicable (front-end improvement).

[0091] On the other hand, current scanning devices can only collect images of objects placed in front of the scanner and detect and decode barcodes therein. Each barcode is decoded based on its code table. In the case of a barcode, each coded character is coded with the barcode's bar and space configuration. For example, a character according to CODE39 consists of five bars and four spaces. Of the total nine elements, three are thick (corresponding to a binary value of 1) and six are thin (corresponding to a binary value of 0). It codes for 43 different uppercase alphabets, numbers, and symbols and spaces, and has the ability to code all 128 ASCII characters. An example of a decoding table for CODE39 is shown in Figure 5. The decoding unit processes the detected code and returns the decoded string.

[0092] From the database, data can be obtained that includes what is needed to make up a complete information set. For example, the database may include that in the case of medical products, an expiration date is required, or in the case of vaccines, a shipping period that must not be exceeded. Such constraints can be added from the database. Additionally, changed requirements can be loaded in a similar manner. For example, recall information may be added to the database at the time of shipping, which will result in the scanned item being marked as "expired" or "recalled," thereby avoiding its use.

[0093] Optionally, the display may indicate to the user which code or codes will ultimately be used since the code or codes contain all the necessary information. This may be achieved, for example, by a colored or black and white overlay over the code(s), a colored or black and white frame around the code(s), a symbol such as a dot, star, or arrow next to the code(s), or any other visible indication. The remaining information may then be embellished by additional information selected from the master data of the database. Examples of additional information include manufacturer information, hospital item number, or item description.

[0094] However, marking of devices is not always done in a form where the information is contained in a single barcode or data matrix. GS1 standards allow the information to be distributed among several barcodes. When using a traditional barcode scanner, the user must decide which code to scan first, or leave that decision to the barcode scanner. However, in these cases, there is a high possibility of errors in the documentation, as detailed above.

[0095] According to the invention, multiple codes can be read simultaneously. If the complete information set is not available from a single code, two or more codes can be combined as described above.

[0096] The invention aims to make documentation and assistance as easy as possible for the user. The display can therefore cooperate with different colour brushes, e.g. overlays, so that the user becomes aware and aware of the different coding standards over time. Master data can be used to define how the item should be coded, which codes are expected and what information should be documented. A brush in this case is a coloured overlay showing the marking of a particular area, like a brush for highlighting paper.

[0097] Below some examples of medical device coding are detailed with reference to figures 4a to 4z, it should be noted that the same principles can be applied to multi-disciplinary devices as well.

[0098] As an example of a medical device, an implant is detailed below. It is common for implants to have coded information regarding GTIN / LIC+PNC, expiry date, lot or serial number. Known coding formats for medical devices include: Pharmaceutical central number (PZN, in German "Pharmazentralnummer") (shown in Figure 4a) EAN Code (European Article Number) (EAN13 +PZN) (shown in Figure 4b) EAN13 (shown in Figure 4c) GS1(01) (shown in Figure 4d) GS1(01)+GS1(17 / 10) (shown in Figures 4e and 4f) GS1(01 / 17 / 10)+EAN 13 (shown in Figure 4g) GS1(* / 10)+EAN 13 (shown in Figure 4h) GS1 (01 / 17 / 10) (shown in Figures 4i and 4j) GS1 (01 / 17 / 21) (shown in Figure 4k) GS1+HIBC (shown in Figure 4l) HIBC (shown in Figures 4m and 4n) Non-standardized codes (shown in Figures 4o and 4p) OCR combined with barcode (as shown in Figure 4q) OCR only (as shown in Figure 4r) Sterile products (shown in Figures 4s, 4t, and 4u) Card number (as shown in Figure 4v) Document or resterilize implants Electronic instructions for use (eIFU) (shown in Figure 4w) QR Code (shown in Figure 4x) · Expiry date proof.

[0099] In the above, OCR, or optical character recognition, functionality includes recognizing text containing multiple individual characters, and also recognizing symbols, specifically graphic symbols such as the "lot" designation in Figure 4p or Figure 4q, or the hourglass symbol in Figure 4q, and thus may also be referred to as image recognition and / or may include machine learning to learn the meaning of commonly used symbols.

[0100] Figure 4a shows the PZN. In Germany, medicines are given a Pharmaceutical Product Number (PZN). In the EU, the Pharmaceutical Product Number (PPN) is used. The PZN or the PPN standard has no priority for implant documentation or reordering. However, for proper tracking and in-house restocking, these codes need to be detected and properly interpreted. The PZN should be marked in a different manner than the GS1 code(s), i.e. with a different color, shape and / or symbol.

[0101] Figure 4b shows the combination EAN13+PZN. Since PZN does not correspond to EAN13, both codes must be interpreted to identify the goods. When creating a new master data, the user should be asked if the goods contain another machine-readable code. The combination of PZN and EAN13 is important so that when ordering, the data can be sent correctly to goods management. Once the master data has been created, the goods should be identified by PZN or EAN13. The PZN should be marked in a different manner than the GS1 code(s), i.e. with a different color, shape and / or symbol.

[0102] Figure 4c shows EAN13. For articles with EAN13, it is assumed that the expiry date and lot or serial number are entered manually. When scanning and thus creating a new master data, the user should be asked if the article contains another machine-readable code and if the expiry date and lot or serial number should be documented. In general, manual entry should be possible, but mandatory if necessary for the device. For example, manually recording a lot number for a bandage is inappropriate for a device. Also, documenting an implant without a lot number does not fulfill the documentation requirement. This is detailed below with respect to the different GS1 coding formats.

[0103] Figure 4d shows a GS1 with identifier (01). There is a device with a barcode containing a GTIN with identifier (01) and no other codes. A prior art scanner can process an EAN13 immediately, but after scanning a GS1 (01) code, it waits to process another scanned code until there is a user interaction, for example pressing a "Proceed" button. In this case, the master data may indicate that no further codes are expected for this device. Manual entry of lot information may also rely on the master data.

[0104] Figures 4e and 4f show GS1 with identifiers other than identifier (01). If the product is known to have multiple GS1 codes, the user is prompted to change the camera perspective to capture multiple codes.

[0105] Figure 4g shows a GS1 code and an EAN13 on the same package. If the product is known to have a complete GS1 code present next to the EAN13, the user is prompted to scan the GS1 code. In this case, the EAN13 should not be processed.

[0106] Figure 4h shows an item with an EAN13 rather than a GS1 identifier (01). If the master data defines that the product has a second code or that a lot number needs to be scanned, the reading device will not process the EAN13 and will wait until the lot number is scanned.

[0107] Figure 4i to 4k show codes that contain the necessary related information set in one code. Figure 4l shows HIBC code. HIBC code should be marked in a different color than GS1 code. Medical products should be identified by HIBC code or GS1 code. Figure 4m and 4n also show HIBC code. From one single HIBC code, product ID, lot number and serial number should be retrievable.

[0108] Figures 4o to 4r show non-standardized codes. The code in Figure 4o should not be used for documentation. A reading device can read the text to identify what information the codes in Figures 4p and 4q contain. Figure 4r does not contain any computer readable code. To process the information in Figure 4r, a reading device may use character recognition (OCR) or take a photo to document the product. Figure 4v also does not belong to any standard. The information in it may be obtained by OCR or documented by taking a photo.

[0109] Figures 4s to 4u show codes for sterile materials. A reading device can identify that the medical product is a sterile material and obtain, for example, the expiry date of the same.

[0110] Figures 4w and 4x show code that leads to a website or instructions with product information that is opened by a reading device and displayed to the user on a display device.

[0111] FIG. 6 shows a schematic diagram of a reading device 1 according to a preferred embodiment of the present disclosure. The reading device 1 comprises a capture device 2, an interpretation device 4, a display device 6 and a database 8. The capture device 2 is preferably a camera, particularly preferably a smartphone camera. A user or a device uses the capture device 2 to capture an image of a barcode set or to scan a barcode set. The barcode is extracted from or detected in the captured image. The interpretation device 4 decodes the extracted barcode and detects which product information is stored in a single barcode. The interpretation device 4 then identifies whether a given set of related information is available in the barcode or in the linked database 8. The interpretation device 4 selects the related code and highlights the related code on the display device 6.

[0112] In a specific embodiment, a method for capturing multiple computer-readable codes on a medical product or a package of a medical product for user task assistance is described below. The capture device 2 is a camera, preferably a smartphone camera. The user scans a set of computer-readable codes with the camera. The set of computer-readable codes is, for example, a barcode set, as shown in Figs. 2c and 2d. The camera captures an image of the computer-readable codes. In the captured image, the barcode is detected. The interpretation device 4 decodes the code data stored in the captured barcode. That is, the interpretation device 4 decodes the bars in the barcode based on an interpretation table, as shown in Fig. 5. The interpretation device 4 further interprets what information is stored in the barcode.

[0113] In the example of FIG. 2c, the interpretation device 4 can recognize that the top barcode contains the product ID, the middle barcode contains expiry date data and a lot number, whereas the bottom barcode contains the product ID, expiry date data and a lot number. The interpretation device 4 can link the medical product to a database by its product ID and can determine whether the captured barcode and / or the database is provided with a predefined set of related information. The interpretation device 4 can obtain information which product information is required to obtain a complete set of related information. The information is obtained from the database 8 or is stored in the interpretation device 4. Furthermore, product information, such as sterilization date, maximum transport period or maximum storage period, can be stored in the database 8. In this case, the interpretation device 4 determines that all the required information is available in the barcode since the product ID, expiry date and lot number are provided.

[0114] The interpreting device then selects one or more associated codes. In FIG. 2c, the bottom barcode is the associated code since it contains all the necessary product information and no other code can add information that is not stored in the bottom barcode. Based on the positive identification and that the associated codes have been selected, the interpreting device 4 sends a code marker for all codes to the display device 6. In this case, the code marker is, for example, a green box marking the bottom code as an associated code. The other two barcodes are highlighted with yellow or orange boxes to indicate that they are valid and GS1 compliant but are not associated codes. The user further receives visual feedback, for example that all associated information is available and the scan is complete.

[0115] Another example of a barcode is shown in Fig. 2d. The interpretation device 4 detects that in these barcodes, the upper barcode contains the product ID. It is marked with the (01) identifier according to the GS1 standard. The lower barcode contains the lot number (10) and the expiry date (17). The interpretation device 4 determines that a predefined set of related information is available in the captured barcode. A positive determination is therefore made. Since no barcode contains all the necessary product information by itself, the interpretation device 4 selects both barcodes as related codes. The interpretation device 4 therefore combines the information obtained from the two barcodes to obtain the predefined set of related information. As a result, both barcodes are marked in green on the display device 6.

[0116] If the interpretation device 4 determines that a barcode set does not have a predefined set of associated information present in the barcode, the display device 6 will display a warning or message to the user indicating that the data is incomplete, and the user will receive a request to scan more barcodes.

[0117] Particularly if the reading device is a mobile device with a camera and a display, the user can easily interact with the reading device and it can be used before or during surgery.

[0118] Although some aspects have been described in the context of an apparatus, it is understood that these aspects also represent a description of a corresponding method, with the apparatus or parts thereof corresponding to method steps or features of method steps. Similarly, aspects described in the context of a method step also represent a description of a corresponding apparatus or part of an apparatus or part of an article or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, such an apparatus may perform one or more of the most important method steps. The following items were included in the claims as originally filed: (Item 1) 1. A method for capturing a plurality of computer readable codes, preferably bar codes and / or two-dimensional codes, of a medical product or a package of a medical product for assisting a user, comprising: Capturing an image of said medical product or its packaging carrying said code by means of a capture device (2), preferably a smartphone with a camera; extracting at least first and second computer readable codes within the captured image; - interpreting said code data by an interpretation device (4) and obtaining from said code data at least product ID information and further product information, preferably expiry information; determining whether a predetermined set of relevant information about the medical product is included in the captured code and / or information in a linked database (8) based on the obtained information; selecting the second code as an associated code if the acquired information of the first code is completely contained in the acquired information of the second code, or selecting the first code and the second code as the associated codes if each code contains at least a portion of information of the predetermined associated information set; transmitting a code marker for each code to a display device (6) based on the results of the identifying step and the selecting step; displaying an overlay image having the captured image and markings of the code responsive to the code markers for adding auxiliary visual information to the code to provide the user with an improved overview of the computer readable code. (Item 2) 2. The method of claim 1, wherein the code marker includes information indicating the color and / or shape of an overlay around the barcode and / or text and / or symbols on or near each code. (Item 3) adding said product information to said (master data) database by said interpretation device (4); determining by said interpretation device (4) whether said database (8) contains said predetermined required information, preferably said product ID, expiry date and lot number; if said identification is negative, assigning a warning or message to said code marker; displaying to the user, by the display device (6), the overlay image in which the warning or message is assigned to the code; 3. The method according to item 1 or 2, further comprising: (Item 4) assigning a green box to each of the code markers, the green box surrounding each of the codes if one or more of the codes are identified as including the predetermined set of associated information about the medical product; displaying to the user, by the display device (6), the overlay image in which each of the codes is surrounded by a green box; 4. The method according to any one of items 1 to 3, further comprising: (Item 5) assigning a hidden flag to each of said codes if one or more of said codes are identified as invalid and not compliant with a standard for computer readable codes; displaying said overlay image with each of said codes covered / hidden / dimmed; 5. The method according to any one of items 1 to 4, further comprising: (Item 6) linking the obtained product ID to the database (8) by the interpretation device (4) by storing the product ID in the database or by linking the product ID of the code data of the related code to an existing related product ID in the database (8); adding, by said interpretation device (4), said further product information data to said database of said product IDs; determining by said interpretation device (4) whether said database (8) of linked product IDs contains said predetermined required information, preferably an expiry date and / or a lot number; 4. The method according to item 3, further comprising: (Item 7) - assigning at least a portion of the obtained information of the at least first and / or second code to the code marker of each of the codes; displaying the portion of the obtained information next to the code in the overlay image to assist the user with further information regarding the medical product; 7. The method according to any one of items 1 to 6, further comprising: (Item 8) A reading device (1) for capturing a number of computer readable codes on a medical product or on a package of the medical product for assisting a user, comprising: a capture device (2) configured to capture an image of the medical product or its packaging having the code and to extract at least first and second computer readable codes within the captured image; An interpretation device (4), interpreting the code data to obtain at least product ID information and further product information from the code data; Identifying, based on the acquired information, whether a predetermined set of relevant information about the medical product is included in the captured code and / or in information in a linked (master data) database (8) stored in the memory of the reading device or in the memory of a server; Selecting the second code as an associated code if the information of the code data of the first code is completely contained in the information of the code data of the second code, or selecting the first code and the second code as the associated codes if each code contains at least a portion of the information of the predetermined associated information set; an interpretation device (4) configured to transmit a code marker for each code to a display device based on the identification result and the selection result; a display device (6) configured to display an overlay image having the captured image and a marking of the code in response to the code markers for adding auxiliary visual information to the code to provide the user with an improved overview of the computer readable code; A reading device (1). (Item 9) The reading device (1) of item 8, characterized in that the interpretation device is configured to identify that the first captured code complies with the GS1 standard and that another GS1 standard barcode may also be captured, and if the identification is positive, the display device displays feedback to the user to capture another computer-readable code. (Item 10) 10. The reading device (1) according to item 8 or 9, characterized in that the interpretation device (4) has character recognition capabilities and is configured to link a computer readable code to a description not defined in the computer readable code. (Item 11) 11. The reading device (1) according to any one of items 8 to 10, characterized in that the interpretation device (4) determines whether the captured code belongs to a sterile material or not and, if the determination is positive, the interpretation device (4) obtains additional information about the sterile material from the database, preferably the expiry date. (Item 12) 12. The reading device (1) according to any one of claims 8 to 11, characterized in that the interpretation device (4) is adapted to identify that not all information of the predetermined relevant information set is available and that an electrically powered medical product is provided / used, the interpretation device (4) stopping the functioning, preferably the power supply, of the medical product and the display device (6) displaying a warning to the user. (Item 13) 12. The reading device (1) according to any one of claims 8 to 11, characterized in that the interpretation device (4) has a predefined set of selectable code markers stored in the memory, the code markers including information indicating the color and / or shape of an overlay around the barcode and / or the text and / or symbols on or near each code. (Item 14) A reading system (100) for capturing a plurality of computer readable codes on a medical product or a package of the medical product for assisting a user, comprising: At least one reading device (1) according to any one of items 8 to 13 as a client device; a central server device connected to said at least one reading device (1) and having a storage with a (master) database (8) for transmitting and receiving data to and from said reading device (1); A reading system (100) comprising: (Item 15) 8. A computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of items 1 to 7. [Explanation of symbols]

[0119] 1: Reading device 2: Capture device 4: Interpretation equipment 6: Display equipment 8: Database 100: Reading system 1000: Method 1010: Capturing an image 1020: Step of extracting code from within an image 1030: Steps to interpret the code 1040: Identifying a Predetermined Information Set 1050: Steps to select a code 1060: Transmitting code markers 1070: Step of displaying an overlay image

Claims

1. 1. A method for capturing a plurality of computer readable codes on a medical product or packaging of the medical product to assist a user, comprising: capturing an image of the medical product or the packaging of the medical product with a capture device; extracting at least a first computer readable code and a second computer readable code in the captured image; interpreting the first computer readable code and the second computer readable code with an interpretation device and obtaining at least product identification information and further product information associated with each of the first computer readable code and the second computer readable code; determining whether a predetermined set of associated information about the medical product is included in information and / or a linked database associated with each of the first computer readable code and the second computer readable code based on the obtained information; selecting the second computer readable code as an associated code if the obtained information associated with the first computer readable code is completely contained in the obtained information associated with the second computer readable code, or selecting the first computer readable code and the second computer readable code as the associated codes if each of the information associated with the first computer readable code and the second computer readable code includes at least a portion of information of the predetermined associated information set; transmitting a code marker for each of the first computer readable code and the second computer readable code to a display device based on a result of the identifying step and a result of the selecting step; displaying an overlay image having the captured image and markings of each of the first computer readable code and the second computer readable code in response to the code markers for adding auxiliary visual information to each of the first computer readable code and the second computer readable code to provide the user with an improved overview of the first computer readable code and the second computer readable code.

2. 2. The method of claim 1 , wherein one or more of the code markers include a color and / or shape of an overlay around the barcode and / or information indicative of text and / or symbols on or near each of the first computer readable code and the second computer readable code.

3. adding, by said interpretation device, said product information to said database; determining, by said interpretation device, whether said database contains said predetermined required information; if said identification is negative, assigning a warning or message to at least one of said code markers of said first computer readable code and said second computer readable code; displaying, by the display device, the overlay image to the user, the warning or message being assigned to at least one of the first computer readable code and the second computer readable code; The method of claim 1 further comprising:

4. linking, by said interpretation device, said obtained product ID to said database by storing said product ID in said database or by linking said product ID of said association code to an existing associated product ID in said database; adding, by said interpretation device, said further product information data to said database of product IDs; determining, by said interpreting device, whether said database of linked product IDs contains said predetermined required information; The method of claim 3 further comprising:

5. Identifying each of the first computer readable code and the second computer readable code having respective associated information, the respective associated information including the predetermined set of associated information about the medical product; displaying the overlay image having a green box surrounding each of the first computer readable code and the second computer readable code having respective associated information identified as comprising the predetermined set of associated information about the medical product; The method of claim 1 further comprising:

6. A method of identifying each of the first computer-readable code and the second computer-readable code, each of which has associated therewith information that has been identified as invalid because it does not conform to a standard corresponding to the computer-readable code; displaying the overlay image with each of the first and second computer readable codes having associated therewith information that has been identified as invalid because it does not conform to a standard corresponding to the computer readable code being covered / hidden / dimmed; The method of claim 1 further comprising:

7. A method of the present invention comprising the steps of: assigning at least a portion of information associated with each of at least one of the first computer readable code and the second computer readable code to the code marker of each of the first computer readable code and the second computer readable code; displaying the portion of information next to each of the first computer readable code and the second computer readable code in the overlay image to assist the user with further information regarding the medical product; The method of claim 1 further comprising:

8. The method of claim 1 , wherein the plurality of computer readable codes are bar codes and / or two-dimensional codes.

9. The method of claim 1 , wherein the capture device is a smartphone having a camera.

10. The method of claim 1 , wherein the predetermined required information includes the product ID, expiration date, and / or lot number.

11. The method of claim 1, wherein the product information is an expiration date.

12. 1. A reading device for capturing a plurality of computer readable codes on a medical product or a package of the medical product to assist a user, comprising: a capture device configured to capture an image of the medical product or the packaging of the medical product and extract at least a first computer readable code and a second computer readable code within the captured image; An interpretation device, interpreting the first computer readable code and the second computer readable code to obtain at least product identification information and further product information associated with each of the first computer readable code and the second computer readable code; Identifying, based on the obtained information, whether a predetermined set of associated information about the medical product is included in information associated with each of the first computer readable code and the second computer readable code and / or in a linked database stored in a memory of the reading device or a memory of a server; selecting the second computer readable code as an associated code if information associated with the first computer readable code is completely contained in information associated with the second computer readable code, or selecting the first computer readable code and the second computer readable code as the associated codes if information associated with the first computer readable code and the second computer readable code each contain at least a portion of information of the predetermined associated information set; an interpretation device configured to transmit a code marker of each of the first computer readable code and the second computer readable code to a display device based on the identifying result and the selecting result; a display device configured to display an overlay image having the captured image and a marking of each of the first computer readable code and the second computer readable code in response to the code markers for adding auxiliary visual information to each of the first computer readable code and the second computer readable code to provide the user with an improved overview of the first computer readable code and the second computer readable code; A reading device comprising:

13. 13. The reading device of claim 12, wherein the interpretation device is configured to determine whether information associated with at least one of the first computer-readable code and the second computer-readable code conforms to the GS1 standard and another GS1 standard barcode may also be captured, and if the determination is positive, the display device displays feedback to the user to capture another computer-readable code.

14. 13. The reading device of claim 12, wherein the interpretation device has character recognition capabilities and is configured to link at least one of the first computer readable code and the second computer readable code to a description not defined in the first computer readable code and the second computer readable code.

15. The reading device of claim 12, wherein the interpretation device determines whether information associated with at least one of the first computer readable code and the second computer readable code belongs to a sterile material, and if the determination is positive, the interpretation device obtains additional information about the sterile material from the database.

16. 13. The reading device of claim 12, wherein the interpretation device is adapted to, upon determining that information associated with each of the first computer readable code and the second computer readable code does not include all information within the predetermined set of associated information, disable an associated powered medical product and display a warning to the user via the display device.

17. 13. The reading device of claim 12, wherein the interpretation device has a predetermined set of selectable code markers stored in the memory, the code markers including information indicative of the color and / or shape of an overlay around the barcode and / or text and / or symbols on or near each code.

18. A reading device as described in claim 15, wherein the additional information about the sterile material is an expiration date.

19. The reading device of claim 16, wherein the function of the medical product is a power source.

20. A reading device as described in claim 12, further comprising a central server device having a storage with a database and connected to the at least one reading device for transmitting and receiving data to and from the at least one reading device.

21. When executed by a computer, capturing an image of the medical product or packaging of said medical product with a capture device; extracting at least a first computer readable code and a second computer readable code in the captured image; interpreting the first computer readable code and the second computer readable code with an interpretation device and obtaining at least product identification information and further product information associated with each of the first computer readable code and the second computer readable code; determining whether a predetermined set of associated information about the medical product is included in information and / or a linked database associated with each of the first computer readable code and the second computer readable code based on the obtained information; selecting the second computer readable code as an associated code if the obtained information associated with the first computer readable code is completely contained in the obtained information associated with the second computer readable code, or selecting the first computer readable code and the second computer readable code as the associated codes if each of the information associated with the first computer readable code and the second computer readable code includes at least a portion of information of the predetermined associated information set; transmitting a code marker for each of the first computer readable code and the second computer readable code to a display device based on a result of the identifying step and a result of the selecting step; and displaying an overlay image having the captured image and markings of each of the first and second computer readable codes in response to the code markers for adding auxiliary visual information to each of the first and second computer readable codes to provide a user with an improved overview of the first and second computer readable codes.

Images