Operator terminal and monitor for automation

Abstract

The invention relates to a Human-Machine-Interface (HMI) system (7) designed to enhance user interaction through the integration of barcodes (4) into clickable buttons (5) within a graphical user interface (GUI) (6) on a display (8) connected to a computer (2) and a method to use this system (7). The system allows a user (1) to activate actions (9) by scanning the integrated barcode (4) with a barcode reader (3), which transfers the obtained data (10) to the computer (2) and converts it into a click event. This method streamlines processes by reducing manual input, particularly beneficial in environments requiring high traceability, such as logistical processes for bioprocessing. The system (7) supports various barcode types, including two-dimensional Data-Matrix-Codes and QR-Codes, and can be configured with both wired and wireless connections. The GUI (6) accommodates multiple buttons (5) capable of performing diverse functions, enhancing operational flexibility and efficiency. It offers a seamless solution for maintaining workflow continuity and accuracy in industrial applications.

Description

[0001] Foreignfiling text P24-215

[0002] - 1 -

[0003] Operator terminal and monitor for automation

[0004] The hereby described invention discloses a method and a system for to perform a Human-Machine-Interaction process using a graphical user

[0005] 5 interface with an integrated barcode.

[0006] Technical Field

[0007] The invention deals with the technological area of bioprocess consumable logistics.

[0008] Background and description of the prior art

[0009] In the field of human-machine interaction, particularly within the context of

[0010] 15 logistical processes in Biomonitoring, e.g. for monitoring of clean rooms, the integration of barcode technology into user interfaces has emerged as a pivotal innovation. Barcodes serve as a means of encoding information that can be easily scanned and interpreted by various devices, facilitating efficient data entry and retrieval. The use of barcodes is widespread across

[0011] 20 industries, enabling traceability and management of consumables throughout supply chains.

[0012] Existing patents, such as US9047385B1 , describe interactive barcode systems that enhance user engagement by providing visual feedback through graphical user interfaces (GUIs). This patent outlines methods for associating barcodes with specific actions, allowing users to interact with digital systems more intuitively. For instance, barcodes can be scanned to retrieve information or confirm actions, thus streamlining workflows in environments where speed and accuracy are critical.

[0013] 30

[0014] Another relevant patent, US10915721 B1 , focuses on mobile computing devices that communicate with barcode scanners to provide real-time Foreignfiling text P24-215

[0015] - 2 - feedback during scanning operations. This technology is particularly useful in healthcare settings, where barcode scanners are utilized to manage patient information and medication administration. The integration of feedback mechanisms — such as visual indicators on mobile devices —

[0016] 5 ensures that healthcare workers receive immediate confirmation of successful scans, thereby reducing errors and improving operational efficiency.

[0017] Despite these advancements, challenges remain in achieving seamless human-machine interaction, especially when users must navigate between physical actions, like scanning a barcode, and digital responses, like confirming an action on a touch- / screen. In particular at a GUI panel the user has always to free his hands in order to be able to confirm an action or select a menu via the keyboard or on the touch-screen. This is more and

[0018] 15 more the case with the increase of traceability where the user has to handle a barcode scanner during execution of action to trace for example consumables or hardware equipment used during the process.

[0019] Therefore need for a more integrated approach that combines the physical

[0020] 20 interaction of scanning with the digital confirmation of actions is evident, particularly in processes that require high accuracy and traceability.

[0021] The task of this patent application is therefore to find an efficient way to remote control the environmental settings of clean rooms, isolators or the like while still working inside.

[0022] Summary of the invention

[0023] This task has been solved by a method to perform a Human-Machine-

[0024] 30 Interaction process, comprising the following steps of Providing at least one clickable button within a graphical user interface on a display connected to a computer to a human user, wherein the at least one clickable button Foreignfiling text P24-215

[0025] - 3 - causes an action for the Human-Machine-Interaction process if activated, and wherein the at least one clickable button includes an integrated barcode which contains data to activate the at least one clickable button; Scanning the integrated barcode with a barcode reader which has a data

[0026] 5 connection to the computer to obtain the data to activate the at least one clickable button; Transferring the obtained data to activate the at least one clickable button to the computer via the data connection; and Activating the at least one clickable button by converting the obtained data to a click event in the graphical user interface via the computer. Central feature of the method is that the clickable Button has an integrated barcode: That means it involves providing a clickable button within a graphical user interface (GUI) on a display which is connected to a computer. The button includes an integrated barcode, which contains data necessary to activate the button. This integration allows users to perform actions without needing to

[0027] 15 touch the screen, thereby maintaining workflow efficiency by enabling a hands-free interaction, which is particularly advantageous in environments requiring high traceability. To use the button with the integrated barcode a barcode reader scans the integrated barcode and transfers the data to the computer. The process converts the barcode data into a click event in the

[0028] 20 GUI. This results in the advantage that the seamless execution of digital commands through physical scanning, reducing the need for manual input and minimizing errors.

[0029] Advantageous and therefore preferred further developments of this invention emerge from the associated subclaims and from the description and the associated drawings.

[0030] One of those preferred further developments of the disclosed method comprise that the performed Human-Machine-Interaction process is a

[0031] 30 logistical process to provide consumables for a bioprocess. This enables the streamlining of supply chain operations, ensuring that the delicate bioprocess-consumables are tracked efficiently and accurately. And while Foreignfiling text P24-215

[0032] - 4 - the invented method of using the integrated barcode with the click-event initiating data could be used for many different processes, in the field of applying bioprocess-related consumables, like in a lab and / or a clean room, it is specifically advantageous if the user does not need to touch any screen

[0033] 5 or I / O device directly, due to hygienic requirements.

[0034] Another one of those preferred further developments of the disclosed method comprise that the user scans the integrated barcode with the barcode reader during the logistical process when the consumables are in place to confirm that the bioprocess can be continued. This enhances the operational efficiency by providing a quick and reliable method for confirming process steps, reducing downtime.

[0035] Another one of those preferred further developments of the disclosed

[0036] 15 method comprise that standard media plates plates are used as consumables. Those media plates are used as a growth media for the culturing of bacteria. They are used as a general-purpose, non-selective media providing enough nutrients to allow for a wide variety of microorganisms to grow and therefore it is used for a wide range of

[0037] 20 applications, including culture storage, enumeration of cells, isolation of pure cultures, and many other uses in the bioprocessing field. This makes the application of the invented method for media plates as consumables very helpful.

[0038] Another one of those preferred further developments of the disclosed method comprise that Trypticase soy agar (TSA) plates are used as standard media plates. TSA plates are in particular suitable for moderately to non fastidious qrowing bacteria and therefore a first choice for many of the mentioned applications in Biomonitoring.

[0039] 30

[0040] Another solution to this task is an system to perform a Human-Machine- Interaction process, comprising a display to perform the interaction with a Foreignfiling text P24-215

[0041] - 5 - user, a computer connected to the display, a graphical user interface provided by the computer, a barcode reader with a data connection to the computer, wherein the graphical user interface is configured to display at least one barcode which is integrated in a clickable button of the graphical

[0042] 5 user interface and contains data to activate the clickable button, and wherein the system is configured to perform the previously described method. Regarding the realization of the clickable button, the screen can be a touchscreen or a normal screen with connected I / O devices like a keyboard or a computer mouse, which are adapted to the respective environment wherein the system is used - e.g. in a clean room only specific tools with a suitable IP class are allowed. This provides a comprehensive solution for hands-free interaction, resulting in a enfully integrated system that enhances user efficiency and accuracy in environments requiring frequent data input and confirmation.

[0043] 15

[0044] Another one of those preferred further developments of the disclosed system comprise that the barcode reader is a barcode scanner with a wired or wireless data connection to the computer or a smartphone with a wireless data connection to the computer. The barcode reader can

[0045] 20 therefore be either a handheld device operated by the user like a specific barcode reader or also just a normal smartphone with a camera, or it could be integrated in a suitable hardware setup where the reader is oriented at the screen so it can always scan the barcode within the clickable button. This flexibility allows for various configurations depending on user needs and existing infrastructure, enhancing adaptability and ease of integration.

[0046] Another one of those preferred further developments of the disclosed system comprise that the barcode is a two-dimensional Data-Matrix-Code, in particular a QR-Code, or a linear barcode. This variety ensures

[0047] 30 compatibility with different scanning technologies and user preferences, increasing the system's applicability across various industries. Foreignfiling text P24-215

[0048] - 6 -

[0049] Another solution to this task is an graphical user interface (GUI) for the described system comprising with at least one clickable button with an integrated barcode, wherein the barcode contains data to signal the computer to activate the at least one clickable button in the graphical user

[0050] 5 interface. To enable the function of confirming an action one clickable button is enough. It might be the case though, that more than one action is required to initiate. The invented system includes the usage of more than one button with an integrated barcode and an action event encoded in the data of the barcode. This feature then allows users to signal the computer to activate the button, providing a direct, efficient method for executing commands without manual input.

[0051] Another one of those preferred further developments of the disclosed GUI comprise that the at least one clickable button is configured to perform the

[0052] 15 functions of yes, no, cancel, exit, start, stop or ready. This list is not exclusive - it just shows the most common action events to be triggered by the integrated barcode. It provides users with a versatile tool for managing diverse tasks, enhancing operational flexibility and responsiveness.

[0053] 20 Detailed description of the invention

[0054] The method and the associated automated system according to the invention and functionally advantageous developments of those are described in more detail below with reference to the associated drawings using at least one preferred exemplary embodiment. In the drawings, elements that correspond to one another are provided with the same reference numerals.

[0055] The drawings show:

[0056] 30

[0057] Figure 1 : a schematic diagram of an example of the invented system Figure 2: a schematic flowchart of an example of the invented method Foreignfiling text P24-215

[0058] - 7 -

[0059] Figure 3: an example of a clickable button with an integrated QR code Figure 4: an example setup for a first step of the invented method

[0060] Figure 5: an example setup for a second step of the invented method

[0061] 5 The working example of the invention involves a system and method designed to facilitate human-machine interaction through the integration of barcodes 4 into clickable buttons 5 on a graphical user interface 6 (GUI). This approach addresses the need for seamless interaction without requiring users 1 to free their hands from a barcode scanner / reader 3, thereby enhancing workflow efficiency, particularly in environments demanding high traceability.

[0062] The system, as illustrated in a preferred embodiment in Figure 1 , comprises several key components. The user 1 interacts with the HMI-system 7

[0063] 15 through a display 8 connected to a computer 2. The display 8 is equipped with a user interface 6 that features at least one clickable button 5. Each button 5 is integrated with a barcode 4, which contains data 10 necessary to activate the button 5. This integration allows the user 1 to confirm actions or navigate menus by scanning the barcode 4 with a barcode reader 3,

[0064] 20 which is connected to the computer 2. The barcode reader 3 can be configured to connect via wired or wireless means, offering flexibility in setup and operation.

[0065] The method, depicted in a preferred embodiment in Figure 2, begins with providing the GUI 6 that includes the button 5 with the integrated barcode 4. The user 1 employs a barcode reader 3 to scan the barcode 4, which is a critical step in the process. This action transfers the barcode data 10 to the computer 2, where it is converted into a click event within the GUI 6. This conversion activates the button 5, allowing the user 1 to perform the

[0066] 30 desired action without physically touching the screen 8. This method is particularly advantageous in logistical processes, such as confirming the Foreignfiling text P24-215

[0067] - 8 - placement of consumables in bioprocessing, where maintaining continuity of operations is crucial.

[0068] In some embodiments, the barcode 4 integrated into the button 5 may

[0069] 5 contain smart information to distinguish between different buttons or menus on the screen. This feature ensures that the correct action is triggered, enhancing the precision of the interaction. The system 7 can be configured to support various types of barcodes 4, including two-dimensional Data- Matrix-Codes, QR-Codes, or linear barcodes, providing compatibility with diverse scanning technologies.

[0070] The system's adaptability extends to the barcode reader 3, which may be a traditional scanner or a smartphone with a barcode scanning application. This versatility allows users 1 to choose the most suitable device for their

[0071] 15 operational needs, ensuring seamless integration into existing workflows. The GUI 6 is designed in a further preferred embodiment to accommodate multiple clickable buttons 5, each capable of performing a range of functions such as yes, no, cancel, exit, start, stop, or ready. This configurability provides users 1 with a versatile tool for managing diverse

[0072] 20 tasks, enhancing operational flexibility and responsiveness.

[0073] In Figures 3 to 5 a further preferred embodiment of the invention is shown. The method described in this working example involves two primary steps, which facilitate the seamless interaction between users 1 and the HMI- system 7 through the integration of barcodes 4 into clickable buttons 5 on the GUI 6. Figure 3 shows an example of a clickable button 5 with an integrated barcode 4, in this case a QR code.

[0074] In the first step, as shown in Figure 4, the user 1 employs a barcode reader

[0075] 30 3 to scan the TSA plates which are involved in the respective logistical process. This step is crucial as it initiates the interaction by capturing the necessary data encoded in the barcode 4. The barcode 4 can contain smart Foreignfiling text P24-215

[0076] - 9 - information that helps to distinguish between different buttons 5 or menus on the screen 8, ensuring that the correct action is triggered.

[0077] In the second step, as shown in Figure 5, the user 1 confirms the

[0078] 5 placement of the TSA plates on the rack by scanning the barcode 4 integrated into the button 5 on the GUI 6. This action serves as a confirmation of the task's completion, allowing the bioprocess to continue without interruption. If, for some reason the barcode reader cannot be used, it is still possible that the user 1 simply places the scanner on a support and uses his finger to click the button 5. This flexibility in interaction methods enhances user convenience and ensures that the process can be completed efficiently regardless of the user's preference or the specific operational context.

[0079] 15 These steps collectively streamline the workflow by reducing the need for manual input and ensuring that actions are confirmed quickly and accurately. This method addresses the need for efficient human-machine interaction in environments where maintaining continuity and accuracy is paramount, such as in bioprocessing and other logistical operations.

[0080] 20

[0081] Regarding the software which is required to convert the data in the barcode 4 to the mentioned clickable event, this software is configured in this working example to use the barcode reader 3 and when a barcode 4 is scanned, according to the content, it will send a event ‘click, keypress, move on’ on the corresponding button 5. In summary, the software will react to the used barcode reader 3 and convert the content to a windows event. In our case, it will be a click event on the button 5.

[0082] The advantage of the invented method is easily explained by just imaging

[0083] 30 the following scenario: The user 1 is using the scanner to scan the barcodes 4 of consumables. So he has the consumables in one hand and the reader in the other. He has to scan 3 or 4 consumables and place them Foreignfiling text P24-215

[0084] - 10 - on the instrument. Between two scanning he has to confirm on the described HMI system 7 that he put the scanned consumable in place in order to let the instrument make a rotation to present another place to put the next consumable. In this case he can use the button 5, instead of

[0085] 5 dropping the barcode reader 3 and can still use the other to get the next consumable.

[0086] Effectively, it is also possible in an alternative solution to just print the used barcodes 4 and stick them on the used instrument and / or consumable, but then it wouldn't enable a ‘contextual’ list of Code, but the user 1 then has to select the good one (yes, no, cancel, exit, start, stop, ... ). The advantage of having it on the HMI system 7 as invented, is to have contextual barcodes 4.

[0087] 15 Overall, this invention offers a novel solution to the challenge of maintaining efficient human-machine interaction in environments where users 1 must manage physical devices and digital interfaces simultaneously. By integrating barcodes 4 into clickable buttons 5, the system streamlines operations, reduces the need for manual input, and minimizes the potential

[0088] 20 for errors, thereby improving productivity and accuracy in various industrial applications.

[0089] 30 Foreignfiling text P24-215

[0090] - 11 -

[0091] List of references

[0092] 1 User

[0093] 2 Computer

[0094] 5 3 Barcode reader

[0095] 4 Barcode

[0096] 5 Clickable Button

[0097] 6 Graphical User Interface (GUI)

[0098] 7 Human-Machine-Interface (HMI) system

[0099] 8 Display / Touchscreen

[0100] 9 action information

[0101] 10 obtained data

[0102] 15

[0103] 20

[0104] 30

Claims

Foreignfiling text P24-215- 12 -Patent claims1. Method to perform a Human-Machine-Interaction process (7), the following steps comprising:5 • Providing at least one clickable button (5) within a graphical user interface (6) on a display (8) connected to a computer (2) to a human user (1 ), wherein the at least one clickable button(5) causes an action (9) for the Human-Machine-Interaction process (7) if activated, and wherein the at least one clickable button (5) includes an integrated barcode (4) which contains data to activate the at least one clickable button (5);• Scanning the integrated barcode (4) with a barcode reader (3) which has a data connection to the computer (2) to obtain the data (10) to activate the at least one clickable button (5);15 • Transferring the obtained data (10) to activate the at least one clickable button (5) to the computer (2) via the data connection;• Activating the at least one clickable button (5) by converting the obtained data (10) to a click event in the graphical user interface(6) via the computer (2).

202. Method according to claim 1 , wherein the performed Human-Machine-Interaction process (7) is a logistical process to provide consumables for a bioprocess.

3. Method according to claim 2, wherein the user (1 ) scans the integrated barcode (4) with the barcode reader (3) during the logistical process when the consumables are in place to confirm that the bioprocess can be continued.

304. Method according to claim 2 or claim 3, wherein standard media plates are used as consumables.Foreignfiling text P24-215- 13 -5. Method according to claim 4, whereinTrypticase soy agar (TSA) plates are used as standard media plates.

56. System to perform a Human-Machine-Interaction process (7), comprising a display (8) to perform the interaction with a user (1 ), a computer (2) connected to the display (8), a graphical user interface (6) provided by the computer (2), a barcode reader (3) with a data connection to the computer (2), wherein the graphical user interface (6) is configured to display at least one barcode (4) which is integrated in a clickable button (5) of the graphical user interface (6) and contains data to activate the clickable button (5), and wherein the system is configured to perform the method of claims 1 to 5.

157. System according to claim 6, wherein the barcode reader (3) is a barcode scanner with a wired or wireless data connection to the computer (2) or a smartphone with a wireless data connection to the computer (2).

208. System according to claim 6 or claim 7, wherein the barcode (4) is a two-dimensional Data-Matrix-Code, in particular a QR-Code, or a linear barcode.

9. Graphical User Interface (6) for a system according to claims 6 to 8 with at least one clickable button (5) with an integrated barcode (4), wherein the barcode (4) contains data to signal the computer (2) to activate the at least one clickable button (5) in the graphical user interface (6).30 10. Graphical User Interface (6) according to claim 9, whereinForeignfiling text P24-215- 14 - the at least one clickable button (5) is configured to perform the functions of yes, no, cancel, exit, start, stop or ready.51520

Images