Operator terminal and monitor for automation

Abstract

The present invention relates to an automated operator terminal and monitor, particularly a human-machine interface (HMI) system and method of use thereof, designed to improve human-machine interaction by integrating barcodes into clickable buttons in the graphical user interface (GUI) on the display connected to a computer. It allows the user to activate an operation by scanning the integrated barcode with a barcode reader, which transmits the acquired data to the computer and converts it into a click event. This method streamlines the process by reducing manual input, which is particularly advantageous in environments requiring high traceability, such as logistics processes for bioprocessing. The system supports multiple barcode types, including two-dimensional Data Matrix and QR codes, and can be configured with wired and wireless connections. The GUI accommodates multiple buttons that can perform different functions to increase operational flexibility and efficiency. It provides a seamless solution for maintaining continuity and accuracy of workflows in industrial applications.

Description

Technical Field

[0001] The invention described herein discloses a method and system for performing human-computer interaction processes using a graphical user interface with integrated barcodes.

[0002] This invention relates to the technical field of logistics for bioprocess consumables. Background Technology

[0003] In the field of human-computer interaction, especially in the context of biological monitoring, such as logistics processes in cleanroom monitoring, integrating barcode technology into user interfaces has become a key innovation. As a means of encoding information, barcodes can be easily scanned and interpreted by various devices, facilitating efficient data input and retrieval. The widespread use of barcodes across industries has enabled traceability and management of consumables throughout the supply chain.

[0004] Existing patents, such as US9047385B1, describe interactive barcode systems that enhance user engagement by providing visual feedback through a graphical user interface (GUI). This patent outlines methods for associating barcodes with specific actions, enabling users to interact with digital systems in a more intuitive way. For example, barcodes can be scanned to retrieve information or confirm actions, thereby streamlining workflows in environments where speed and accuracy are critical. Another related patent, US10915721B1, focuses on mobile computing devices that communicate with barcode scanners to provide real-time feedback during the scanning process. This technology is particularly useful in the healthcare field, where barcode scanners are used to manage patient information and medication administration. The integration of feedback mechanisms, such as visual indicators on mobile devices, ensures that healthcare professionals receive immediate confirmation of a successful scan, thereby reducing errors and improving operational efficiency.

[0005] Despite these advances, challenges remain in achieving seamless human-computer interaction, particularly when users need to switch between physical operations, such as scanning barcodes and digital responses, such as confirming operations on a touchscreen. Especially on GUI panels, users must always keep their hands free to confirm operations or select menus via keyboard or touchscreen. This is becoming increasingly common with improved traceability, where users need to operate barcode scanners during operations to track consumables or hardware devices used in the process.

[0006] Therefore, especially in processes requiring high accuracy and traceability, there is clearly a need for a more integrated approach that combines the physical interaction of scanning with digital verification of operations.

[0007] Therefore, the objective of this patent application is to find an efficient method for remotely controlling the environmental settings of cleanrooms, isolators, etc., while still operating internally. Summary of the Invention

[0008] This task is addressed by a method for performing a human-computer interaction process, comprising the following steps: providing a human user with at least one clickable button in a graphical user interface (GUI) on a display connected to a computer, wherein, when activated, the at least one clickable button prompts an operation for the human-computer interaction process, and wherein the at least one clickable button includes an integrated barcode containing data for activating the at least one clickable button; scanning the integrated barcode with a barcode reader connected to the computer to obtain the data for activating the at least one clickable button; transmitting the acquired data to the computer via the data connection to activate the at least one clickable button; and activating the at least one clickable button by converting the acquired data into a click event in the GUI via the computer. A key feature of this method is that the clickable button has an integrated barcode: this means that it provides a clickable button within a graphical user interface (GUI) on a display connected to the computer. The button includes an integrated barcode containing the data required to activate the button. This integration allows users to perform operations without touching the screen, thereby maintaining workflow efficiency through hands-free interaction, which is particularly advantageous in environments requiring high traceability. To use the button with the integrated barcode, a barcode reader scans the integrated barcode and transmits the data to the computer. This process converts barcode data into click events in a graphical user interface. The advantage of this is that it allows for seamless execution of digital commands via physical scanning, reducing the need for manual input and minimizing errors.

[0009] Advantageous and therefore preferred further extensions of the invention can be derived from the relevant dependent claims, the description and the drawings.

[0010] One preferred further extension of the disclosed method includes: the performed human-machine interaction process being a logistics process for providing consumables for biological processes. This achieves streamlined supply chain operations, ensuring that delicate biological process consumables are tracked efficiently and accurately. While the method of the present invention, using integrated barcodes with click event initiation data, can be used in many different processes, it is particularly advantageous in areas such as the application of biological process-related consumables in laboratories and / or cleanrooms, where users do not need to directly touch any screens or I / O devices due to hygiene requirements.

[0011] Another preferred further extension of the disclosed method includes: during the logistics process, when consumables are in place, the user scans the integrated barcode using a barcode reader to confirm that the biological process can continue. This improves operational efficiency and reduces downtime by providing a fast and reliable method to confirm process steps.

[0012] Another preferred further extension of the disclosed method includes the use of standard media plates as consumables. These media plates are used as growth media for bacterial cultures. They serve as general-purpose, non-selective media, providing sufficient nutrients for the growth of a variety of microorganisms, and are therefore used in a wide range of applications, including culture storage, cell counting, isolation of pure cultures, and many other uses in the field of biological processing. This makes the method of the present invention particularly useful for the application of media plates as consumables.

[0013] Another preferred further extension of the disclosed method includes using tryptone soybean agar plates as the standard culture medium. TSA plates are particularly suitable for moderately to non-fastidious bacteria and are therefore the preferred choice for many of the applications mentioned in biomonitoring.

[0014] Another solution to this task is a system for performing human-computer interaction processes. This system includes: a display for interacting with the user, a computer connected to the display, a graphical user interface (GUI) provided by the computer, and a barcode reader with a data connection to the computer. The GUI is configured to display at least one barcode integrated into a clickable button on the GUI, containing data for activating the clickable button, and the system is configured to perform the aforementioned method. Regarding the implementation of the clickable button, the screen can be a touchscreen or a regular screen with connected I / O devices, such as a keyboard or mouse, adapted to the environment in which the system is used—for example, in a cleanroom where only specific tools with appropriate IP ratings are permitted. This provides a comprehensive solution for hands-free interaction, forming a fully integrated system that improves user efficiency and accuracy in environments requiring frequent data input and confirmation.

[0015] Another preferred further extension of the disclosed system includes: the barcode reader is a barcode scanner that has a wired or wireless data connection to a computer, or it could be a smartphone that has a wireless data connection to a computer. Therefore, the barcode reader can be a user-operated handheld device, such as a specialized barcode reader, or simply a regular smartphone with a camera, or it can be integrated into a suitable hardware setup where the reader faces the screen so that it can always scan barcodes in clickable buttons. This flexibility allows for various configurations based on user needs and existing infrastructure, enhancing adaptability and ease of integration.

[0016] Another preferred further extension of the disclosed system includes: the barcode being a two-dimensional data matrix code, particularly a QR code, or a linear barcode. This diversity ensures compatibility with different scanning technologies and user preferences, increasing the system's applicability across various industries.

[0017] Another solution to this task is a graphical user interface (GUI) for the system, which includes at least one clickable button with an integrated barcode, wherein the barcode contains data for signaling to the computer to activate the at least one clickable button in the GUI. For enabling confirmation operations, a single clickable button is sufficient. However, it is possible that multiple operations are required to initiate the action. The system of the present invention includes the use of multiple buttons with integrated barcodes, and operation events encoded in the barcode data. This feature thus allows the user to signal to the computer to activate the buttons, providing a direct and efficient method to execute commands without manual input.

[0018] Another preferred further extension of the disclosed GUI (Graphical User Interface) includes at least one clickable button configured to perform "Yes," "No," "Cancel," "Exit," "Start," "Stop," or "Ready" functions. This list is not exhaustive; it only lists the most common action events triggered by integrated barcodes. It provides users with a versatile tool for managing diverse tasks, enhancing operational flexibility and responsiveness. Attached Figure Description

[0019] The method according to the invention, related automation systems, and their functionally advantageous extensions will now be described in more detail with reference to the accompanying drawings and at least one preferred exemplary embodiment. In the drawings, corresponding elements are given the same reference numerals. Wherein: Figure 1 A schematic diagram illustrating an example of the system of the present invention; Figure 2 shows a schematic flowchart of an example of the method of the present invention; Figure 3 shows an example of a clickable button with an integrated QR code; Figure 4 An exemplary setup for the first step of the method of the present invention is shown; Figure 5 illustrates an exemplary setup for the second step of the method of the present invention. Detailed Implementation

[0020] The present invention relates to a system and method designed to facilitate human-computer interaction by integrating a barcode 4 into a clickable button 5 on a graphical user interface (GUI). This method addresses the need for seamless interaction without requiring the user 1 to take their hands off the barcode scanner / reader, thereby improving workflow efficiency, particularly in environments requiring high traceability.

[0021] like Figure 1 As shown in the preferred embodiment, the system includes several key components. User 1 interacts with HMI system 7 via display 8 connected to computer 2. Display 8 is equipped with user interface 6, characterized by at least one clickable button 5. Each button 5 integrates a barcode 4 containing data 10 required to activate button 5. This integration allows user 1 to confirm operations or navigate menus by scanning the barcode 4 using a barcode reader 3 connected to computer 2. Barcode reader 3 can be configured to connect via wired or wireless means, providing flexibility in setup and operation.

[0022] As shown in the preferred embodiment of Figure 2, the method begins with providing a GUI (Graphical User Interface) 6 including a button 5 with an integrated barcode 4. User 1 scans the barcode 4 using a barcode reader 3, which is a crucial step in the process. This operation transmits the barcode data 10 to the computer 2 and is converted into a click event within the GUI 6. This conversion activates the button 5, allowing user 1 to perform the desired operation without actually touching the screen 8. This method is particularly advantageous in logistical processes, such as confirming the placement of consumables during biological treatment, where maintaining operational continuity is crucial.

[0023] In some embodiments, the barcode 4 integrated into button 5 may contain intelligent information to distinguish different buttons or menus on the screen. This feature ensures that the correct action is triggered, thereby improving the accuracy of the interaction. System 7 can be configured to support various types of barcodes 4, including two-dimensional data matrix codes, QR codes, or linear barcodes, to provide compatibility with different scanning technologies.

[0024] The system's adaptability extends to the barcode reader 3, which can be a traditional scanner or a smartphone with a barcode scanning application. This versatility allows user 1 to select the most suitable device for their operational needs, ensuring seamless integration with existing workflows. In another preferred embodiment, the GUI 6 is designed to house multiple clickable buttons 5, each capable of performing a range of functions such as "Yes," "No," "Cancel," "Exit," "Start," "Stop," or "Ready." This configurability provides user 1 with a versatile tool for managing diverse tasks, enhancing operational flexibility and responsiveness.

[0025] Another preferred embodiment of the invention is illustrated in Figures 3 through 5. The method described in this working example comprises two main steps that facilitate seamless interaction between user 1 and HMI system 7 by integrating barcode 4 into clickable button 5 on GUI 6. Figure 3 shows an example of clickable button 5 with integrated barcode 4 (QR code in this example).

[0026] As shown in Figure 4, in the first step, user 1 uses barcode reader 3 to scan the TSA tablet involved in the corresponding logistics process. This step is crucial because it initiates the interaction by capturing the necessary data encoded in barcode 4. Barcode 4 may contain intelligent information that helps distinguish different buttons 5 or menus on screen 8, thereby ensuring that the correct operation is triggered.

[0027] As shown in Figure 5, in the second step, User 1 confirms the placement of the TSA tablet on the shelf by scanning the barcode 4 integrated into button 5 on the GUI 6. This action serves as confirmation of task completion, allowing the biological processing to continue uninterrupted. If, for some reason, the barcode reader is unavailable, User 1 can still simply place the scanner on the shelf and tap button 5 with their finger. This flexibility in the interaction method enhances user convenience and ensures efficient completion of the process regardless of user preferences or specific operational contexts.

[0028] These steps collectively streamline workflows by reducing the need for manual input and ensuring that operations can be quickly and accurately validated. This approach addresses the need for efficient human-machine interaction in environments requiring continuity and accuracy, such as biological processing and other logistical operations.

[0029] Regarding the software required to convert the data in barcode 4 into the clickable event, in this working example, the software is configured to use barcode reader 3, and when barcode 4 is scanned, it sends a "click, press, continue" event to the corresponding button 5, depending on the content. In short, the software reacts to the barcode reader 3 used and converts the content into a Windows event. In this example, this event will be a click event on button 5.

[0030] The advantages of the method of the present invention can be easily explained by imagining the following scenario: User 1 scans the barcode 4 of a consumable using a scanner. Therefore, the user holds the consumable in one hand and the reader in the other. The user needs to scan 3 or 4 consumables and place them on the instrument. Between scans, the user needs to confirm on the HMI system 7 that the scanned consumable has been placed in position so that the instrument can rotate to present another location for the next consumable. In this case, he can use button 5 without putting down the barcode reader 3 and can still use his other hand to get the next consumable.

[0031] In an alternative solution, one could simply print the barcode used (4) and affix it to the instrument and / or consumables used, but this would not enable a context-sensitive code list; instead, the user would then need to select the correct one (Yes, No, Cancel, Exit, Start, Stop, etc.). The advantage of placing it on the invented HMI system (7) is that it has a context-sensitive barcode.

[0032] In summary, this invention provides a novel solution to the challenge of maintaining efficient human-computer interaction in environments where users must simultaneously manage physical devices and digital interfaces. By integrating the barcode 4 into the clickable button 5, the system streamlines operation, reduces the need for manual input, and minimizes the possibility of errors, thereby improving productivity and accuracy in various industrial applications.

[0033] List of reference numerals 1 user 2 Computers 3. Barcode Reader 4. Barcode 5 Clickable buttons 6. Graphical User Interface (GUI) 7 Human-Machine Interface (HMI) System 8 Displays / Touchscreens 9. Operation Information 10. Data obtained.

Claims

1. A method for performing the human-computer interaction process (7) includes the following steps: ● Provide a human user (1) with at least one clickable button (5) in a graphical user interface (6) on a display (8) connected to a computer (2), wherein, if activated, the at least one clickable button (5) prompts an operation (9) for a human-computer interaction process (7), and wherein the at least one clickable button (5) includes an integrated barcode (4) containing data for activating the at least one clickable button (5). ● Use a barcode reader (3) that has a data connection with the computer (2) to scan the integrated barcode (4) to obtain data (10) for activating the at least one clickable button (5); ● The acquired data (10) is transmitted to the computer (2) via the data connection to activate the at least one clickable button (5); ● The at least one clickable button (5) is activated by converting the acquired data (10) into a click event in the graphical user interface (6) via the computer (2).

2. The method according to claim 1, wherein The human-computer interaction process (7) being performed is a logistics process for providing consumables for biological processes.

3. The method according to claim 2, wherein During the logistics process, when the consumables are in place, the user (1) uses the barcode reader (3) to scan the integrated barcode (4) to confirm that the biological process can continue.

4. The method according to claim 2 or 3, wherein Standard culture medium plates are used as consumables.

5. The method according to claim 4, wherein Tryptone soybean agar (TSA) plates were used as the standard culture medium plates.

6. A system for performing the human-computer interaction process (7), the system comprising: A display (8) for performing interaction with a user (1), a computer (2) connected to the display (8), a graphical user interface (6) provided by the computer (2), and a barcode reader (3) having a data connection with the computer (2), wherein the graphical user interface (6) is configured to display at least one barcode (4) integrated in a clickable button (5) of the graphical user interface (6) and containing data for activating the clickable button (5); and wherein the system is configured to perform the method of claims 1 to 5.

7. The system according to claim 6, wherein The barcode reader (3) is a barcode scanner that is wired or wirelessly connected to the computer (2), or a smartphone that is wirelessly connected to the computer (2).

8. The system according to claim 6 or 7, wherein The barcode (4) is a two-dimensional data matrix code, particularly a QR code, or a linear barcode.

9. A graphical user interface (6) for a system according to claims 6 to 8, comprising at least one clickable button (5) with an integrated barcode (4), wherein the barcode (4) contains data for signaling the computer (2) to activate the at least one clickable button (5) in the graphical user interface (6).

10. The graphical user interface (6) according to claim 9, wherein The at least one clickable button (5) is configured to perform the functions of "Yes", "No", "Cancel", "Exit", "Start", "Stop" or "Ready".

Images