Apparatus for optical imaging system, optical imaging system, method and computer program

Abstract

Examples relate to an apparatus for an optical imaging system that includes one or more processors and one or more storage devices. The apparatus is configured to receive sensor data of a sensor of an optical imaging system. The sensor data is received from an optical imaging system, such as a sensor. The sensor data is indicative of a real-time view of the sample acquired by a microscope of the optical imaging system. Further, the device is configured to generate a visual overlay comprising a visual representation of the real-time view and an icon. The visual overlay is generated based on sensor data. Further, the device is configured to transmit a display signal indicative of the visual overlay.

Description

Technical Field

[0001] The embodiments relate to an optical imaging system, such as an optical imaging system for monitoring and controlling the quality of a manufacturing process or workpiece in an industrial environment, and to an optical imaging system, a method, and a computer program. Background Technology

[0002] Optical imaging systems are commonly used in industrial and manufacturing environments to monitor and control the quality of processes or workpieces. These systems can be characterized by digital imaging systems, video capabilities, and precision measuring tools, allowing users to quickly and accurately inspect samples and workpieces, such as products, during or after the manufacturing process. Furthermore, they often feature automated analysis systems capable of providing real-time feedback to the production line, further streamlining quality process control. The software components of the measurement suite are typically built using dedicated software designed to work in conjunction with a microscope. However, this software may be insufficient to allow users to adjust the parameters as needed. Therefore, an improved concept that provides users with the ability to set up process control measurements may be desired. Summary of the Invention

[0003] This expectation is achieved through the subject matter of the independent claims.

[0004] The concept presented in this disclosure is based on the idea that by providing icons indicating multiple possible actions associated with a measurement, measurements for process control can be easily set up. The icons can be correlated with real-time visual data of the sample. Figure 1 It is provided as a visual overlay. This allows users to select icons within the live view window.

[0005] An example provides an apparatus for an optical imaging system, comprising one or more processors and one or more storage devices. The apparatus is configured to receive sensor data from a sensor of the optical imaging system. The sensor data is received from the optical imaging system, for example, from a sensor. The sensor data indicates a real-time view of a sample acquired by a microscope through the optical imaging system. Further, the apparatus is configured to generate a visual overlay comprising a visual representation and icons of the real-time view. The visual overlay is generated based on the sensor data. The icons indicate multiple possible actions associated with a measurement. Further, the apparatus is configured to send a display signal indicating the visual overlay to, for example, a display device. In this way, the display device can be controlled or triggered to display the real-time view and icons of the sample to facilitate, for example, setting measurements for process control. Actions may include generating geometric elements and / or selecting measurement parameters. For example, the icons may indicate multiple actions required to define a measurement (e.g., a dimensional measurement).

[0006] In one example, the device is further configured to receive a trigger signal indicating a user's selection of an icon; and to generate a second visual overlay comprising a visual representation of the real-time view and visual representations of several possible actions. The generation of the second visual overlay is based on sensor data and the trigger signal. Further, the device can be configured to send a second display signal indicating the second visual overlay to, for example, a display device. For example, by generating the second visual overlay, the device can control or trigger a display device to display auxiliary information to the user, such as which geometric elements or measurement parameters can be selected to set the measurement. In this way, the user can easily learn about the possible adjustments that can be made to set the measurement for process control.

[0007] In one embodiment, the device may be further configured to receive a second trigger signal indicating an increased probability of selecting one of a plurality of possible actions. The selection may be performed by a user. Further, the device may be configured to adapt a second visual overlay by highlighting the action (with an increased probability of selection) among the plurality of possible actions. This adaptation is based on the second trigger signal. Further, the device may be configured to send an adapted second display signal to, for example, a display device. The increased probability of selecting an action may be due to a user's intention. For example, a user may move the mouse cursor closer to a geometric element. This movement may indicate an intention by the user to select the geometric element. Thus, the device may highlight geometric elements near or below the mouse cursor. In this way, the user can easily learn about the geometric elements and / or measurement parameters they can select.

[0008] In one embodiment, the second visual overlay can highlight measurement parameters and / or pre-existing geometric elements. In this way, the user can receive visual feedback before making a selection. This can, for example, improve the user experience because the selection can be performed with greater reliability.

[0009] In one embodiment, the second visual overlay may include a representation of each possible action associated with the selected icon among a plurality of actions. In this way, the user can receive visual feedback on each possible action associated with the selected measurement. Thus, the user can understand every possibility of setting the measurement.

[0010] In one embodiment, multiple possible actions may include generating geometric elements and / or defining measurement parameters. In this way, measurements for process control can be fully defined based on multiple possible actions. This allows users to generate measurements for process control using a simplified workflow.

[0011] In one embodiment, multiple possible actions may include different geometric elements to be generated and / or selected. For example, multiple geometric elements of the sample may have already been defined, also referred to as pre-existing geometric elements. Thus, the user can select a suitable geometric element from the pre-existing geometric elements. For example, different geometric elements among the multiple geometric elements may belong to different measurements. In this way, the user can easily select the desired geometric element.

[0012] In one embodiment, multiple possible actions may include different measurement parameters to be defined. For example, a user might expect to perform a dimensional measurement to measure the distance between a primary geometric element and a secondary geometric element. However, the primary geometric element could be a circle, and the distance to be measured could be the distance to the center of the circle or the distance to the edge of the circle. Thus, different measurement parameters can be displayed to the user. In this way, the user can easily select the desired measurement parameters.

[0013] In one embodiment, the device may be further configured to receive a third trigger signal indicating a user's selection of a pre-existing geometric element. Further, the device may be configured to generate a second visual overlay and / or an adapted second visual overlay based on the third trigger signal. For example, if multiple geometric elements are required to define the measurement, the device may highlight only the geometric elements that can be used in conjunction with the selected pre-existing geometric elements. This facilitates measurement setup.

[0014] In one embodiment, the selected pre-existing geometric element can be a primary geometric element, and the adapted second visual overlay can include highlighting possible secondary geometric elements. In this way, the user can receive information about the secondary geometric elements associated with the primary geometric element. The primary and secondary geometric elements can be the geometric elements required to set up / run measurements.

[0015] In one embodiment, the device may be further configured to receive a fourth trigger signal indicating the selection of an action from a plurality of possible actions; and to generate geometric elements and / or perform measurements based on the fourth trigger signal. In this way, the device can perform certain measurements to allow the setup and / or operation of measurements for process control.

[0016] An embodiment provides a device as described herein (e.g., above or below).

[0017] An embodiment provides a method for an apparatus for an optical imaging system. The method includes: receiving sensor data from a sensor of the optical imaging system, the sensor data indicating a real-time view of a sample acquired through a microscope of the optical imaging system. The method further includes: generating a visual overlay based on the sensor data, the visual overlay including a visual representation of the real-time view and icons indicating a plurality of possible actions associated with the measurement; and sending a display signal indicating the visual overlay.

[0018] The various embodiments of this disclosure relate to a corresponding computer program having program code, which, when executed on a processor, performs the methods described above. Attached Figure Description

[0019] Some embodiments of the device and / or method will now be described by way of example only and with reference to the accompanying drawings, in which: Figure 1 A schematic diagram of an example device for an optical imaging system is shown; Figure 2 Examples of several possible actions are shown; Figure 3 A schematic diagram of a system is shown; Figure 4 A flowchart illustrating an example of one method is shown; and Figure 5 A flowchart of another example of one method is shown. Detailed Implementation

[0020] Various embodiments will now be described more fully with reference to the accompanying drawings, which illustrate some embodiments. In the drawings, the thickness of lines, layers, and / or regions may be exaggerated for clarity.

[0021] Figure 1 A schematic diagram of an example of a device 130 for an optical imaging system 100 is shown. Device 130 is responsible for controlling the microscope and various aspects of the optical imaging system 100 as a process control system suitable for industrial environments, and / or for processing various types of sensor data from the optical imaging system 100. Therefore, device 130 can be implemented as a computer system that interacts with various components of the optical imaging system 100, such as sensor 122.

[0022] Device 130 includes one or more processors 134 and one or more storage devices 136, such as Figure 1As shown. Optionally, device 130 also includes one or more interfaces 132. One or more processors 134 are coupled to one or more storage devices 136 and optionally one or more interfaces 132. Typically, the functionality of device 130 may be provided by one or more processors 134 (e.g., for generating display signals) in combination with one or more interfaces 132 (for exchanging data, such as exchanging with sensor 122 to receive sensor data, exchanging with display device 330 to send display signals) and / or one or more storage devices 136 (for storing and / or retrieving data).

[0023] Device 130 is configured to receive sensor data from sensors of optical imaging system 100. This sensor data is received from optical imaging system 100 (e.g., from sensor 122, frame buffer, storage device). The sensor data indicates a real-time view of sample 110 acquired by a microscope (not shown) of optical imaging system 100. The real-time view can be acquired by sensor 122. Sample 110 can be a sample to be tested. For example, sample 110 can be a workpiece in an industrial manufacturing process. For example, optical imaging system 100 can be configured for process control of sample 110. Optical imaging system 100 can be designed for use in a laboratory or factory environment. Optical imaging system 100 can be used to inspect sample 110 (e.g., a product) during the manufacturing process to ensure it meets required specifications and standards.

[0024] The optical imaging system 100 may be equipped with features such as digital imaging (e.g., sensor 122), video capabilities (e.g., display device 330), and precision measuring tools, allowing users (e.g., technicians) to identify defects, monitor variations in the manufacturing process, and / or make adjustments as needed to ensure product quality. Furthermore, many process control microscopes have automated systems capable of rapidly analyzing images and providing feedback to the production line, further streamlining quality process control. However, adjustments to measurements may be necessary for process control, such as to compensate for systematic errors caused by variations in the manufacturing process or to establish measurements for new workpieces.

[0025] Device 130 is configured to generate a visual overlay that includes a visual representation of a real-time view and icons. This visual overlay is generated based on sensor data. The icons indicate multiple possible actions associated with the measurement. An icon is a graphic symbol or image representing an action on a software application, file, folder, and / or processing circuitry (e.g., a part of a computer). The icon can be designed to be easily recognizable and memorable. For example, the icon can be designed to use simple and minimalist visual effects, such as to inform the user of the selected measurement. For example, the icon can use symbols or signs to convey meaning and function, such as combining with geometric elements to display multiple possible new measurements.

[0026] Possible actions can be generating geometric elements, selecting pre-existing geometric elements, selecting measurement parameters, and / or defining measurement parameters. For example, multiple possible actions can allow a user to set measurements for process control.

[0027] The icon can indicate multiple possible actions. Thereby, the user can learn each possible action for measurement among the multiple possible actions through the icon. Alternatively, the icon can only present a part of the possible actions among the multiple possible actions. For example, the icon can show x different measurement parameters and can indicate y different measurement parameters, where x < y. In this way, the recognizability of the icon can be improved. In this case, the second visual overlay layer described below can be used to display each possible action among the multiple possible actions to the user.

[0028] The real-time view refers to a real-time display of the sample 110. For example, the real-time view can allow a user to view a continuous and up-to-date display of the sample 110 being acquired by the microscope.

[0029] The real-time view is particularly useful in process control microscope applications because it allows the user to view an image of the sample 110 in real time and make adjustments to the microscope or the sample 110 as needed.

[0030] For example, the real-time view can be used to inspect and monitor the quality of a workpiece during a manufacturing process, or to adjust the focal length and positioning of the microscope to ensure that the real-time view is correctly acquired.

[0031] The real-time view can be presented in a separate window or as part of the main application window. The real-time view window can carry an icon, for example, as part of a configuration panel. Additionally or alternatively, the icon can be integrated into the real-time view window. For example, the icon can be used to configure measurements for optical inspection, such as dimensional measurement of the sample 110.

[0032] For example, the measurement can be optical inspection, such as surface inspection, texture analysis, defect detection, dimensional measurement, etc. The icon can provide the user with a (first) selection. For example, multiple icons can be displayed, with each icon assigned to a different measurement. The measurement can involve the measurement of measurement parameters, such as measurement parameters of the sample 110 (e.g., the structure of the sample 110) like length, width, height, diameter, radius, roughness, texture, etc. Thereby, an icon can be defined for the measurement of at least one measurement parameter.

[0033] Dimensional measurement refers to the process of determining the size and dimensions of a structure. For example, dimensional measurement can be used to inspect and verify the size and dimensions of a workpiece (e.g., sample 110) during the manufacturing process. This is to ensure that the workpiece meets the required specifications and standards and to detect any deviations or defects that may affect the quality or performance of the workpiece. Dimensional measurements can be performed on a variety of materials and workpieces, including metal parts, plastic components, and electronic devices. Therefore, selecting the desired optical inspection (e.g., dimensional measurement) can be complex.

[0034] Therefore, device 130 is further configured to send a display signal indicative of a visual overlay. For example, this display signal may be sent to display device 330 or a storage device, such as a frame buffer. Optical imaging system 100 may include display device 330. Alternatively, display device 330 may be located external to optical imaging system 100, for example, communicatively coupled to optical imaging system 100. The display signal may contain information about the visual overlay (e.g., sensor data), or it may contain sensor data and data about the visual representation of icons (e.g., information about elements representing icons on a displayed graphical user interface (GUI)). In this way, display device 330 can be used in conjunction with icons to display a real-time view. Optionally, the display signal may contain data for controlling display device 330. Thus, device 130 can trigger the display of a visual overlay on display device 330.

[0035] Displaying icons on a screen makes setting up measurements for process control easier. For example, it reduces the number of clicks required to set up measurements for process control. Multiple clicks might be needed to define a measurement for process control. For instance, measurements for process control might include generating / selecting (pre-existing) geometric elements and selecting measurement parameters. With icons, users can initiate the workflow for setting up measurements for process control using only one icon.

[0036] Furthermore, this visual overlay can provide the user with information about measurement parameters and / or geometric elements associated with the measurement. In this way, the user can easily select the desired measurement parameters and / or geometric elements.

[0037] This icon can reduce the number of clicks required to set up measurements; for example, it eliminates the need to select one icon to define a geometric element and another icon to define measurement parameters. This reduces the time required to perform measurement setup tasks. Furthermore, the GUI can be less simplistic. For example, a configuration panel containing icons might appear less cluttered. This allows users to easily set up measurements for process control and / or improves the user experience.

[0038] Instead of using different icons / buttons to create geometric elements (or secondary geometric elements required for measurement) and measurement parameters, users can simply click a (single) icon (e.g., for angle measurement) to select from multiple possible schemes (e.g., measurements: angle, circle to circle, circle to line, circle to point; secondary geometric elements: tangents between circles, tangents between a circle and a point, bisecting lines, etc.) and corresponding functions. Thus, selecting an icon initiates the workflow, making measurement setup easy. For example, a second visual overlay allows users to select and / or generate geometric elements / measurement parameters directly in the live view, for example, via mouse control.

[0039] For example, if it is necessary to create secondary geometric elements, device 130 can generate an adapted second visual overlay to present possible secondary geometric elements in a live view, for example, after the user has selected a primary geometric element (e.g., information about that selection can be received via a third trigger signal). Thus, the user can receive information about secondary geometric elements associated with the measured geometric element.

[0040] You can select the type of geometric element to be selected / defined and / or the type of measurement parameter by clicking the icon. This icon graphically represents all possible variations in a single image, simplifying the user interface. This facilitates measurement setup.

[0041] The proposed concept is built around two main components: a microscope and an apparatus 130. The microscope includes optical components and can house a display device 330 for observing the sample 110. The apparatus 130 is used to control the optical imaging system 100, process sensor data from the microscope (e.g., sensor 122), and generate display signals.

[0042] Typically, the optical microscope system 100 includes a microscope suitable for examining tiny objects that cannot be viewed (alone) by the human eye. For example, the microscope can provide a view of a sample (such as... Figure 1 The sample 110 shown is optically magnified. This allows for magnification and measurement of the structure of the sample 110 (e.g., structures formed by manufacturing steps) with greater precision. In modern microscopes, optical magnification is typically provided for a camera or imaging sensor (such as the microscope's optical imaging sensor 122). The microscope may also include one or more optical magnification components, such as objectives, for magnifying the view of the sample 110.

[0043] There are various types of optical imaging systems. For example, optical imaging system 100 can be used for process control. Optical imaging system 100 may include an optical system, a camera or detector, and a monitor, such as display device 330. The optical system typically includes optical components, such as lenses, plane mirrors, or other optical components, that work together to produce a high-resolution image (e.g., a real-time view) of sample 110. The camera or detector (e.g., sensor 122) can be used to acquire a real-time view (e.g., a real-time view of sample 110) and convert it into a digital signal that can be displayed on display device 330 and / or analyzed by software (e.g., performed by device 130). Display device 330 can be used to display the real-time view acquired by sensor 122, as well as any other information, such as measurement data, analysis methods, icons (e.g., icons in a configuration panel), or possible actions for measurement.

[0044] In one embodiment, device 130 is further configured to receive a trigger signal indicating a user's selection of an icon; and to generate a second visual overlay comprising a visual representation of a real-time view and visual representations of multiple possible actions. The generation of the second visual overlay is based on sensor data and the trigger signal. The trigger signal may be a first trigger signal. Further, device 130 may be configured to send a second display signal indicating the second visual overlay. For example, the second display signal may be sent to a display device 330 or a storage device (such as a frame buffer). The first trigger signal may be received from an input device or a storage device. The input device may be a keyboard and / or a controller, and may include a mouse, trackball, touchscreen, voice recognition device, or any other device that allows a system user to input information therein.

[0045] The display device 330 can use this second visual overlay to display to the user multiple (e.g., each) possible actions out of a plurality of possible actions. In this way, after selecting an icon, the user can receive an overview of the possible actions associated with the measurement (associated with that icon). Thus, the user can easily obtain information about the geometric elements and / or measurement parameters associated with the measurement. Therefore, multiple icons for multiple geometric elements / measurement parameters associated with the measurement can be omitted (different icons / buttons for geometric elements and measurement parameters can be created using multiple possible schemes).

[0046] In one embodiment, device 130 may be further configured to receive a second trigger signal indicating an increased probability of selecting one of a plurality of possible actions. This second trigger signal may be received from an input device (e.g., the same input device used for the first trigger signal) or a storage device. The selection may be performed by a user. Further, device 130 may be configured to adapt a second visual overlay by highlighting an action (with an increased probability of being selected by the user) among the plurality of possible actions. This adaptation may be based on the second trigger signal. Further, device 130 may be configured to send an adapted second display signal to, for example, a display device 330 or a storage device (such as a frame buffer). The second visual overlay may assist the user with measurement settings. For example, the second visual overlay may highlight possible user choices. The user may be aware in advance of the selection or probability, for example, that clicking a location with the mouse cursor may result in the expected action. For example, the second visual overlay may highlight geometric elements near or below the mouse cursor (e.g., during mouse hover). For example, geometric elements may be highlighted during mouse hover. For example, device 130 can receive information about mouse hover via a second trigger signal. This second trigger signal can indicate mouse hover. Thus, the user can receive visual feedback, for example, before selecting a pre-existing geometric element.

[0047] For example, multiple measurement parameters can be associated with a measurement linked to an icon selected by the user. Multiple measurement parameters, such as the distance measurement from the center of the circle to a second geometric element, or the distance measurement from the edge of the circle to a second geometric element, can be displayed on the display device 330. However, multiple measurement parameters can become cluttered. Therefore, highlighting measurement parameters that may be selected by the user and / or associated with the mouse cursor position can improve the user experience.

[0048] Additionally or alternatively, the second visual overlay can highlight possible secondary geometric elements in the real-time camera view after one or more primary geometric elements have been selected. This can be performed independently of the mouse cursor position. For example, after selecting a pre-existing primary geometric element or generating a new primary geometric element, the second visual overlay can highlight possible secondary geometric elements associated with that primary geometric element to define measurements for process control. Device 130 can receive information about the selection / generation of primary geometric elements via a third trigger signal. The user can then select the desired secondary geometric element by, for example, clicking. In this way, the user can receive visual feedback about possible secondary geometric elements. For example, the user can receive information about a user's selection of a pre-existing geometric element based on the third trigger signal described herein.

[0049] In one embodiment, the second visual overlay can highlight measurement parameters and / or pre-existing geometric elements. For example, when a measurement is to be performed, the corresponding measurement result can be displayed when the mouse hovers over the corresponding area (e.g., different angles of two intersecting lines). For example, device 130 can receive information about mouse hover by receiving a second trigger signal. Thus, device 130 can highlight the measurement parameters associated with the corresponding area in the second visual overlay. By sending the second visual overlay to, for example, a display device 330 or a storage device (such as a frame buffer), the user can obtain information about the measurement parameters. The user can then select the measurement parameters by clicking.

[0050] In one embodiment, the second visual overlay may include a representation of each possible action associated with the selected icon among a plurality of actions. For example, device 130 may receive information about the icon selection via a second trigger signal. Based on the second trigger signal, device 130 may generate a second visual overlay to display to the user each possible action associated with the measurement (associated with the selected icon). In this way, the user can receive information about each possible action he / she can perform. This can improve the user experience and / or facilitate measurement settings.

[0051] In one embodiment, the plurality of possible actions may include generating geometric elements and / or defining measurement parameters. For example, an action could be any task related to measurement, particularly the setup of the measurement.

[0052] In one embodiment, the plurality of possible actions may include different geometric elements to be generated and / or selected. For example, measurement may include primary geometric elements and secondary geometric elements. A first possible action may be to generate / select primary geometric elements. A second possible action may be to generate / select secondary geometric elements. A second visual overlay may present both primary and secondary geometric elements. As described herein, highlighting of secondary geometric elements may be performed based on the selection of primary geometric elements.

[0053] In one embodiment, the multiple possible actions may include different measurement parameters to be defined. For example, dimensional measurements may include measurement parameters that can be associated with icons, such as angles, circle-to-circle, circle-to-line, circle-to-point, tangents between circles, tangents between a circle and a point, bisecting lines, etc. Therefore, multiple measurement parameters can be displayed to the user, allowing the user to easily select the desired measurement parameter.

[0054] In one embodiment, device 130 may be further configured to receive a third trigger signal indicating a user's selection of a pre-existing geometric element. The third trigger signal may be received from an input device (e.g., the same input device used for the first trigger signal) or a storage device. Further, device 130 may be configured to generate a second visual overlay and / or an adapted second visual overlay based on the third trigger signal. For example, if multiple geometric elements are required to define a measurement, device 130 may highlight only the geometric elements that can be used in conjunction with the selected pre-existing geometric element. For example, device 130 may receive information about the selection of a pre-existing geometric element via the third trigger signal. The selected pre-existing geometric element may be assigned as a primary geometric element. Thus, device 130 may determine possible secondary geometric elements associated with the primary geometric elements. Device 130 may further adapt the second visual overlay based on the possible secondary geometric elements. In this way, the user can receive an adapted second display overlay that displays only the possible secondary geometric elements.

[0055] In one embodiment, the selected pre-existing geometric element can be a primary geometric element, and the adapted second visual overlay can include highlighting possible secondary geometric elements. These possible secondary geometric elements can be geometric elements that can be used in conjunction with the primary geometric element to perform measurements. In this way, the user can receive a visual representation of the possible geometric elements to facilitate measurement setup.

[0056] In one embodiment, device 130 may be further configured to receive a fourth trigger signal indicating the selection of an action from a plurality of possible actions; and to generate geometric elements and / or perform measurements based on the fourth trigger signal. The fourth trigger signal may be received from an input device (e.g., the same input device used for the first trigger signal) or a storage device. In this way, the device can perform certain measurements to allow the setting and / or operation of measurements for process control. Thus, different measurements can be set. The setting of measurements may depend on the first and second trigger signals and / or the third and / or fourth trigger signals.

[0057] For example, a user can select an icon and initiate an exemplary workflow as follows. Device 130 can generate a second visual overlay. This second visual overlay can include multiple possible actions associated with a measurement (associated with the selected icon). The user can move the mouse cursor, and device 130 can receive a second trigger signal. This second trigger signal can indicate mouse hover. Device 130 can generate an adapted second visual overlay based on the second trigger signal, for example, highlighting a pre-existing geometric element associated with mouse hover. Thus, the user is aware of the possible pre-existing geometric elements to be selected during mouse hover. The user can select the highlighted pre-existing geometric element by clicking during mouse hover. Device 130 can receive information about the user's selection of the pre-existing geometric element via a third trigger signal. Device 130 can adapt the second visual overlay based on the third trigger signal, for example, to highlight possible secondary geometric elements associated with the selected pre-existing geometric element. Thus, the user is aware of the possibility of selecting a secondary geometric element. Device 130 can receive information about the user's selection of the secondary geometric element via another third trigger signal. Device 130 can adapt a second visual overlay based on another third trigger signal, for example, to display possible measurement parameters. Thus, the user is informed of the selected measurement parameter. The user can move the mouse cursor, and device 130 can receive another second trigger signal. This second trigger signal can indicate mouse hover associated with the measurement parameter. Device 130 can generate an adapted second visual overlay based on this second trigger signal, for example, highlighting the measurement parameter associated with the mouse hover. Thus, the user is informed of the possible measurement parameters to be selected. The user can select the highlighted measurement parameter by clicking. Device 130 can receive information about the selected measurement parameter via a fourth trigger signal. In this way, the user can easily set up measurements for process control. This workflow is for illustrative purposes only. The number and / or order and / or trigger signals used may vary.

[0058] like Figure 1As shown, one or more optional interfaces 132 are coupled to one or more corresponding processors 134 at device 130. In embodiments, the one or more processors 134 may be implemented using one or more processing units, one or more processing devices, or any means for processing, such as a processor, computer, or programmable hardware component that can operate with corresponding adaptive software. Similarly, the functionality of the one or more processors 134 may also be implemented in software, and then the software may be executed on one or more programmable hardware components. Such hardware components may include general-purpose processors, digital signal processors (DSPs), microcontrollers, etc. The one or more processors 134 are capable of controlling one or more interfaces 132 such that any data transfer occurring on the one or more interfaces 132 and / or any interaction involving the one or more interfaces 132 can be controlled by the one or more processors 134.

[0059] In one embodiment, device 130 may include a memory (e.g., one or more storage devices 136) and at least one or more processors 134 operatively coupled to the memory and used to perform the methods described below.

[0060] In an embodiment, one or more interfaces 132 may correspond to any means for acquiring, receiving, transmitting, or providing analog or digital signals or information, such as any connector, contact, pin, register, input port, output port, conductor, channel, etc., which allows the provision or acquisition of signals or information. One or more interfaces 132 may be wireless or wired and may be configured to exchange information (e.g., transmit or receive signals) with other internal or external components.

[0061] Device 130 may be a computer, processor, control unit, (F)programmable array ((F)PLA), (F)programmable gate array ((F)PGA), graphics processing unit (GPU), application-specific integrated circuit (ASIC), integrated circuit (IC), or system-on-a-chip (SoC). Device 130 may be part of optical imaging system 100. Alternatively, device 130 may be located outside optical imaging system 100, for example, it may be part of display device 330.

[0062] Further details and aspects will be described in conjunction with the embodiments described below. Figure 1 The embodiments shown may include one or more embodiments corresponding to the proposed concepts or those described below (e.g., Figures 2 to 5 One or more optional additional features of one or more aspects described.

[0063] Figure 2 Examples of several possible actions are shown. For ease of explanation, Figure 2The image shows more than one mouse cursor. As mentioned above, multiple possible actions can be overlaid on the live view of the sample.

[0064] As in Figure 2 As can be seen, multiple possible actions pertain to multiple measurement parameters. For example, the distance between circle 220 and trapezoid 230 can be measured. However, there may be different measurement parameters, namely, the possibility of measuring the distance between circle 220 and trapezoid 230 (e.g., the distance to the center of circle 220 or the distance to the edge of circle 220). During mouse hover, the device (e.g., reference...) Figure 1 The described device can generate an adapted second visual overlay to highlight measurement parameters, such as measurement parameter 232 forming the center of circle 220. Measurement parameter 232 can be highlighted if the mouse cursor is located in region 234. For example, region 234 can be assigned to measurement parameter 232. Thus, the user can easily select the desired measurement parameter based on several regions defined in the live view. Optionally, region 236 can be assigned to measurement parameter 238, and region 240 can be assigned to measurement parameter 242. For example, the device can generate the second visual overlay based on the selection of icon 228. The device can receive information about the selection of icon 228 via a first trigger signal.

[0065] For example, the second measurement could be a tangent measurement. This tangent can be measured starting from point 250. This second measurement can be selected by clicking icon 226. Tangent 256 can be highlighted while the mouse hovers over area 254. For example, the third measurement could be a measurement of the angle associated with trapezoid 230. This third measurement can be selected by clicking icon 224. Measurement parameter 264 can be highlighted while the mouse hovers over area 262.

[0066] Using the icons in configuration panel 210 reduces the number of checkbox / icon options needed to define specific geometric elements and / or measurement parameters. Furthermore, an intuitive user interaction with the real-time image can be provided by offering a visual overlay. This overlay allows the user to explicitly select the desired geometric elements and / or measurement parameters. In this way, the user receives a visual representation of the action to be selected.

[0067] Additionally or alternatively, user input can be received via touch gestures. For example, touch functionality can be enabled after selecting the desired geometric element and / or measurement parameters and / or icons, provided all available options are displayed. In this case, highlighting the available gestures may not be necessary.

[0068] Further details and aspects will be described in conjunction with the examples above and / or below. Figure 2The example shown may include one or more optional additional features that correspond to one or more aspects associated with the proposed concept, or to those mentioned above (e.g., Figure 1 ) and / or the following (e.g. Figure 3-5 One or more examples of ).

[0069] Some embodiments relate to an optical imaging system that includes, for example: Figure 1 The aforementioned device. Alternatively, an optical imaging system can be used as... Figure 1 The device is part of or connected to it.

[0070] Figure 3 A schematic diagram of a system 300 (e.g., an optical imaging system 300) is shown. This optical imaging system 300 may include reference... Figure 1 The aforementioned device and display device 330. The display device 330 may be a component of the computer system 320. For example, the microscope 310 may include or be communicatively coupled to the device. Alternatively, the computer system 320 may include the device. The microscope 310 may be communicatively coupled to the display device 330. Thus, the device may transmit display signals from the microscope 310 to the display device 330 or a storage device, such as a frame buffer.

[0071] Figure 3 A schematic diagram of system 300 is shown, which is configured to perform the methods described herein, for example, referring to... Figure 4 or Figure 5 The method is illustrated. System 300 includes a microscope 310 and a computer system 320. The microscope may include the devices described above, such as reference optics. Figure 1 The apparatus is shown. Microscope 310 is configured to acquire images and is connected to computer system 320. Computer system 320 is configured to perform at least a portion of the methods described herein. Computer system 320 may be configured to execute machine learning algorithms. Computer system 320 and microscope 310 may exist as separate entities or may be integrated into a common housing. Computer system 320 may be part of the central processing system of microscope 310, and / or may constitute a sub-component of microscope 310, such as a sensor, actuator, camera, or illumination unit.

[0072] Computer system 320 may be a local computer device (such as a personal computer, laptop, tablet, or mobile phone) having one or more processors and one or more storage devices, or it may be a distributed computer system (such as a cloud computing system having one or more processors and one or more storage devices distributed in various locations, such as local clients and / or one or more remote server clusters and / or data centers). Computer system 320 may include any circuitry or combination of circuits. In one embodiment, computer system 320 may include one or more processors, which may be of any type. Here, "processor" may mean any type of computing circuitry, such as, but not limited to, a microprocessor for a microscope or microscope component (e.g., a camera), a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field-programmable gate array (FPGA), or any other type of processor or processing circuitry. Other types of circuitry that may be included in computer system 320 may be custom circuitry, application-specific integrated circuits (ASICs), etc., such as one or more circuits (e.g., communication circuitry) used in wireless devices like mobile phones, tablets, laptops, two-way radios, and similar electronic systems. Computer system 320 may include one or more storage devices, which may include one or more storage elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives for processing removable media such as optical discs (CDs), flash memory cards, digital video discs (DVDs), etc. Computer system 320 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, trackball, touchscreen, voice recognition device, or any other device that allows a system user to input information into and receive information from computer system 320.

[0073] Further details and aspects will be described in conjunction with the examples above and / or below. Figure 3 The example shown may include one or more optional additional features that correspond to one or more aspects associated with the proposed concept, or to those mentioned above (e.g., Figure 1-2 ) and / or the following (e.g. Figure 5 One or more examples of ).

[0074] Figure 4An example flowchart of method 400 is shown. Method 400 is applicable to an optical imaging system and includes receiving 410 sensor data from a sensor of the optical imaging system. This sensor data is used to indicate a real-time view of a sample observed through a microscope via the optical imaging system. Furthermore, method 400 includes generating 420 a visual overlay based on the sensor data, which contains a visualization of the real-time view and an overlay configuration window. This configuration window is used to set sample measurement parameters. Additionally, method 400 includes transmitting 430 a display signal indicating the visual overlay. This display signal can be transmitted to a display device or storage device (such as a frame buffer) as part of the optical imaging system. Method 400 can be derived from a reference... Figure 1 The aforementioned device performs the operation.

[0075] Further details and aspects will be described in conjunction with the examples above and / or below. Figure 3 The example shown may include one or more optional additional features that correspond to one or more aspects associated with the proposed concept, or to those mentioned above (e.g., Figure 1-3 One or more examples of ) and / or the following (e.g. Figure 5 One or more examples of ).

[0076] Figure 5 Another example flowchart of method 500 is shown. Method 500 can be provided by a device (as in the reference). Figure 1 The described device works in conjunction with a display device and an input device.

[0077] In step 502, method 500 begins. In step 504, the user can perform an icon selection action. Icon selection can be achieved through an input device (such as a mouse, touchpad, or keyboard). Icon selection information can be received at this device. The icon may be associated with measurement parameters and / or geometric elements. Based on the icon selection information, the device can generate a second visual overlay. The generated second visual overlay can be transmitted from the device to a display device.

[0078] A second visual overlay (containing a real-time view of the sample) can be displayed on a display device. In step 506, the user can select and / or generate geometric elements. For example, the user can select a pre-existing geometric element by clicking on it. The device can receive information about selecting a pre-existing geometric element and / or generating a geometric element. Based on this information, the device can generate an adapted second visual overlay. This adapted second visual overlay can indicate all possible actions. The adapted second visual overlay can be transmitted from the device to the display device.

[0079] The display device can show all possible actions in 508. In 510, an action can be highlighted when the mouse hovers over the display device. In 512, the user can select (especially by clicking) measurement parameters and / or geometric elements. Information about the selection of geometric elements and / or measurement parameters in 512 can be transmitted to the device. The device can perform the measurement action based on this information. In 514, the method ends.

[0080] Further details and aspects will be described in conjunction with the examples above. Figure 4 The example shown may include one or more optional additional features that correspond to one or more aspects associated with the proposed concept, or to those mentioned above (e.g., Figure 1-4 One or more examples of ).

[0081] Some or all of the steps of the method may be performed by (or using) hardware devices, such as processors, microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important steps of the method may be performed by such devices.

[0082] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. This implementation can be carried out using non-volatile storage media, such as digital storage media like floppy disks, DVDs, Blu-ray discs, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory, on which electronically readable control signals are stored. These control signals can (or can be used to) cooperate with a programmable computer system to execute corresponding methods. Therefore, the digital storage medium can be computer-readable.

[0083] Some embodiments of the invention include a data carrier having electronically readable control signals that are capable of working in conjunction with a programmable computer system to perform one of the methods described herein.

[0084] Generally, embodiments of the present invention can be implemented as a computer program product having program code, which, when run on a computer, performs one of the methods. For example, the program code may be stored on a machine-readable medium.

[0085] Other embodiments include a computer program for performing one of the methods described herein, the program being stored on a machine-readable medium.

[0086] In other words, embodiments of the present invention provide a computer program having program code that, when run on a computer, performs one of the methods described herein.

[0087] Therefore, another embodiment of the invention provides a storage medium (or data carrier, or computer-readable medium) having a computer program stored thereon, which, when executed by a processor, performs one of the methods described herein. Data carriers, digital storage media, or recording media are typically tangible and / or non-transient. Another embodiment of the invention is an apparatus as described herein, comprising a processor and a storage medium.

[0088] Therefore, another embodiment of the invention provides a data stream or signal sequence representing a computer program for performing one of the methods described herein. This data stream or signal sequence can, for example, be configured to be transmitted via a data communication connection, such as via the Internet.

[0089] Another embodiment includes a processing means, such as a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

[0090] Another embodiment includes a computer on which a computer program for performing one of the methods described herein is mounted.

[0091] According to the invention, another embodiment includes an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a recipient. For example, the recipient may be a computer, a mobile device, or a storage device. The apparatus or system may, for example, include a file server for transmitting the computer program to the recipient.

[0092] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0093] If some aspects describe an apparatus or system, then these aspects should also be understood as descriptions of the corresponding methods, and vice versa. For example, modules, apparatus, or functional aspects of an apparatus or system may correspond to features (such as method steps) in the corresponding method. Therefore, aspects related to the method should also be understood as descriptions corresponding to corresponding modules, elements, attributes, or functional features of the corresponding apparatus / system.

[0094] The following claims are now included in the detailed description, each of which may stand alone as an example. It should also be noted that although a dependent claim in a claim refers to a specific combination with one or more other claims, other examples may also include combinations of that dependent claim with the subject matter of any other dependent or independent claim. Such combinations are expressly stated herein unless an individual case expressly states that a particular combination is not included. Furthermore, features contained in a claim should also apply to any other independent claim, even if the claim is not directly defined as dependent on that independent claim.

[0095] Aspects and features associated with a particular example in a previous example may also be combined with one or more subsequent examples to replace the same or similar features in the subsequent examples, or to introduce these features additionally into the subsequent examples.

[0096] List of reference numerals 100 Optical Imaging System 110 samples 122 Sensors 130 equipment 132 interface 134 processor 136 storage devices 200 Configuration Panel 220 yuan Icons 224, 226, and 228 230 trapezoid Measurement parameters: 232, 238, 242, 256, 264 Areas 236, 234, 240, 254, and 262 associated with the measured parameters. 250 points 300 system 310 Microscope 320 Computer System 330 display device 400 methods 410 Receive sensor data from the sensor of the optical imaging system. 420 Generate visual overlay layers 430 Sends a display signal instructing the visual overlay layer. 500 methods 502 Start Method 504 Select Measurement Parameters / Geometric Elements 506 Select / Generate Geometry Elements 508 indicates possible actions. 510 Highlight during mouse hover 512 Select the desired measurement parameters / geometric elements 514 End Method

Claims

1. An apparatus (130) for an optical imaging system, comprising one or more processors (134) and one or more storage devices (136), wherein, The device (130) is configured to receive sensor data of a sensor of the optical imaging system from the optical imaging system, the sensor data being indicative of a real-time view of a sample acquired by a microscope of the optical imaging system; generate a visual overlay based on the sensor data, the visual overlay comprising a visual representation of the real-time view and icons indicative of a plurality of possible actions associated with a measurement; and send a display signal indicative of the visual overlay.

2. The device (130) according to claim 1, wherein The device (130) is further configured to receive a trigger signal, the trigger signal being indicative of a selection of the icons by a user; generate a second visual overlay based on the sensor data and the trigger signal, the second visual overlay comprising the visual representation of the real-time view and a visual representation of a plurality of possible actions; and send a second display signal indicative of the second visual overlay.

3. The device (130) according to claim 2, wherein The device (130) is further configured to receive a second trigger signal, the second trigger signal being indicative of an increased likelihood of a selection of an action of the plurality of possible actions by the user; adapt the second visual overlay by highlighting the action of the plurality of possible actions based on the second trigger signal; and send the adapted second display signal.

4. The device (130) according to claim 3, wherein the second visual overlay is for highlighting at least one of a measurement parameter or a pre-existing geometric element.

5. The device (130) according to any one of claims 2 to 4, wherein the second visual overlay comprises a representation of each possible action of the plurality of actions associated with the selected icons.

6. The device (130) according to any one of the preceding claims, wherein the plurality of possible actions comprises at least one of generating a geometric element or defining a measurement parameter.

7. The device (130) according to any one of the preceding claims, wherein the plurality of possible actions comprises different geometric elements to be generated.

8. The device (130) according to any one of the preceding claims, wherein the plurality of actions comprises different measurement parameters to be defined.

9. The device (130) according to any one of claims 2 to 8, wherein, The device (130) is further configured to receive a third trigger signal, the third trigger signal being indicative of a selection of a pre-existing geometric element by the user; and wherein generating the second visual overlay or the adapted second visual overlay is further based on the third trigger signal.

10. The device (130) according to claim 9, wherein the selected pre-existing geometric element is a primary geometric element and the adapted second visual overlay comprises a highlighting of possible secondary geometric elements.

11. The device (130) according to any of the preceding claims, wherein The device (130) is further configured to receive a fourth trigger signal, the fourth trigger signal being indicative of a selection of an action from the plurality of possible actions; and performing at least one of generating a geometric element or executing a measurement based on the fourth trigger signal.

12. An optical system (100) comprising: a device (130) according to any one of the preceding claims.

13. A method (400) for a device of an optical imaging system, comprising: receiving (410), from an optical imaging system, sensor data of a sensor of the optical imaging system, the sensor data being indicative of a real-time view of a sample acquired by a microscope of the optical imaging system; generating (420), based on the sensor data, a visual overlay, the visual overlay comprising a visual representation of the real-time view and icons indicative of a plurality of possible actions associated with a measurement; and transmitting (430) a display signal indicative of the visual overlay.

14. The method (400) of claim 13, further comprising: receiving a trigger signal, the trigger signal being indicative of a selection of the icons by the user; generating, based on the sensor data and the trigger signal, a second visual overlay, the second visual overlay comprising the visual representation of the real-time view and visual representations of a plurality of possible actions; and transmitting a second display signal indicative of the second visual overlay.

15. A computer program having a program code for performing the method of claim 13 or 14, when the computer program is executed on a processor.

Images