Apparatus for optical imaging systems, optical imaging systems, methods, and computer programs

Abstract

The example relates to an apparatus for an optical imaging system comprising one or more processors and one or more storage devices. The apparatus is configured to receive sensor data from the sensors of the optical imaging system. The sensor data is received from the optical imaging system, for example, from the sensors. The sensor data represents a live view of the sample through the microscope of the optical imaging system. Furthermore, the apparatus is configured to generate a visual overlay including a visual representation and icons of the live view. The visual overlay is generated based on the sensor data. Furthermore, the apparatus is configured to transmit a display signal indicating the visual overlay.

Description

Technical Field

[0002] ,

[0005] ,

[0004] , , , ,

[0003] ,

[0001] The present example relates to an optical imaging system such as an optical imaging system used in an industrial environment to monitor and control a manufacturing process or the quality of a workpiece, and also relates to an optical imaging system, a method, and a computer program. <​​​​​​​​​​​​​​​​​​This example provides an apparatus for an optical imaging system, comprising one or more processors and one or more storage devices. The apparatus is configured to receive sensor data from the sensors of the optical imaging system. The sensor data is received from the optical imaging system, for example, from the sensors. The sensor data represents a live view of the sample through the microscope of the optical imaging system. Furthermore, the apparatus is configured to generate a visual overlay including a visual representation of the live view and icons. The visual overlay is generated based on the sensor data. The icons represent several possible actions associated with the measurement. Furthermore, the apparatus is configured to send a display signal indicating the visual overlay to, for example, a display device. In this way, the display device can be controlled or triggered to display the live view of the sample and icons, for example, to facilitate setting up the measurement for process control. The actions may be the generation of geometric elements and / or the selection of measurement parameters. For example, the icons may represent several actions necessary to define a measurement, such as dimensional measurement.

[0006] In one example, the device may be further configured to receive a trigger signal indicating a user's selection of an icon and generate a second visual overlay including a visual representation of the live view and a visual presentation of several possible actions. The generation of the second visual overlay is based on sensor data and the trigger signal. Furthermore, the device may be configured to send a second display signal indicating the second visual overlay to, for example, a display device. By generating the second visual overlay, the device can control or trigger the display device to display auxiliary information for the user, such as which geometric elements, measurement parameters can be selected to set up the measurement. In this way, the user can be easily informed of the possible adjustments that can be made to set up the measurement for process control.

[0007] In one example, the device may be further configured to receive a second trigger signal indicating an increased likelihood of selecting one of several possible actions. The selection may be performed by the user. Furthermore, the device may be configured to adapt a second visual overlay by highlighting the action (with an increased likelihood of being selected) among several possible actions. The adaptation is based on the second trigger signal. Furthermore, the device may be configured to send the adapted second display signal to, for example, a display device. The increased likelihood of selecting an action may be caused by user intent. For example, the user may move the mouse cursor near a geometric element. This movement may indicate the user's intent to select the geometric element. Thus, the device may highlight the geometric elements near or below the mouse cursor. In this way, the user can be easily informed of the selectable geometric elements and / or measurement parameters.

[0008] For example, a second visual overlay could be used to highlight measurement parameters and / or existing geometric elements. In this way, the user can receive visual feedback before making a selection. This can improve the user experience, for example, by allowing the selection to be made with greater confidence.

[0009] For example, the second visual overlay might include representations of each possible action associated with the selected icon. In this way, the user can receive visual feedback on all possible actions associated with the selected measurement. Thus, the user can be informed of each possibility for setting up the measurement.

[0010] In one example, multiple possible actions may include generating geometric elements and / or defining measurement parameters. In this way, measurements for process control can be fully defined based on multiple possible actions. Therefore, users can use a simple workflow to generate measurements for process control.

[0011] For example, multiple possible actions may involve different geometric elements to be generated and / or selected. For instance, multiple geometric elements for a sample may already be defined and are also called existing geometric elements. Therefore, the user can select the appropriate geometric element from the existing elements. For example, different geometric elements among multiple geometric elements may belong to different measurements. In this way, the user can easily select the desired geometric element.

[0012] For example, multiple possible actions may involve different measurement parameters to be defined. For instance, a user might desire a dimensional measurement to determine the distance between a primary and a secondary geometric element. However, the primary geometric element might be a circle, and the distance to be measured could be from the center of the circle or from its edges. Therefore, different measurement parameters can be presented to the user. In this way, the user can easily select the desired measurement parameter.

[0013] In one example, the device may be further configured to receive a third trigger signal indicating the user's selection of existing geometric elements. Furthermore, the device may be configured to generate a second visual overlay and / or an adapted second visual overlay based on the third trigger signal. For example, if multiple geometric elements are required to define a measurement, the device can highlight only the geometric elements that can be used together with the selected existing geometric elements. In this way, setting up the measurement can be simplified.

[0014] In one example, the selected existing geometric element may be a primary geometric element, and the applied second visual overlay may include highlighting of possible secondary geometric elements. In this way, the user can receive information about the secondary geometric elements associated with the primary geometry. The primary and secondary geometric elements may be the geometric elements required to set up / perform a measurement.

[0015] In one example, the device may be further configured to receive a fourth trigger signal indicating the selection of one of several possible actions, and to perform the generation and / or measurement of geometric elements based on the fourth trigger signal. In this way, the device can implement specific means for enabling the setting and / or execution of measurements for process control.

[0016] Embodiments provide devices such as those described herein, for example, those described above or below.

[0017] Embodiments provide a method for an apparatus for an optical imaging system. The method includes receiving sensor data from the optical imaging system's sensors, which represent a live view of a sample through the microscope of the optical imaging system. The method further includes generating a visual overlay based on the sensor data, which includes a visual representation of the live view and icons indicating a plurality of possible actions associated with the measurement, and transmitting a display signal indicating the visual overlay.

[0018] Various examples of this disclosure relate to corresponding computer programs having program code for performing the above-described methods when the computer program is executed on a processor. [Brief explanation of the drawing]

[0019] Some examples of apparatus and / or methods are described below, for illustrative purposes only, with reference to the attached drawings. [Figure 1] This is a schematic diagram showing an example of equipment for an optical imaging system. [Figure 2] This figure shows examples of multiple possible actions. [Figure 3] This is a schematic diagram illustrating the system. [Figure 4] An example of the method is shown in the flowchart. [Figure 5] Here is another example of the method, as shown in the flowchart.

[0020] Detailed explanation Various examples are described in more detail below, with reference to the attached diagrams illustrating several examples. In the diagrams, the thickness of lines, layers, and / or areas may be exaggerated for clarity.

[0021] Figure 1 shows a schematic diagram of an example of a device 130 for an optical imaging system 100. The device 130 is responsible for the task of controlling various aspects of the microscope of the optical imaging system 100, which may be an optical imaging system 100 for process control in an industrial environment, and various aspects of the optical imaging system 100 as a whole, as well as / or the task of processing various types of sensor data from the optical imaging system 100. As a result, the device 130 can be implemented as a computer system that works in conjunction with various components of the optical imaging system 100, such as the sensor 122.

[0022] As shown in Figure 1, the device 130 comprises one or more processors 134 and one or more storage devices 136. Optionally, the device 130 further comprises one or more interfaces 132. One or more processors 134 are coupled to one or more storage devices 136 and to the optional one or more interfaces 132. Generally, the functions of the device 130 may be provided by one or more processors 134 (for example, for generating display signals) cooperating with one or more interfaces 132 (for exchanging data) and / or one or more storage devices 136 (for storing and / or retrieving data).

[0023] The device 130 is configured to receive sensor data from the sensors of the optical imaging system 100. The sensor data is received from the optical imaging system 100, for example, from sensor 122, frame buffer, and storage device. The sensor data shows a live view of sample 110 through the microscope (not shown) of the optical imaging system 100. The live view may be acquired by sensor 122. Sample 110 may be a sample to be tested. For example, sample 110 may be a workpiece in an industrial manufacturing process. For example, the optical imaging system 100 may be configured for process control of sample 110. The optical imaging system 100 may be designed for use in a laboratory or factory environment. The optical imaging system 100 can be used during a manufacturing process to inspect sample 110, such as a product, to ensure that it meets the required specifications and standards.

[0024] The optical imaging system 100 may have functions such as digital imaging (e.g., sensor 122), video capabilities (e.g., display device 330), and precision measurement tools, which enable users such as technicians to identify defects, monitor changes in the manufacturing process, and / or make adjustments as needed to ensure product quality. In addition, many process control microscopes have automated systems that can quickly analyze images and provide feedback to the production line, further streamlining the quality process control. However, there may be a need to adjust the measurements for process control, for example, to compensate for systematic errors caused by changes in the manufacturing process or to set up measurements for new workpieces.

[0025] Device 130 is configured to generate a visual overlay that includes a visual representation of the live view and icons. The visual overlay is generated based on sensor data. The icons indicate a plurality of possible actions associated with the measurement. An icon is a graphic symbol or image that represents a software application, file, folder, and / or action in a part of a processing circuit, such as on a computer. The icons can be designed to be easily recognizable and memorable. For example, the icons can be designed to use, for example, simple and minimal visuals to inform the user of the selected measurement. For example, the icons can use metaphors or symbols to convey meaning and function, such as displaying a plurality of possible new measurements together with geometric elements.

[0026] Possible actions can be the generation of geometric elements, the selection of existing geometric elements, the selection of measurement parameters, and / or the definition of measurement parameters. For example, a plurality of possible actions can enable the user to set up measurements for process control.

[0027] An icon can indicate a plurality of possible actions. Thus, each possible action of the plurality of possible actions for measurement can be notified to the user by the icon. Alternatively, the icon can also present only some of the possible actions of the plurality of possible actions. For example, the icon can indicate x different measurement parameters, can indicate y different measurement parameters, and x < y. In this way, the recognition of the icon can be enhanced. In this case, each possible action of the plurality of possible actions can be displayed to the user using a second visual overlay as described later.

[0028] Live view refers to the real-time display of sample 110. For example, the live view may enable the user to view a continuous and up-to-date display of sample 110 being acquired by a microscope.

[0029] The live view can be particularly useful in a process control microscope application to enable the user to view an image of sample 110 in real time and make adjustments to the microscope or sample 110 as needed.

[0030] For example, the live view can be used to inspect and monitor the quality of a workpiece during a manufacturing process or to adjust the focus and positioning of the microscope to ensure that the live view is properly acquired.

[0031] The live view can be presented in a separate window or as part of the main application window. An icon can be associated with the live view window, for example, as part of a settings panel. Additionally or alternatively, the icon may be integrated into the live view window. For example, the icon can be used to set measurements for optical inspection such as the dimensional measurement of sample 110.

[0032] For example, the measurement could be an optical inspection such as surface inspection, texture analysis, defect detection, or dimensional measurement. Icons may provide the user with a (first) selection. For example, multiple icons may be displayed, each assigned to a different measurement. The measurement may involve measuring measurement parameters of sample 110, such as length, width, height, diameter, radius, roughness, texture, or structure of sample 110. Therefore, an icon can be defined for the measurement of at least one measurement parameter.

[0033] Dimensional measurement refers to the process of determining the size and dimensions of a structure. For example, dimensional measurement may be used during the manufacturing process to inspect and verify the size and dimensions of a workpiece, such as sample 110. This is done to ensure that the workpiece meets the required specifications and standards, and to detect any deviations or defects that may affect the quality or performance of the workpiece. Dimensional measurement can be performed on a wide range of materials and workpieces, including metal parts, plastic parts, and electronic devices. Therefore, the selection of the desired optical inspection, such as dimensional measurement, can be complex.

[0034] Therefore, the device 130 is further configured to transmit a display signal indicating a visual overlay. For example, the display signal can be transmitted to a display device 330 or a storage device such as a frame buffer. The optical imaging system 100 may include a display device 330. Alternatively, the display device 330 may be external to the optical imaging system 100 and may be communicatively coupled to the optical imaging system 100, for example. The display signal may include information about the visual overlay, such as sensor data, or it may include sensor data and data relating to the visual representation of icons, such as information about elements representing icons of a graphical user interface (GUI) to be displayed. In this way, the display device 330 can be used to display a live view in combination with icons. Optionally, the display signal may include data for controlling the display device 330. Therefore, the device 130 can be triggered to display the visual overlay on the display device 330.

[0035] Displaying icons on the display device can simplify the setup of measurements for process control. For example, it can reduce the number of clicks required to set up measurements for process control. Defining measurements for process control may require multiple clicks. For example, measurements for process control may include generating / selecting (existing) geometric elements and selecting measurement parameters. By selecting an icon, the user can start a workflow to set up measurements for process control using only one icon.

[0036] Furthermore, the visual overlay can provide the user with information about the measurement parameters and / or geometric elements associated with the measurement. In this way, the user can easily select the desired measurement parameters and / or geometric elements.

[0037] Icons can reduce the number of clicks required to set up measurements, for example, by eliminating the need to select icons for defining geometric elements and other icons for defining measurement parameters. Therefore, the time required to perform the task of setting up measurements can be reduced. Furthermore, the GUI can be made simpler. For example, the settings panel containing icons can be made less cluttered. Thus, users can easily set up measurements for process control and / or improve the user experience.

[0038] Instead of using separate icons / buttons to create geometric elements (or secondary geometric elements required for measurement) and measurement parameters, multiple possible solutions (e.g., measurements: angles, circle-to-circle, circle-to-line, circle-to-point; secondary geometric elements: tangents between circles, tangents between circles and points, bisectors) and their respective functions can be selected by clicking a (single) icon (e.g., for angle measurement). Thus, selecting an icon can initiate a workflow for easily setting up measurements. For example, a second visual overlay could enable the user to select and / or generate geometric elements / measurement parameters directly within the live view, for example, by mouse control.

[0039] For example, if it is necessary to create secondary geometric elements, the device 130 can generate an adapted second visual overlay to present possible secondary geometric elements in the live view, for example, after a primary geometric element has been selected by the user (for example, information regarding this selection may be received by a third trigger signal). Thus, the user can receive information about secondary geometric elements associated with the geometric element belonging to the measurement.

[0040] The selection of geometric elements and / or measurement parameter types to be selected / defined can be done by clicking icons. For a simplified user interface, the icons can graphically represent all possible deformations within a single image. In this way, setting up measurements can be made easier.

[0041] The proposed concept is centered around two main components: a microscope equipped with optical components and capable of housing a display device 330 used to view a sample 110, and a device 130 used to control an optical imaging system 100, process sensor data from the microscope, such as a sensor 122, and generate a display signal.

[0042] Generally, an optical microscope system 100 includes a microscope suitable for examining objects that are too small to be examined with the human eye (alone). For example, the microscope may provide optical magnification of a sample such as the sample 110 shown in Figure 1. Thus, the structure of the sample 110, for example, a structure formed by a manufacturing process, can be magnified and measured with improved precision. In modern microscopes, optical magnification is often provided by a camera or imaging sensor, for example, the microscope's optical imaging sensor 122. The microscope may further include one or more optical magnification components used to magnify the image of the sample 110, such as an objective lens.

[0043] Various different types of optical imaging systems exist. For example, optical imaging system 100 may be used for process control. Optical imaging system 100 may comprise an optical system, a camera or detector, and a monitor or display device 330. The optical system may typically comprise optical components such as lenses, mirrors, or other optical components that work together to produce a high-resolution image of the sample 110, e.g., a live view. The camera or detector, e.g., sensor 122, may be used to acquire a live view, e.g., a live view of the sample 110, and convert it into a digital signal that can be displayed on the display device 330 and / or analyzed using software, e.g., software implemented by the apparatus 130. The display device 330 may be used to display the live view acquired by sensor 122, and any additional information such as measurement data, analysis methods, icons, e.g., icons in a settings panel, or possible actions for measurement.

[0044] In one example, the device 130 may be further configured to receive a trigger signal indicating a user's selection of an icon and generate a second visual overlay including a visual representation of the live view and a visual presentation of several possible actions. The generation of the second visual overlay is based on sensor data and the trigger signal. The trigger signal may be the first trigger signal. Furthermore, the device 130 may be configured to transmit a second display signal indicating the second visual overlay. For example, the second display signal may be transmitted to a display device 330 or a storage device such as a frame buffer. The first trigger signal may be received from an input device or a storage device. The input device may be a keyboard and / or controller, which may include a mouse, trackball, touchscreen, voice recognition device, or any other device that allows a system user to input information.

[0045] A second visual overlay can be used via the display device 330 to display to the user multiple possible actions, for example, each possible action. In this way, after selecting an icon, the user can receive an overview of the possible actions related to the measurement associated with the icon. Thus, the user can be informed of the geometric elements and / or measurement parameters associated with the measurement in an easy manner. Therefore, multiple icons for multiple geometric elements / measurement parameters associated with the measurement (separate icons / buttons for creating geometric elements and measurement parameters with multiple possible solutions) can be omitted.

[0046] In one example, the device 130 may be further configured to receive a second trigger signal indicating an increased likelihood of selecting one of several possible actions. The second trigger signal may be received from an input device (e.g., the same input device as in the case of the first trigger signal) or a storage device. The selection may be made by the user. Furthermore, the device 130 may be configured to adapt a second visual overlay by highlighting the action (that is more likely to be selected by the user) among several possible actions. The adaptation may be based on the second trigger signal. Furthermore, the device 130 may be configured to send the adapted second display signal to, for example, a display device 330 or a storage device such as a frame buffer. The second visual overlay may provide the user with assistance in setting up the measurement. For example, the second visual overlay may be for highlighting the user's possible selections. Before a selection can be made, or the user may be informed in advance of the possibility that, for example, a mouse cursor click at a particular location could lead to an intended action. For example, the second visual overlay may be for highlighting geometric elements near or below the mouse cursor (e.g., during mouseover). For example, a geometric element can be highlighted while the mouse is hovering over it. For instance, device 130 may receive information about the mouseover via a second trigger signal. The second trigger signal may indicate the mouseover. Thus, the user can receive visual feedback before selecting, for example, an existing geometric element.

[0047] For example, multiple measurement parameters can be associated with a measurement linked to an icon selected by the user. Multiple measurement parameters, such as distance measurements from the center of a circle to a secondary geometric element, or distance measurements from the edge of a circle to a secondary geometric element, can be displayed on the display device 330. However, multiple measurement parameters can become crowded. Therefore, the user experience can be improved by highlighting measurement parameters that may be selected by the user and / or are associated with the mouse cursor position.

[0048] Additionally or alternatively, a second visual overlay may be used to highlight possible secondary geometric elements within the live camera view after a primary geometric element has been selected. This can be done independently of the mouse cursor position. For example, after the selection of an existing primary geometric element, or after the generation of a new primary geometric element, the second visual overlay may highlight possible secondary geometric elements associated with the primary geometric element to define measurements for process control. Apparatus 130 may receive information regarding the selection / generation of a primary geometric element via a third trigger signal. A desired secondary geometric element can then be selected, for example, by the user clicking it. In this way, the user can receive visual feedback regarding possible secondary geometric elements. For example, the user may receive information regarding the user's selection of an existing geometric element based on a third trigger signal, as described herein.

[0049] For example, the second visual overlay may be for highlighting measurement parameters and / or existing geometric elements. For instance, if measurements are to be performed, each measurement can be displayed when the mouse hovers over each area (e.g., different angles of two intersecting lines). For example, device 130 may receive information about mouseovers by receiving a second trigger signal. Thus, device 130 can highlight the measurement parameters associated with each area within the second visual overlay. The measurement parameters can be made known to the user by sending the second visual overlay to, for example, a display device 330 or a storage device such as a frame buffer. The user can then select the measurement parameters by clicking on them.

[0050] In one example, the second visual overlay may include representations of each possible action associated with the selected icon. For instance, device 130 may receive information about the icon selection via a second trigger signal. Based on the second trigger signal, device 130 may generate a second visual overlay to display to the user each possible action related to the measurement associated with the selected icon. In this way, the user can receive information about each possible action that can be performed. Thus, the user experience may be improved and / or the measurement setup may be made easier.

[0051] For example, multiple possible actions may include generating geometric elements and / or defining measurement parameters. For instance, an action could be any task related to measurement, particularly one related to setting up the measurement.

[0052] In one example, multiple possible actions may involve different geometric elements to be generated and / or selected. For example, measurement may involve primary and secondary geometric elements. A first possible action may be generating / selecting primary geometric elements. A second possible action may be generating / selecting secondary geometric elements. A second visual overlay may present both primary and secondary geometric elements. As described herein, highlighting of secondary geometric elements may be performed based on the selection of primary geometric elements.

[0053] For example, multiple possible actions may include different measurement parameters to be defined. For instance, dimensional measurement may include measurement parameters such as angles, circle-to-circle, circle-to-line, circle-to-point, tangents between circles, tangents between a circle and a point, and angle bisectors, which may be associated with icons. Thus, multiple measurement parameters can be displayed to the user, allowing them to easily select the desired measurement parameter.

[0054] In one example, the device 130 may be further configured to receive a third trigger signal indicating a user's selection of existing geometric elements. The third trigger signal may be received from an input device (e.g., the same input device as for the first trigger signal) or a storage device. Furthermore, the device 130 may be configured to generate a second visual overlay and / or an adapted second visual overlay based on the third trigger signal. For example, if multiple geometric elements are required to define a measurement, the device 130 may highlight only the geometric elements that can be used together with the selected existing geometric elements. For example, the device 130 may receive information regarding the selection of existing geometric elements via the third trigger signal. The selected existing geometric elements may be assigned as primary geometric elements. Thus, the device 130 can determine possible secondary geometric elements associated with the primary geometric elements. The device 130 may further adapt the second visual overlay based on the possible secondary geometric elements. In this way, the user can receive an adapted second visual overlay that displays only the possible secondary geometric elements.

[0055] In one example, the selected existing geometry element may be a primary geometry element, and the applied second visual overlay may include highlights of possible secondary geometry elements. These possible secondary geometry elements may be geometry elements that can be used to perform measurements together with the primary geometry element. In this way, the user can receive a visual representation of the possible geometry elements for setting up the measurement.

[0056] In one example, the device 130 may be further configured to receive a fourth trigger signal indicating the selection of one of several possible actions, and to perform the generation and / or measurement of geometric elements based on the fourth trigger signal. The fourth trigger signal may be received from an input device (e.g., the same input device as in the case of the first trigger signal) or a storage device. In this way, the device can implement specific means to enable the setting and / or execution of measurements for process control. Thus, different measurements can be set. The setting of measurements may depend on the first and second trigger signals, as well as / or the third and / or fourth trigger signals.

[0057] For example, when a user selects an icon, an exemplary workflow may be initiated as follows: Device 130 may generate a second visual overlay. The second visual overlay may include several possible actions related to the measurement associated with the selected icon. When the user moves the mouse cursor, Device 130 may receive a second trigger signal. The second trigger signal may indicate a mouseover. Based on the second trigger signal, Device 130 may generate an adapted second visual overlay that, for example, highlights an existing geometric element associated with the mouseover. Thus, the user can be informed of possible selections of existing geometric elements during the mouseover. The user can select the existing geometric element by clicking on the highlighted element during the mouseover. Device 130 may receive information regarding the user's selection of an existing geometric element via a third trigger signal. Based on the third trigger signal, Device 130 may adapt the second visual overlay to highlight possible secondary geometric elements associated with the selected existing geometric element. Thus, the user can be informed of the possibility of selecting secondary geometric elements. The device 130 may receive information regarding the user's selection of secondary geometric elements via another third trigger signal. Based on the other third trigger signal, the device 130 may adapt a second visual overlay, for example, to display possible measurement parameters. Thus, the user can be informed of the selection of measurement parameters. When the user moves the mouse cursor, the device 130 may receive another second trigger signal. The other second trigger signal may indicate a mouseover associated with a measurement parameter. Based on the other second trigger signal, the device 130 may generate an adapted second visual overlay that highlights the measurement parameter associated with the mouseover, for example. Thus, the user can be informed of the possible selection of measurement parameters. The user can select a measurement parameter by clicking on the highlighted one.The device 130 may receive information regarding the selection of measurement parameters via a fourth trigger signal. In this way, the user can easily set up measurements for process control. This workflow is for illustrative purposes only. The number and / or order of trigger signals and / or the trigger signals used may vary.

[0058] As shown in Figure 1, one or more optional interfaces 132 are coupled to one or more processors 134 in the device 130. In the example, one or more processors 134 may be implemented using any processing means, such as one or more processing units, one or more processing devices, processors, computers, or programmable hardware components that can operate with appropriately adapted software. Similarly, the functions of the one or more processors 134 described may be implemented in software, which is then executed by one or more programmable hardware components. Such hardware components may include general-purpose processors, digital signal processors (DSPs), microcontrollers, and the like. One or more processors 134 can control one or more interfaces 132, and as a result, any data transfers that occur through one or more interfaces 132, and / or any interactions that one or more interfaces 132 may be involved in, can be controlled by one or more processors 134.

[0059] In one embodiment, the apparatus 130 may include memory, for example, one or more storage devices 136, and at least one or more processors 134 operably coupled to the memory and configured to perform the methods described below.

[0060] In the example, one or more interfaces 132 may correspond to any means for acquiring, receiving, transmitting, or providing analog or digital signals or information, such as any connectors, contacts, pins, registers, input ports, output ports, conductors, lanes, etc., that enable the provision or acquisition of signals or information. One or more interfaces 132 may be wireless or wired and may be configured to communicate with further internal or external components, such as transmitting or receiving signals, information.

[0061] The device 130 may be a computer, processor, control unit, (field)programmable logic array ((F)PLA), (field)programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASIC), integrated circuit (IC), or system-on-a-chip (SoC) system. The device 130 may be part of the optical imaging system 100. Alternatively, the device 130 may be outside the optical imaging system 100 and may be part of, for example, a display device 330.

[0062] Further details and embodiments will be mentioned in relation to the examples described below. The example shown in Figure 1 may have one or more additional optional features corresponding to one or more embodiments mentioned in relation to the proposed concept, or to one or more examples described below (e.g., Figures 2 to 5).

[0063] Figure 2 shows an example of multiple possible actions. For illustrative purposes, Figure 2 shows two or more mouse cursors. As mentioned earlier, multiple possible actions can be displayed overlaid on the sample's live view.

[0064] As shown in Figure 2, multiple possible actions belong to multiple measurement parameters. For example, the distance between circle 220 and trapezoid 230 can be measured. However, there may be possibilities to measure the distance between circle 220 and trapezoid 230 using different measurement parameters, namely, from the center of circle 220 or from the edge of circle 220. During mouseover, the device (e.g., the device described with reference to Figure 1) can generate an adapted second visual overlay to highlight a measurement parameter, for example, measurement parameter 232 from the center of circle 220. If the mouse cursor is within area 234, measurement parameter 232 can be highlighted. For example, area 234 may be assigned to measurement parameter 232. Thus, the user can easily select the desired measurement parameter based on several areas defined within the live view. Optionally, area 236 may be assigned to measurement parameter 238, and area 240 may be assigned to measurement parameter 242. For example, the device may generate a second visual overlay based on the selection of icon 228. The device may receive information regarding the selection of icon 228 via a first trigger signal.

[0065] For example, the second measurement could be a tangent measurement. The tangent can be measured from point 250. The second measurement can be selected by clicking icon 226. The tangent 256 may be highlighted while hovering the mouse over area 254. For example, the third measurement could be a measurement of the angle associated with trapezoid 230. The third measurement can be selected by clicking icon 224. The measurement parameter 264 may be highlighted while hovering the mouse over area 262.

[0066] Using the icons in the settings panel 210 reduces the number of checkboxes / icon options required to define specific geometric elements and / or measurement parameters. Furthermore, providing a visual overlay allows for intuitive user interaction with the live image. The overlay can enable the user to uniquely select the desired geometric elements and / or measurement parameters. In this way, the user receives a visual representation of the action they are about to select.

[0067] Additionally or alternatively, touch controls can be used to receive user input. For example, touch controls can be enabled when all possible options are displayed after selecting a desired geometric element and / or measurement parameter and / or icon. In this case, highlighting of possible actions may not be required.

[0068] Further details and embodiments are referenced in relation to the examples described above and / or below. The example shown in Figure 2 may have one or more optional additional features corresponding to the proposed concept or one or more embodiments referenced in relation to one or more examples described above (e.g., Figure 1) and / or below (e.g., Figures 3-5).

[0069] Some embodiments relate to an optical imaging system comprising the apparatus described in relation to Figure 1. Alternatively, the optical imaging system may be part of the apparatus described in relation to Figure 1, or connected to the apparatus.

[0070] Figure 3 shows a schematic diagram of system 300, for example, an optical imaging system 300. The optical imaging system 300 may comprise the apparatus described with reference to Figure 1 and a display device 330. The display device 330 may be part of a computer system 320. For example, a microscope 310 may comprise the apparatus or be communicatively coupled to the apparatus. Alternatively, the computer system 320 may comprise the apparatus. The microscope 310 can be communicatively coupled to the display device 330. Thus, the apparatus can transmit display signals from the microscope 310 to the display device 330 or to a storage device such as a frame buffer.

[0071] Figure 3 shows a schematic diagram of a system 300 configured to carry out the methods described herein, for example, the methods described with reference to Figure 4 or Figure 5. The system 300 includes a microscope 310 and a computer system 320. The microscope may be equipped with the apparatus described above, for example, with reference to Figure 1. The microscope 310 is configured to capture images and is connected to the computer system 320. The computer system 320 is configured to carry out at least a portion of the methods described herein. The computer system 320 may be configured to run machine learning algorithms. The computer system 320 and the microscope 310 may be separate entities, or they may be integrated within a single common housing. The computer system 320 may be part of the central processing system of the microscope 310, and / or the computer system 320 may be part of the dependent components of the microscope 310, such as sensors, actors, cameras, or illumination units of the microscope 310.

[0072] The computer system 320 may be a local computer device (e.g., a personal computer, laptop, tablet computer, or mobile phone) comprising one or more processors and one or more storage devices, or it may be a distributed computer system (e.g., a cloud computing system comprising one or more processors and one or more storage devices distributed to various locations such as local clients and / or one or more remote server farms and / or data centers). The computer system 320 may include any circuit or combination of circuits. In one embodiment, the computer system 320 may include one or more processors, which can be of any kind. As used herein, the processor may be intended to be any kind of computing circuit, such as a microprocessor for a microscope or microscopic component (e.g., a camera), a microcontroller, a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multicore processor, a field-programmable gate array (FPGA), or any other kind of processor or processing circuit. Other types of circuits that may be included in the computer system 320 may be custom circuits, application-specific integrated circuits (ASICs), etc., such as one or more circuits (communication circuits, etc.) used in wireless devices such as mobile phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 320 may also include one or more storage devices that may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives that handle removable media such as compact discs (CDs), flash memory cards, digital video discs (DVDs), etc.The computer system 320 may also include a display device, one or more speakers and a controller which may include a keyboard and / or mouse, trackball, touchscreen, voice recognition device, or any other device which enables a user of the system to input information into and receive information from the computer system 320.

[0073] Further details and embodiments are referred to in relation to the examples described above and / or below. The example shown in Figure 3 may have one or more embodiments referred to in relation to the proposed concept, or one or more optional additional features corresponding to one or more examples described above (e.g., Figures 1-2) and / or below (e.g., Figure 5).

[0074] Figure 4 shows a flowchart of an example of Method 400. Method 400 for an optical imaging system includes receiving sensor data from the optical imaging system's sensors 410. The sensor data represents a live view of the sample through the microscope of the optical imaging system. Furthermore, Method 400 includes generating a visual overlay based on the sensor data, including a visual presentation of the live view and a settings window. The settings window is for setting up the measurement of the sample. Furthermore, Method 400 includes transmitting a display signal 430 that displays the visual overlay. The display signal may be transmitted to a display device or a storage device such as a frame buffer that is part of the optical imaging system. Method 400 may be carried out by the apparatus described with reference to Figure 1.

[0075] Further details and embodiments are referred to in relation to the examples described above and / or below. The example shown in Figure 3 may include one or more optional additional features corresponding to one or more embodiments referred to in relation to the proposed concept or one or more examples described above (e.g., Figures 1-3) and / or below (e.g., Figure 5).

[0076] Figure 5 shows a flowchart of another example of Method 500. Method 500 can be carried out by the apparatus (described with reference to Figure 1) in conjunction with a display device and an input device.

[0077] In 502, method 500 is initiated. In 504, the user can select an icon. The selection of an icon can be performed using an input device such as a mouse, touchpad, or keyboard. Information regarding the selection of an icon can be received by the device. The icons may be associated with measurement parameters and / or geometric elements. Based on the information regarding the selection of an icon, the device can generate a second visual overlay. The generated second visual overlay can be transmitted from the device to a display device.

[0078] A second visual overlay (including a sample live view) can be displayed on the display device. In 506, the user can select and / or generate geometric elements. For example, the user can select a geometric element by clicking on an existing geometric element. Information regarding the selection of an existing geometric element and / or the generation of a geometric element can be received by the device. Based on this information, the device can generate an adapted second visual overlay. The adapted second visual overlay may show all possible actions. The adapted second visual overlay can be transmitted from the device to the display device.

[0079] The display device can display all possible actions in 508. In 510, possible actions can be highlighted while the mouse hovers over the display device. In 512, the user can select measurement parameters and / or geometric elements, particularly with a single click. Information regarding the selection of geometric elements and / or measurement parameters in 512 can be sent to the device. The device can perform the measurement based on this information. In 514, the method can be terminated.

[0080] Further details and embodiments are referred to in connection with the examples described above. The example shown in Figure 4 may have one or more embodiments referred to in connection with the proposed concept, or one or more optional additional features corresponding to one or more of the examples described above (e.g., Figures 1 to 4).

[0081] Some or all of the method steps may be performed by a hardware device (or by using a hardware device), such as a processor, microprocessor, programmable computer, or electronic circuit. In some embodiments, one or more of the most critical steps may be performed by such a device.

[0082] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation is feasible using a non-transient recording medium, which is a digital recording medium, etc., that stores electronically readable control signals and cooperates (or can cooperate) with a programmable computer system to carry out each method. Examples include floppy disks, DVDs, Blu-rays, CDs, ROMs, PROMs and EPROMs, EEPROMs, or FLASH memory. Thus, the digital recording medium may be computer-readable.

[0083] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system so as to carry out any of the methods described herein.

[0084] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code, which operates to perform one of the methods when the computer program product is executed on a computer. This program code may be stored, for example, on a machine-readable carrier.

[0085] Another embodiment includes a computer program stored in a machine-readable carrier for carrying out any of the methods described herein.

[0086] Therefore, in other words, embodiments of the present invention are computer programs having program code for carrying out any of the methods described herein when the computer program is executed on a computer.

[0087] Accordingly, another embodiment of the present invention is a recording medium (or data carrier or computer-readable medium) containing a stored computer program for carrying out any of the methods described herein when executed by a processor. The data carrier, digital recording medium, or recording medium is typically tangible and / or non-transient. Another embodiment of the present invention is an apparatus, such as those described herein, comprising a processor and a recording medium.

[0088] Therefore, another embodiment of the present invention is a data stream or signal sequence representing a computer program for carrying out any of the methods described herein. The data stream or signal sequence may be configured to be transmitted, for example, over a data communication connection, such as the Internet.

[0089] Another embodiment includes processing means, for example, a computer or programmable logic device configured or adapted to carry out any of the methods described herein.

[0090] Another embodiment includes a computer having an installed computer program for carrying out any of the methods described herein.

[0091] Another embodiment of the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for carrying out any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0092] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to carry out any of the methods described herein. Generally, the methods are advantageously carried out by any hardware device.

[0093] When several embodiments are described in relation to a device or system, these embodiments should also be understood as descriptions of the corresponding methods, and vice versa. For example, a block, device, or functional embodiment of a device or system may correspond to a feature of the corresponding method, such as a step. Accordingly, embodiments described in relation to a method should also be understood as descriptions of the corresponding block, corresponding element, characteristic, or functional feature of the corresponding device or system.

[0094] The following claims are incorporated into the detailed description herein, and each claim may stand on its own as a separate example. While a dependent claim in a claim refers to a specific combination with one or more other claims, it should also be noted that other examples may also include combinations of a dependent claim with the subject matter of any other dependent or independent claim. Unless it is stated that a particular combination is not intended in an individual case, such combinations are expressly proposed herein. Furthermore, features of a claim should also be included in any other independent claim, even if the claim is not directly defined as dependent on any other independent claim.

[0095] Aspects and features described in relation to a particular example of a prior example may be combined with one or more further examples to replace identical or similar features of further examples, or to introduce additional features to further examples. [Explanation of symbols]

[0096] 100 Optical Imaging Systems 110 samples 122 sensors 130 Equipment 132 Interfaces 134 processors 136 Storage Devices 200 Settings Panel 220 yen 224,226,228 icons 230 Trapezoid 232,238,242,256,264 Measurement parameters 236,234,240,254,262 Regions associated with measurement parameters 250 points 300 Systems 310 Microscope 320 Computer Systems 330 Display Devices 400 ways 410 Receive sensor data from the sensor of the optical imaging system. Generate 420 visual overlays 430 Sends a display signal indicating a visual overlay. 500 ways 502 Start Method 504 Select measurement parameters / geometry elements 506 Select / Generate Geometry Elements Show 508 possible actions 510 Highlights on mouseover 512 Select the desired measurement parameter / geometry element. 514 End method

Claims

1. An apparatus (130) for an optical imaging system, comprising one or more processors (134) and one or more storage devices (136), wherein the apparatus (130) The optical imaging system receives sensor data from the sensor of the optical imaging system, which shows a live view of the sample through the microscope of the optical imaging system. Based on the sensor data, a visual overlay is generated, which includes a visual representation of the live view and icons indicating multiple possible actions associated with the measurement. A device (130) configured to transmit a display signal indicating the aforementioned visual overlay.

2. The aforementioned device (130) further, Upon receiving a trigger signal indicating the user's selection of the aforementioned icon, Based on the sensor data and the trigger signal, a second visual overlay is generated, which includes the visual representation of the live view and the visual representations of the plurality of possible actions. The apparatus (130) according to claim 1, configured to transmit a second display signal indicating the second visual overlay.

3. The aforementioned device (130) further, The user receives a second trigger signal indicating an increased likelihood of selecting one of the multiple possible actions. Based on the second trigger signal, the second visual overlay is adapted by highlighting one of the multiple possible actions. The apparatus (130) according to claim 2, configured to transmit the adapted second display signal.

4. The apparatus (130) according to claim 3, wherein the second visual overlay is for highlighting at least one of a measurement parameter or an existing geometric element.

5. The apparatus (130) according to any one of claims 2 to 4, wherein the second visual overlay includes representations of each possible action of the plurality of actions associated with the selected icon.

6. The apparatus (130) according to any one of claims 1 to 5, wherein the plurality of possible actions include at least one of generating geometric elements or defining measurement parameters.

7. The apparatus (130) according to any one of claims 1 to 6, wherein the plurality of possible actions include different geometric elements to be generated.

8. The apparatus (130) according to any one of claims 1 to 7, wherein the plurality of actions include different measurement parameters to be defined.

9. The aforementioned device (130) further, It is configured to receive a third trigger signal indicating the user's selection of an existing geometric element, The apparatus (130) according to any one of claims 2 to 8, wherein the generation of the second visual overlay or at least one of the adapted second visual overlays is further based on the third trigger signal.

10. The apparatus (130) according to claim 9, wherein the selected existing geometric element is a primary geometric element, and the adapted second visual overlay includes highlighting of possible secondary geometric elements.

11. The aforementioned device (130) further, A fourth trigger signal is received indicating the selection of one of the multiple possible actions. The apparatus (130) according to any one of claims 1 to 10, configured to perform at least one of generating or measuring a geometric element based on the fourth trigger signal.

12. An optical system (100) comprising the apparatus (130) according to any one of claims 1 to 11.

13. A method (400) for an apparatus for an optical imaging system, The optical imaging system receives sensor data from the optical imaging system's sensors, which shows a live view of the sample through the microscope of the optical imaging system (410), Based on the sensor data, a visual overlay is generated (420) which includes a visual representation of the live view and icons indicating a plurality of possible actions associated with the measurement. A method (400) comprising transmitting a display signal indicating the visual overlay (430).

14. Receiving a trigger signal indicating the user's selection of the aforementioned icon, Based on the sensor data and the trigger signal, a second visual overlay is generated, which includes the visual representation of the live view and the visual representations of the plurality of possible actions. The method (400) of claim 13, further comprising transmitting a second display signal indicating the second visual overlay.

15. A computer program having program code for carrying out the method according to claim 13 or 14 when the program is executed on a processor.

Images