Apparatus for an optical imaging system, optical imaging system, method and computer program
The integration of a visual overlay with a live view and configuration window in optical imaging systems addresses the inefficiency of adjusting measurement parameters, allowing for real-time adjustments and improved user experience in industrial process control.
Patent Information
- Application Number
- PCT/EP2024/062368
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-28
- Filing Date
- 2024-05-04
- Publication Date
- 2024-11-07
AI Technical Summary
Current optical imaging systems in industrial settings lack an intuitive user interface for adjusting measurement parameters during process control, requiring users to switch between different views, which can be time-consuming and inefficient.
An apparatus and method that generate a visual overlay combining a live view and a configuration window, allowing users to directly view and edit measurement parameters, such as dimensional measurements, within the same interface, eliminating the need to switch between different GUIs.
This approach enhances user experience by enabling users to adjust measurement parameters in real-time while viewing the live sample, improving process control efficiency and reducing the time needed to execute tasks relevant to manufacturing processes.
Smart Images

Figure EP2024062368_07112024_PF_FP_ABST
Abstract
Description
[0001] Apparatus for an Optical Imaging System, Optical Imaging System, Method and Computer Program
[0002] Technical field
[0003] Examples relate to an optical imaging system, such as an optical imaging system used in industrial settings to monitor and control a quality of a manufacturing process or work piece, and to an optical imaging system, a method, and a computer program.
[0004] Background
[0005] Optical imaging system are often used in industrial and manufacturing settings for the purpose of monitoring and controlling the quality of processes or work pieces. The optical imaging system may feature digital imaging systems, video capabilities, and precision measurement tools, which allow users to quickly and accurately inspect samples and work pieces, such like products, during or after the manufacturing process. In addition, they often have automated analysis systems that can provide real-time feedback to production lines, further streamlining the quality process control. A software component of the measurement set is typically established through the use of specialized software that is designed to work with the microscope. However, the software component may be not sufficient to adjust needed parameters by a user. Thus, there may be a desire for an improved concept for providing a user a possibility to adjust a parameter for process control.
[0006] Summary
[0007] This desire is addressed by the subject-matter of the independent claims.
[0008] The concept proposed in the present disclosure is based on the insight, that a user experience can be increased by providing a visual overlay of a live view and a configuration window for configuring a dimension a measurement. The visual overlay may allow a user to directly view and / or edit a parameter of a measurement, such like a dimensional measurement. Therefore, a switch between different views to view either the live view or to a edit a parameter of the measurement can be omitted. Examples provide an apparatus for optical imaging system, comprising one or more processors and one or more storage devices. The apparatus is configured to receive sensor data of a sensor of the optical imaging system. The sensor data is received from the optical imaging system, e.g., from the sensor. The sensor data is indicative of a live view of a sample through a microscope of the optical imaging system. Further, the apparatus is configured to generate a visual overlay comprising a visual representation of the live view and a configuration window for configuring a measurement of the sample. The visual overlay is generated based on the sensor data. Further, the apparatus is configured to transmit a display signal indicative of the visual overlay. The display signal may be transmitted to a display device, e.g., part of the optical imaging system. The configuration window may be a graphical user interface (GUI) element that serves as a container for displaying information, content, or for providing a visual frame for interacting with the application. The configuration window may be specially to control and / or check a result of a measurement, such like a dimensional measurement, and / or to edit / modify the measurement. By generating the visual overlay a GUI can be displayed to the user, enabling the user to set and / or adjust a parameter of the measurement while viewing the live view. Thus, a user experience can be improved. For example a switch between different GUIs can be omitted. For example, the configuration window may enable a user to view data, examine result, change a parameter of the measurement relevant to the process control of a work piece and view the live view simultaneously.
[0009] In an example, the apparatus may be further configured to obtain a trigger signal indicative of a desire of a user of the optical imaging system to adjust the measurement of the sample. Further, the apparatus may be configured to generate the visual overlay based on the trigger signal. By obtaining the trigger signal the visual overlay can be generated if requested by the user. The trigger signal can be based on an input, such like a click on a GUI element displayed on the display device. For example, the optical imaging system may transmit information about a click on the GUI element to the apparatus. Thus, the apparatus can provide the display signal if the user requests / needs the visual overlay.
[0010] In an example, the configuration window may be a floating window. A floating window is a type of GUI element in software that allows a window to be detached from the main application window and positioned independently on the screen. This allows the user to move the configuration window so that it overlays the live view in a less interesting position. In an example, the configuration window may allow a configuration of a fitting algorithm, a nominal value and / or a tolerance for the measurement. Thus, a desired parameter to adjust a measurement of a process control can be displayed to the user. In this way, the user can be enabled to modify a relevant parameter of the measurement, e.g., a dimensional measurement.
[0011] In an example, the apparatus may be further configured to obtain the trigger signal from an input device of a display device. Further the apparatus may be configured to transmit the display signal to the display device associated with the input device the trigger signal is received from. Thus, by transmitting the displayed signal to the display associated with the input device it can be ensured that the visual overlay is displayed on a desired display device. For example, the input device may be a mouse, a keyboard, a touchscreen.
[0012] In an example, the apparatus may be further configured to obtain the trigger signal by determining a deviation of a measurement result from a predefined value. The predefined value may be of value defined by a manufacturing step, e.g., a diameter of a drilled hole, a length of an edge. In this way, a user can be informed about the deviation of a work piece in a facilitated way. Thus, the desire of the user can be determined based on a need of the user to adjust the measurement.
[0013] In an example, the configuration window may be further for displaying a measurement result of a measurement. The measurement can be a measurement for which parameters have been set by the user, e.g., using the visual overlay. Alternatively the measurement can be a measurement performed based on a default measurement setting. By displaying the measurement result the user can adjust measurement parameters more precisely and / or can be informed about a parameter of work piece. Thus, an adjustment of the measurement can be improved.
[0014] In an example, the configuration window may be further for configuring the displayed measurement result. In this way, a user of the optical imaging system can adjust a displayed measurement result, e.g., to improve a perceptibility.
[0015] Embodiments provide an apparatus as described herein, e.g., above or below. Embodiments provide a method for an apparatus for an optical imaging system. The method comprises receiving, from the optical imaging system, sensor data of a sensor of the optical imaging system. The sensor data is indicative of a live view of a sample through a microscope of the optical imaging system. Further, the method comprises generating, based on the sensor data, a visual overlay comprising a visual presentation of the live view and a configuration window. The configuration window is for configuring a measurement of the sample. Further, the method comprises transmitting a display signal indicative of the visual overlay.
[0016] Various examples of the present disclosure relate to a corresponding computer program with a program code for performing the above method when the computer program is executed on a processor.
[0017] Short description of the Figures
[0018] Some examples of apparatuses and / or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which
[0019] Fig. 1 shows a schematic diagram of an example of an apparatus for an optical imaging system;
[0020] Fig. 2 shows an example of a configuration window;
[0021] Fig. 3 shows a schematic illustration of a system; and
[0022] Fig. 4 shows a flow chart of an example of a method.
[0023] Detailed Description
[0024] Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and / or regions may be exaggerated for clarity.
[0025] Fig. 1 shows a schematic diagram of an example of an apparatus 130 for an optical imaging system 100. The apparatus 130 is tasked with controlling various aspects of a microscope of the optical imaging system 100, which may be an optical imaging system 100 for process control in industrial settings, and of the entire optical imaging system 100 and / or with processing various types of sensor data of the optical imaging system 100. Consequently, the apparatus 130 may be implemented as a computer system, which interfaces with the various components of the optical imaging system 100, e.g., the sensor 122.
[0026] The apparatus 130 comprises, as shown in Fig. 1, one or more processors 134 and one or more storage devices 136. Optionally, the apparatus 130 further comprises one or more interfaces 132. The one or more processors 134 are coupled to the one or more storage devices 136 and to the optional one or more interfaces 132. In general, the functionality of the apparatus 130 may be provided by the one or more processors 134 (e.g., for generating the display signal), in conjunction with the one or more interfaces 132 (for exchanging data, e.g., with the sensor 122 to receive the sensor data, the display device 330 to transmit the display signal) and / or with the one or more storage devices 136 (for storing and / or retrieving data).
[0027] The apparatus 130 is configured to receive sensor data of a sensor of the optical imaging system 100. The sensor data is received from the optical imaging system 100, e.g., from the sensor 122. The sensor data is indicative of a live view of the sample 110 through a microscope (not shown) of the optical imaging system 100. The live view may be acquired by the sensor 122. The sample 110 may be a sample under test. For example, the sample 110 may be a work piece of an industrial manufacturing process. For example, the optical imaging system 100 may be configured for a process control of the sample 110. The optical imaging system 100 may be designed for use in a laboratory or factory environment. The optical imaging system 100 can be used to inspect the sample 110, such like a product, during the manufacturing process to ensure that it meets required specifications and standards.
[0028] The optical imaging system 100 may be equipped with features such as digital imaging (e.g., the sensor 122), video capabilities (e.g., a display device 330), and precision measurement tools, which allow users, such like technicians, to identify defects, monitor changes in a manufacturing process, and / or make adjustments as necessary to ensure product quality. In addition, many process control microscopes have automated systems that can quickly analyze images and provide feedback to production lines, further streamlining the quality process control. However, there may be a need for adjusting the measurement for process control, e.g., to compensate for systematic errors caused by changes in a manufacturing process or set up a measurement for a new work piece.
[0029] Thus, the apparatus 130 is further configured to generate a visual overlay comprising a visual representation of the live view in the configuration window. The live view refers to a realtime display of the sample 110. For example, a live view may allow the user to see a continuous and up-to-date display of the sample 110 being acquired by the microscope.
[0030] A live view can be particularly useful in process control microscopy applications, as it allows the user to see an image of the sample 110 in real-time and make adjustments to the microscope or sample 110 as needed.
[0031] For example, a live view may be used to inspect and monitor the quality of a work piece during the manufacturing process, or to adjust the focus and positioning of the microscope to ensure that the live view is properly acquired.
[0032] The live view may be presented in a separate window or as a portion of a main application window. The live view window may be accompanied by the configuration window. The configuration window is for configuring a measurement of the sample 110. For example, the configuration window can be used to configure a measurement of an optical inspection such as a dimensional measurement of the sample 110.
[0033] For example, the measurement may be an optical inspection such as surface inspection, texture analysis, defect detection, dimensional measurement. The configuration window may provide a GUI element to set or adjust a parameter of the measurement. For example, the configuration window may be presented in a separate window or as a portion of the main application window (e.g., analogously to the live view). The measurement can involve the measurement of a measurement parameter such as length, width, height, diameter, radius, roughness, texture of the sample 110, e.g., a structure of the sample 110. That is, the measurement parameter may be part of a dimensional measurement. For example, the measurement parameter may be a result of the measurement, e.g., a dimensional measurement. The measurement parameter may be no value of a setting to perform the measurement, for example. For example, a setting of an illumination source to illuminate a sample for image acquisition may be no measurement parameter in the sense of this application. That is, the configuration window (see Fig. 2) for configuring a measurement of the sample may be for configuring the measurement for measuring a measurement parameter of the sample.
[0034] A dimensional measurement refers to a process of determining a size and dimension of a structure. For example, a dimensional measurements may be used to inspect and verify the size and dimensions of work pieces, e.g., the sample 110, during the manufacturing process. This is done to ensure that the work piece meets the required specifications and standards, and to detect any deviation or defect that may affect the quality or performance of the work piece. Dimensional measurements can be performed on a wide range of materials and work pieces, including metal parts, plastic components, and electronic devices.
[0035] The apparatus 130 is further configured to transmit a display signal indicative of the visual overlay, e.g., to a display device 330 or a storage device, such like a frame buffer. The optical imaging system 100 may comprise the display device 330 and / or the storage device. Alternatively, the display device 330 may be external to the optical imaging system 100, e.g., communicatively coupled to the optical imaging system 100. For example, the display device 330 may comprise the storage device. The display signal may comprise information about the visual overlay, e.g., the sensor data or may compromise the sensor data and data about a visual representation of the configuration window, such like a GUI element. In this way, the display device 330 can be used to display the live view in combination with the configuration window. Optionally, the display signal may comprise data to control the display device 330. Thus, the apparatus 130 can trigger display the visual overlay on the display device 330.
[0036] The visual overlay allows a user to directly view and edit the parameters of the measurement, such like a dimensional measurement. Therefore, a need to view and edit a parameter of the measurement in another specific area in the software can be omitted. A parameter relevant for controlling a manufacturing process can be displayed in one main window of the display device 330.
[0037] Thus, a time needed to execute a task relevant for controlling a manufacturing process can be reduced. The user does not need to jump between the live view window, where the measurement is shown and a window to display and edit a measurement result or measurement. In this way, an intuitive user interface can be provided, which may improve a user experience. Instead of jumping back and forth between the live view camera window and a window where the measurement result is displayed, the visual overlay enables the user to edit / modify the measurement result directly right where it is displayed.
[0038] The proposed concept is built around two main components - the microscope, which comprises the optical components, and which may house the display device 330 being used to view the sample 110, and the apparatus 130, which is used to control the optical imaging system 100, process sensor data of the microscope, e.g., the sensor 122, and to generate a display signal.
[0039] In general, an optical imaging system 100 comprises a microscope that is suitable for examining objects that are too small to be examined by the human eye (alone). For example, a microscope may provide an optical magnification of a sample, such as a sample 110 shown in Fig. 1. Thus, a structure of the sample 110, e.g., formed by a manufacturing step, can be magnified and measured with increased precision. In modern microscopes, the optical magnification is often provided for a camera or an imaging sensor, such as the optical imaging sensors 122 of the microscope. The microscope may further comprise one or more optical magnification components that are used to magnify a view of the sample 110, such as an objective.
[0040] There are a variety of different types of optical imaging systems. For example, the optical imaging system 100 may be used for process control. The optical imaging system 100 may comprise an optical system, a camera or detector and a monitor or a display device 330. The optical system may typically comprise optical parts, such like lenses, mirrors, or other optical components that work together to produce a high-resolution image, e.g., the live view, of the sample 110. The camera or detector, e.g., the sensor 122, may be used to acquire the live view, e.g., the live view, of the sample 110 and converts it into a digital signal that can be displayed on the display device 330 and / or analyzed using a software, e.g., performed by the apparatus 130. The display device 330 may be used to display the live view acquired by the sensor 122, as well as any additional information such as measurement data or analyzes methods, e.g., displayed in the configuration window.
[0041] In an example, the apparatus 130 may be further configured to obtain a trigger signal indicative of a desire of a user of the optical imaging system 100 to adjust the measurement of a sample 110. For example, the live view may be displayed in a live view camera window. The user could click or select an icon, such like a measurement output label, or an area in the live view camera window to trigger an opening of the configuration window. Information about the click or the selection of the user can be transmitted to the apparatus 130, e.g., from the display device 330 or the optical imaging system 100. The information about the click or the selection can be obtained by receiving at the apparatus 130 the trigger signal.
[0042] Thus, the apparatus 130 can generate the visual overlay based on the trigger signal, such that the visual overlay may be only generated if triggered by the user. In this case, the generation of the visual overlay can be adapted to an actual use case, which may reduce computing resources needed. For example, the user can trigger with the click or the selection a generation of the visual overlay and thus an opening of the configuration window. In this way, the user experience can be improved. For example, if the user wants to view, edit, modify and / or delete a parameter (like lower limit, upper limit, nominal value, tolerance and / or fitting algorithm) of the measurement, the user can trigger the opening of the configuration window easily.
[0043] Additionally or alternatively, the trigger signal can be obtained by determining a trigger event. In an example, the apparatus 130 may be further configured to obtain the trigger signal by determining a deviation of a measurement result from a predefined value. For example, the apparatus 130 can determine a deviation of a measurement result from a predefined value. The predefined value may be a parameter of the sample 110, e.g., of a structure formed by a manufacturing step. For example, the sample 110 may comprise a hole with the predefined diameter or an edge with a predefined length. The predefined value may be defined by a manufacturing step of the manufacturing process of the sample 110. Thus, the user can be informed about a deviation of the sample 110 from a predefined value, e.g., an erroneous manufacturing process.
[0044] The structure of the sample 110 may be measured during process control. If a deviation of the measurement result exceeds a threshold, e.g., a measured value of a diameter of structure is more than 1% larger as the predefined value, the apparatus 130 can generate the visual overlay. In this way, the user can be informed about the deviation and can take certain measures to counteract or to adjust the measurement. In an example, the configuration window may be a floating window. A floating window may have its own separate title bar and can be moved, resized, and / or minimized independently of the main application window, e.g., the live view window. The floating window may be used to provide a separate display area for measurement tools (e.g., a parameter of the measurement, such like used fitting algorithm, upper limit, lower limit) and other information (e.g., a measurement result) part of the configuration window. For example, the floating window may provide access to measurement tools, image analysis functions (fit or draw of geometry elements), and other features, while the main application window may display a live video feed, e.g., the live view, from the microscope acquiring the sample 110.
[0045] In an example, the configuration window may allow the configuration of a fitting algorithm, a nominal value and / or a tolerance for the measurement. For example, the configuration of the nominal value can comprise a configuration of a lower limit and / or an upper limit. For example, a tolerance may be a lower tolerance and / or an upper tolerance in % or in measurement unit. The tolerance can be linked / unlinked to easily enter a / an symmetrical / asymmetrical tolerance. For example, a fitting algorithm may be polynomial regression, nonlinear regression, least square fitting. The fitting algorithm may be used to analyze the dimensions of a work piece or a structure of the workpiece (e.g., a hole an edge), such as its size, shape, or surface profile. The fitting algorithm can be used to identify trends or patterns in the manufacturing process, or to detect deviations or defects in the work piece that may affect its quality or performance.
[0046] In an example, the configuration window may provide an operator a possibility to mark a measurement, e.g., mark a measurement as passed or failed. The configuration window may comprise, e.g., a check box, an icon to mark a measurement. For example, if no nominal value is set, the apparatus cannot automatically determine whether a measurement is passed or failed. Thus, the operator can mark the measurement as pass of fail. The apparatus 130 may receive a status trigger signal indicative of the status of the measurement selected by the operator. The apparatus 130 may store the measurement as assessed by the operator. In this way, the measurement can be identified for post-controls. Thus, a process control can be improved.
[0047] In an example, the apparatus 130 may be further configured to obtain the trigger signal from an input device of a display device 330. Further, the apparatus 130 may be configured to transmit the display signal to the display device 330 associated with the input device the trigger signal is received from. In this way, it can be ensured that the visual overlay is displayed on a display device 330 actually used by the user.
[0048] In an example, the configuration window may be further for displaying a measurement result of a measurement. By displaying the measurement result in the configuration window, the user can easily access relevant information about the measurement of the sample 110. Thus, the configuration window can be used to display information related to the measurement of the sample 110 using the display device 330. In this way, the user can be informed about relevant information associated with the measurement and still can view the live view of the sample 110.
[0049] In an example, the configuration window may be further for configuring the displayed measurement result. Thus, the user can adjust the displayed measurement result. For example, the user may adjust a color, a thickness, a unit, a background, an opacity to display the measurement result. In this way, the measurement result can be adapted to a need or desire of the user. For example, if the measurement result deviates from a predefined value by a certain threshold, the measurement result can be highlighted. Thus, an awareness of the user on the sample 110 not fulfilling a production requirement can be increased.
[0050] As shown in Fig. 1 the optional one or more interfaces 132 is coupled to the respective one or more processors 134 at the apparatus 130. In examples the one or more processors 134 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. Similar, the described functions of the one or more processors 134 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. The one or more processors 134 is capable of controlling the one or more interfaces 132, so that any data transfer that occurs over the one or more interfaces 132 and / or any interaction in which the one or more interfaces 132 may be involved may be controlled by the one or more processors 134. In an embodiment the apparatus 130 may comprise a memory, e.g., the one or more storage devices 136 and at least one or more processors 134 operably coupled to the memory and configured to perform the method described below.
[0051] In examples the one or more interfaces 132 may correspond to any means for obtaining, receiving, transmitting or providing analog or digital signals or information, e.g., any connector, contact, pin, register, input port, output port, conductor, lane, etc. which allows providing or obtaining a signal or information. The one or more interfaces 132 may be wireless or wireline and it may be configured to communicate, e.g., transmit or receive signals, information with further internal or external components.
[0052] The apparatus 130 may be a computer, processor, control unit, (field) programmable logic array ((F)PLA), (field) programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASICs), integrated circuits (IC) or system-on-a-chip (SoCs) system. The apparatus 130 may be part of the optical imaging system 100. Alternatively, the apparatus 130 may be external to optical imaging system 100, e.g., may be part of a display device 330.
[0053] More details and aspects are mentioned in connection with the examples described below. The example shown in Fig. 1 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described below (e.g., Fig. 2 - 4).
[0054] Fig. 2 shows an example of a configuration window 200. The configuration window 200 may display a parameter relevant for a measurement, e.g., a dimensional measurement. The configuration window 200 may enable a user to adjust a parameter of the measurement. For example, the configuration window 200 may provide a user interface, such as text box 220, 222, 224, 226, 228, a drop-down menu 230, 232, 234, a checkbox, an icon 240. As shown in Fig. 2 text boxes 220, 222, 224, 226, 228 may be provided for adjusting a lower limit 220, a nominal value 222, an upper limit 224, a lower tolerance 226, an upper tolerance 228, for example. Also a drop-down menu for adjusting a lower / upper tolerance value 230 / 232 or for selecting a fitting algorithm 234 may be provided. Further, the configuration window 200 may display the measurement result, e.g., using a slider 250 to increase a perceptibility of the measurement result. For example, as shown in Fig. 2 the measurement result may be displayed, e.g., in green, above the slider 250. Further, the slider 250 may indicate that the measurement value is within the tolerances. Further, an additional indication when the measurement result is within the tolerances can be displayed in a text display area 260, e.g., displaying green text.
[0055] Thus, the configuration window 200 may provide user relevant information about the measurement of the sample. In this way, the user can easily perform a process control, e.g., change the measurement result and / or adjust the measurement.
[0056] More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 2 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1) and / or below (e.g., Fig. 3 - 4).
[0057] Some embodiments relate to an optical imaging system comprising an apparatus as described in connection with Fig. 1. Alternatively, an optical imaging system may be part of or connected to an apparatus as described in connection with Fig. 1.
[0058] Fig. 3 shows a schematic illustration of a system 300, e.g., an optical imaging system 300. The optical imaging system 300 may comprise an apparatus as described with reference to Fig. 1 and a display device 330. The display device 330 may be part of a computer system 320. For example, the microscope 310 may comprise or can be communicatively coupled to the apparatus. Alternatively, the computer system 320 may comprise the apparatus. The microscope 310 can be communicatively coupled to the display device 330. Thus, the apparatus can transmit the display signal from the microscope 310 to the display device 330 or a storage device, such like a frame buffer, for example.
[0059] Fig. 3 shows a schematic illustration of a system 300 configured to perform a method described herein, e.g., with reference to Figs. 4. The system 300 comprises a microscope 310 and a computer system 320. The microscope may comprise the apparatus as described above, e.g., with reference to Fig 1. The microscope 310 is configured to take images and is connected to the computer system 320. The computer system 320 is configured to execute at least a part of a method described herein. The computer system 320 may be configured to execute a machine learning algorithm. The computer system 320 and microscope 310 may be separate entities but can also be integrated together in one common housing. The computer system 320 may be part of a central processing system of the microscope 310 and / or the computer system 320 may be part of a subcomponent of the microscope 310, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 310.
[0060] The computer system 320 may be a local computer device (e.g., personal computer, laptop, tablet computer or mobile phone) with one or more processors and one or more storage devices or may be a distributed computer system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example, at a local client and / or one or more remote server farms and / or data centers). The computer system 320 may comprise any circuit or combination of circuits. In one embodiment, the computer system 320 may include one or more processors which can be of any type. As used herein, processor may mean any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), for example, of a microscope or a microscope component (e.g., camera) or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 320 may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 320 may include one or more storage devices, which may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like. The computer system 320 may also include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, trackball, touch screen, voice-recognition device, or any other device that permits a system user to input information into and receive information from the computer system 320. More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 3 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 2) and / or below (e.g., Fig. 4).
[0061] Fig. 4 show a flow chart of an example of a method 400. The method 400 for an optical imaging system comprises receiving 410, from the optical imaging system, sensor data of a sensor of the optical imaging system. The sensor data is indicative of a live view of a sample through a microscope of the optical imaging system. Further, the method 400 comprises generating 420, based on the sensor data, a visual overlay comprising a visual presentation of the live view and a configuration window. The configuration window is for configuring a measurement of the sample. Further, the method 400 comprises transmitting 430 a display signal indicative of the visual overlay. The display signal may be transmitted to a display device, e.g., part of the optical imaging system or a storage device, such like a frame buffer. The method 400 may be performed by an apparatus as described with reference to Fig. 1.
[0062] More details and aspects are mentioned in connection with the examples described above. The example shown in Fig. 4 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 3).
[0063] Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
[0064] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non- transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable. Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
[0065] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.
[0066] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
[0067] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0068] A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.
[0069] A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.
[0070] A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
[0071] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein. A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
[0072] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.
[0073] If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method and vice versa. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.
[0074] The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.
[0075] The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example. List of reference Signs optical imaging system sample sensor apparatus interface processor storage device configuration window , 222, 2224, 226, 228 text box , 232, 234 drop-down menu system microscope computer system display device method receiving sensor data of a sensor of the optical imaging system generating a visual overlay transmitting a display signal indicative of the visual overlay
Claims
Claims1. An apparatus (130) for an optical imaging system (100), comprising one or more processors (134) and one or more storage devices (136), wherein the apparatus (130) is configured to: receive, from the optical imaging system (100), sensor data of a sensor (122) of the optical imaging system (100), the sensor data indicative of a live view of a sample through a microscope of the optical imaging system (100); generate, based on the sensor data, a visual overlay comprising a visual representation of the live view and a configuration window (200) for configuring a measurement of the sample; and transmit a display signal indicative of the visual overlay.
2. The apparatus (130) according to claim 1, wherein the apparatus is further configured to: obtain a trigger signal indicative of a desire of a user of the optical imaging system (100) to adjust the measurement of a sample; and generate the visual overlay based on the trigger signal.
3. The apparatus (130) according to any one of the preceding claims, wherein the configuration window (200) is a floating window.
4. The apparatus (130) according to any one of the preceding claims, wherein the configuration window (200) allows a configuration of at least one of a fitting algorithm, a nominal value or a tolerance for the measurement.
5. The apparatus (130) according to any one of the claims 2 - 4, wherein the apparatus is further configured to: obtain the trigger signal from an input device of a display device (330); and transmit the display signal to the display device (330) associated with the input device the trigger signal is received from.
6. The apparatus (130) according to any one of the claims 2 - 5, wherein the apparatus is further configured to: obtain the trigger signal by determining a deviation of a measurement result from a predefined value.
7. The apparatus (130) according to any one of the preceding claims, wherein the configuration window (200) is further for displaying a measurement result of a measurement.
8. The apparatus (130) according to claim 7, wherein the configuration window (200) is further for configuring the displayed measurement result.
9. An optical imaging system (100), comprising an apparatus (130) according to any one of the preceding claims.
10. A method (400) for an apparatus for an optical imaging system, comprising: receiving ( 10), from the optical imaging system, sensor data of a sensor of the optical imaging system, the sensor data indicative of a live view of a sample through a microscope of the optical imaging system; generating (420), based on the sensor data, a visual overlay comprising a visual representation of the live view and a configuration window for configuring a measurement of the sample; and transmitting (430) a display signal indicative of the visual overlay.
11. The method (400) according to claim 10, further comprising: obtain a trigger signal indicative of a desire of a user of the optical imaging system to adjust the measurement of a sample; and generating the visual overlay based on the trigger signal.
12. A computer program having a program code for performing a method according to claim 10 or 11 when the program is executed on processor.
Citation Information
Patent Citations
Digital microscope imaging system and method
EP3318836A1
Dimension Measuring Apparatus, Dimension Measuring Method, And Program For Dimension Measuring Apparatus
US20120194673A1
Information acquisition apparatus
US20180213148A1
Image measurement method, image measurement program, image measurement device, and object manufacture method
US20190139247A1
Microscope system and corresponding system, method and computer program
WO2022194965A1
