Graphic overlay for sample camera in x-ray microscope

Abstract

A user interface rendered on a display of a microscopy system comprises sample motion controls for moving the object stage subsystem and an image region in which the image data from the camera is displayed along with a graphic overlay indicating different directions of movement provided by the sample motion controls. This overlay preferably provides a dynamic coordinate system graphic highlighting the coordinate axes of 3-axis stage of the object stage subsystem. The graphic can provide additional information such as a direction of motion as the user hovers over different motion controllers, to help the user understand the direction of motion before they click to apply it.

Description

GRAPHIC OVERLAY FOR SAMPLE CAMERA IN X-RAY MICROSCOPERELATED APPLICATIONS

[0001] This application claims the benefit under 35 USC 119(e) of U.S. Provisional Application No. 63 / 487,067, filed on February 27, 2023, which is incorporated herein by reference in its entirety.BACKGROUND OF THE INVENTION

[0002] X-ray microscopy (XRM) is a powerful imaging technique for analyzing internal structure on the micro to nano scale. XRM systems provide high resolution images or projections of samples, allowing for detailed study of their properties. XRM systems use a beam of x-rays to illuminate the samples, which is then imaged using a detector.

[0003] X-ray computed tomography (CT) is a non-destructive technique for inspecting and analyzing internal structures of samples. Tomographic volume data sets are reconstructed from a series of these projections via standard CT reconstruction algorithms, as the samples are scanned at different angles.

[0004] There are a number of different configurations for x-ray CT systems . In x-ray microscopy systems, because the x-ray sources and detectors are large and the samples or objects being scanned are typically small, the x-ray sources and detectors are largely fixed, while the samples are rotated in the x-ray beam.

[0005] In many cases, the samples scanned in X-ray microscopy systems have a priori unknown shapes. Even in the cases where a CAD model is available or the object is from a dimensionally-known core-sample, for example, the exact alignment of the sample is often unknown. Moreover, the alignment may be changed when different regions of interest are selected and the object is realigned in the beam path. It is also often necessary to move the sample completely out of the beam path, for example in order to obtain baseline performance information concerning the X-ray source and the detector subsystem or for taking reference images. This leads to the problem that while the sample is moved to be scanned (mostly rotated) or moved out of the beam path, loaded or unloaded, the sample might collide with the scanning setup (the parts of the X-ray-source or detector that are most proximate to the sample). The challenge of avoiding collisions is often made more difficult by the fact that the X-ray source and / or the detector will need to be moved into close proximity to the sample for optimal system performance.

[0006] Similar setups also exist in other microscopy / tomography systems operating in other regions of the electromagnetic spectrum such as optical coherence tomography and confocal microscopy (optical projection tomography). Still other examples include scanning electron microscopes (SEMs) and focused ion beam (FIB) systems - i.e. charged particle imaging systems.SUMMARY OF THE INVENTION

[0007] In the typical setup, an object stage subsystem is provided that can position and rotate the sample in the system's beam path. Often, the object stage subsystem has a 3-axis stage including an x-axis stage, a y-axis stage, and a z-axis stage. This allows a region of interest of the sample to be located within the beam path. This 3-axis stage in turn is mounted on a theta stage that rotates the sample in the beam path. This setup allows the sample to be rotated in the system's frame of reference so that the sample can be scanned at different angles to allow tomographic reconstruction.

[0008] The fact that the 3 -axis stage is rotated by the theta stage makes manual control of the 3-axis stage challenging. The directions of the x-axis stage and the z-axis stage is a function of the angle of the theta stage. Said another way, the coordinate axis or frame of reference of the 3-axis stage changes relative to the system's frame of reference as a function of the angular position of the theta stage. Thus, it is often difficult for the user to intuitively understand the relationship between the system's and user's reference frames and the coordinate axis of the 3-axis stage.

[0009] This invention concerns the ability to overlay a dynamic coordinate system graphic onto the live image of the sample. The graphic highlights the coordinate axis of the 3-axis stage. Moreover, the graphic can provide additional information such as a direction of motion as the user hovers over different motion controllers, to help the user understand the direction of motion before they click to apply it.

[0010] In general, according to one aspect, the invention features a user interface rendered on a display of a microscopy system. This system includes a sample camera for capturing images of a sample loaded into object stage subsystem of the microscopy system and a computer that processes image data from the camera. The user interface comprises sample motion controls for moving the object stage subsystem and an image region inwhich the image data from the camera is displayed along with a graphic overlay indicating different directions of movement provided by the sample motion controls.

[0011] In a current example, the sample motion controls include controls for a 3 -axis stage of the object stage subsystem and the graphic overlay includes directions for the 3- axis stage.

[0012] In addition, the sample motion controls include controls for a theta stage of the object stage subsystem and the graphic overlay includes directions for the theta stage.

[0013] Preferably, the graphic overlay is updated when a user hovers over different ones of the sample motion controls.

[0014] In general, according to one aspect, the invention features an X-ray microscopy system. It comprises an X-ray source subsystem for generating X-rays and an object stage subsystem for holding a sample in the X-rays. A detector subsystem detects the X-rays after interaction with the sample and a sample camera captures images or projections of a sample loaded into the object stage subsystem. A computer receives and processes projections from the detector subsystem and generates a user interface. The generated interface includes sample motion controls for moving the object stage subsystem and an image region in which the image data from the camera is displayed along with a graphic overlay indicating different directions of movement provided by the sample motion controls.

[0015] The above and other features of the invention including various novel details of construction and combinations of parts, and other advantages, will now be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular method and device embodying the invention are shown by way of illustration and not as a limitation of the invention. The principles and features of this invention may be employed in various and numerous embodiments without departing from the scope of the invention.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In the accompanying drawings, reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale; emphasis has instead been placed upon illustrating the principles of the invention. Of the drawings:

[0017] Fig. 1 is a schematic diagram of an x-ray microscopy system to which the present invention is applied in one embodiment;

[0018] Figs. 2, 3, 4, 5, 6A, 6B, and 6C show a user interface generated by the system.DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The invention now will be described more fully hereinafter with reference to the accompanying drawings, in which illustrative embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0020] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Further, the singular forms and the articles "a", "an" and "the" are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms: includes, comprises, including and / or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Further, it will be understood that when an element, including component or subsystem, is referred to and / or shown as being connected or coupled to another element, it can be directly connected or coupled to the other element or intervening elements may be present.

[0021] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0022] Fig. 1 is a schematic diagram of a XRM system 200 to which the present invention is applicable.

[0023] The illustrated microscopy system 200 is an X-ray CT system and generally includes several subsystems. An X-ray source subsystem 102 generates a polychromatic orpossibly monochromatic X-ray beam 103. An object stage subsystem 110 with object holder 112 holds a sample or object 114 in the beam and positions and repositions it to enable scanning of the sample 114 in the stationary beam 103, 105. A detector subsystem 118 detects the beam 105 after it has been modulated by the sample. A base, such as a platform or optics table 107, provides a stable foundation for the microscopy system 200 and its subsystems.

[0024] In general, the obj ect stage subsystem 110 has the ability to position and rotate the sample 114 in the beam 103. Thus, the object stage subsystem 110 will typically include linear and rotation stages. The illustrated example has a precision 3-axis stage 150 that translates and positions the sample along the x, y, and z axes, very precisely but only over relatively small ranges of travel. This allows a region of interest of the object 114 to be located within the beam 103 / 105. The 3-axis stage 150 is mounted on a theta stage 152 that rotates the 3-axis stage 150 and thus sample 114 in the beam around the y-axis. The theta stage 152 is in turn mounted on the base 107.

[0025] Thus, the flume of reference or coordinate system of the 3-axis stage 150 is related to the flame of reference or coordinate system 10 of the microscopy system 200 by the angular position of the theta stage 152.

[0026] The source subsystem 102 will typically be either a synchrotron x-ray radiation source or alternatively a “laboratory x-ray source” in some embodiments.

[0027] As used herein, a “laboratory x-ray source” is any suitable source of x-rays that is not a synchrotron x-ray radiation source. Laboratory x-ray source 102 can be an X-ray tube, in which electrons are accelerated in a vacuum by an electric field and shot into a target piece of metal, with x-rays being emitted as the electrons decelerate in the metal. Typically, such sources produce a continuous spectrum of background x-rays combined with sharp peaks in intensity at certain energies that derive from the characteristic lines of the selected target, depending on the type of metal target used.

[0028] In one example, source subsystem 102 is a rotating anode type or microfocused source, with a Tungsten target. Targets that include Molybdenum, Gold, Platinum, Silver or Copper also can be employed. Preferably a transmission target configuration is used in which the electron beam strikes the thin target from its backside. The x-rays emitted from the other side of the target are used as the beam 103.

[0029] The x-ray beam generated by source subsystem 102 is often conditioned to suppress unwanted energies or wavelengths of radiation. For example, undesired wavelengths present in the beam are eliminated or attenuated, using, for instance, energy filters (designed to select a desired x-ray energy range (bandwidth)) held in a filter wheel 160. These energy filters typically include an 'air' filter corresponding to no filter along with a set of low energy filters for filtering lower energy x-rays and high energy filters for filtering higher energy x-rays.

[0030] When the object 114 is exposed to the X-ray beam 103, the X-ray photons or particles, which propagate through the sample 114, form a modulated beam 105 that is received by the detector subsystem 118. In some other examples, an objective lens is used to form an image onto the detector subsystem 118 of the microscopy system 200.

[0003] ] Typically, a magnified projection image of the object 114 is formed on the detector subsystem 118. The magnification of the x-ray stage is equal to the inverse ratio of the source-to-object distance 202 and the source-to-detector distance 204.

[0032] To achieve high resolution, an embodiment of the x-ray CT system 200 further utilizes a very high resolution detector 124-1 of the detector subsystem 118 in conjunction with positioning the sample 114 close to the x-ray source system 102. In one implementation of the high-resolution detector 124-1, a scintillator is used in conjunction with a microscope objective to provide additional optical magnification in a range between 2x and lOOx, or more. The scintillator converts the x-rays into an optical image that can be detected by a camera.

[0033] Other detectors are often included as part of the detector subsystem 118. For example, the detector subsystem 118 can include a lower resolution detector 124-2. This could be a flat panel detector and camera or a detector with a lower magnification microscope objective, in examples. Configurations of one, two, or even more detectors 124 of the detector subsystem 118 are possible.

[0034] Preferably, two or more detectors 124-1, 124-2 are mounted on a turret 122 of the detector subsystem 118, so that they can be alternately rotated into the path of the modulated beam 105 from the sample 114.

[0035] Typically, the source subsystem 102 and the detector subsystem 118 are mounted on respective z-axis stages. For example, in the illustrated example, the sourcesubsystem 102 is mounted to the base 107 via a source stage 154, and the detector subsystem 118 is mounted to the base 107 via a detector stage 156. In practice, the source stage 154 and the detector stage 156 are lower precision, high travel-range stages that allow the source subsystem 102 and the detector subsystem 118 to be moved into position, often very close to the object during scanning and then be retracted to allow the object to be removed from, a new object to be loaded onto, and / or the object to be repositioned on the object holder 112 of the object stage subsystem 110.

[0036] The present microscopy system 200 has an optical camera 210 such as a video camera that collects image data of the sample 114 held in the object holder 112. This camera is typically mounted directly or indirectly to the system base 107 via a mounting system 215, such as a bracket. Typically, optical camera 210 collects the images in the visible portion of the spectrum and / or in the adjacent spectral regions such as the infrared. Usually, the optical camera 210 has a CCD or CMOS image sensor. Also included is a light source 212 that illuminates the object in the spectral regions employed by the optical camera.

[0037] The operation of the microscopy system 200 and the scanning of the obj ect 114 is controlled by a computer subsystem 224 that often includes an image processor 220 and a controller 222.

[0038] The computer system 224 includes one or more processors 260 along with their data storage resources such as disc or solid-state drives, and memory MEM. The processors 260 execute an operating system 262 and various applications run on that operating system 262 to allow for user control and operation of the microscopy system 200. Particularly, a user interface application 250 executes on the operating system 262 and generates a user interface that is rendered on a display device 236 connected to the computer subsystem 224. The user interface enables the operator to control the system and view projection, images and tomographic reconstructions. User input device(s) 235 such as a touch screen, computer mouse, and / or keyboard enable interaction between the operator and the computer subsystem 124. A graphic overlay app 252 adds graphical information to the image data from the camera 210 for display on the display device 236.

[0039] The controller 222 allows the computer subsystem 224 to control and manage components in the X-ray CT microscope 200 under software control. The controller might be a separate computer system adapted to handle realtime operations or an applicationprogram executing on the processor 260. The source subsystem 102 includes a control interface 130 allowing for its control and monitoring by the controller 222. Similarly, the object stage subsystem 110 and the detector subsystem 118 have respective control interfaces 132, 134 for allowing for their control and monitoring by the computer subsystem 224 via the controller 222.

[0040] To configure the microscopy system 200 to scan the sample and to adjust other parameters such as the geometrical magnification, the operator utilizes the user interface rendered on the display device 236 and generated by the user interface application 250 to adjust the source-to-object distance 202 and the source-to-detector distance 204 by respective operation of the source stage 154 and detector stage 156 to achieve the desired scanning setup.

[0004] ] Specifically, the source stage 154 and detector stage 156 include respective motor encoder systems or other actuator systems that allow the computer system 224 via the controller 222 to position the respective x-ray source subsystem 102 and the detector subsystem 118 to specified positions via the control interfaces 130, 134. Further, the source stage 154 and detector stage 156 signal the controller 222 of their actual positions.

[0042] The operator of the system under automatic control operates the object stage subsystem 110 to perform the CT scan via computer subsystem, the controller 222 and the control interfaces 130, 132, 134. Typically, the object stage subsystem 110 will position the object by rotating the object about an axis that is orthogonal to the optical axis of the x- ray beam 103, 105 by controlling the theta stage 152 and / or position the sample in the x, y, z axes directions using stage 150.

[0043] Using the user interface rendered on the display device 236 by the user interface app 250, the operator defines / selects the scanning set up including the acquisition parameters via the UI devices 235. These acquisition parameters include x-ray source voltage settings that help to determine the X-ray energy spectrum and exposure time and number of frames on the X-ray source subsystem 102. The operator also typically selects other settings such as the field of view of the X-ray beam 103 incident upon the sample 114, the number of X-ray projection images to create for the sample 114, and the detector 124-1, 124-2 selected. Generally, the acquisition parameters include X-ray source voltage, X-ray source filtration, camera exposure time, number of frames, and overall number of projections and the scanning setup includes the angles to rotate the sample by the stagesubsystem 110. In addition, the source-to-object distance 202 and the source-to-detector distance 204 are often specified and these are converted to the necessary positions or settings for the source stage 154 and detector stage 156 as part of the scanning setup.

[0044] Operation:

[0045] Fig. 2 shows the user interface 500 generated by the user interface app 250 executing on the operating system 262 of the computer system 224 and rendered on the display device 236.

[0046] In the illustrated mode, the user interface 500 includes an optical camera pane318. This displays the current image data received from the optical camera 210 along with any graphic overlays generated by the graphic overlay app 252.

[0047] To the left of the optical camera pane 318 are the source Z stage control area 338. The functions include a step size indicator indicating the steps that the source stage 154 will move in response to each user input. It includes a current position display. Also included is a user data entry line along with a "Go" button that allows the user to enter a desired absolute position for the source stage 154.

[0048] To the right of the optical camera pane 318 is a similar detector control area 340 for the detector stage 156 providing Z-axis control functions. Here again, it includes a step size indicator indicating the steps that the source stage will move. Also included is a current position display. Finally, the user can enter a desired absolute position.

[0049] Sample motion controls are located at the bottom of the window. A sample x- position control area 330 enables the movement of the object holder 112 and thus the sample or object 114 along the x-axis by control of the x-axis stage of the 3-axis stage 150, a sample y-position control area 332 enables the movement along the y-axis by control of the y-axis stage of the 3-axis stage 150, a sample z-position control area 334 enables the movement along the z-axis by control of the z-axis stage of the 3-axis stage 150, sample theta control area 336 enables the rotation of the object holder 112, 3-axis stage 150, and thus the sample or object 114 by control of the theta stage 152.

[0050] Each of the control areas 330, 332, 334, and 336 include separate step size indicators 392. Here, the user can enter the desired step size using the user interface devices. Also included are movement controls 394, back and forward, that allow the decrease or increase of the associated stage. These further include a pause button that willarrest the movement of the corresponding stage. The current position of the corresponding stage is indicated by an absolute location indicator 398. Finally, the user can move to a desired absolute position by entering in the desired position in a data entry line 396 and then selecting the associated "Go” button 395 using the user interface devices 235.

[0005] ] In addition, the user interface includes a coordinate overlay 380 that is a graphical set of indicators that are overlaid on top of the image data from camera 210 and rendered in the optical camera pane 318 by the graphic overlay app 252. This overlay indicates the relative direction within the frame of reference of the system 200 and user for the respective axes of the 3 -axis stage 150. This is important information to the since each of the Z-axis and the X-axis of the 3-axis stage 150 are dependent on the current angular position of the theta stage 152.

[0052] The coordinate overlay 380 is usefill because the operator cannot generally look at the sample 114 in the image data from the optical camera pane 210 and know in which direction the X axis and Z axis of the 3-axis stage 150 are pointed unless they also are aware of the current angle of the theta stage and mentally apply the transformation between the coordinate system of the system to the coordinate system of the 3-axis stage 150. Now, this information is provided by the coordinate overlay 380.

[0053] In more detail, for the Z axis of the 3-axis stage 150, the coordinate overlay 380 includes a negative Z axis indicator 382 and a positive Z axis indicator 383. In a similar vein, the coordinate overlay 380 further includes a negative X axis indicator 384 and a positive X axis indicator 385. Finally, the coordinate overlay 380 further includes a negative Y axis indicator 386 and a positive Y axis indicator 387. It should be noted, however, that the direction of the Y axis is not dependent on the angle of the theta stage. Finally, there is a theta indicator 388 that indicates the two directions of rotation for the theta stage 152.

[0054] Further, in the preferred embodiment, the coordinate overlay 380 is dynamic. In the illustrated example, the user has used the user interface devices 235 to manipulate the mouse pointer 390 so that it is in the area of the Y axis controls 332. As a result, the corresponding axis indicators in the coordinate overlay 380 are indicated by the graphic overlay app 252 updating the overlay 380 based on mouse position information generated by the user interface app 250. hi the current example, the mouse pointer 390 is in the Y- axis stage area 332. As a result, the negative Y axis indicator 386 and the positive Y axisindicator 387 changes color or are otherwise indicated such as with size changes, blinking or other common approaches.

[0055] To give another example, if the user were to move the mouse pointer 390 such that it was in the Z axis control area 334, the graphic overlay app 252 would update the overlay 380 based on mouse position information generated by the user interface app 250, specifically the negative Z axis indicator 382 and the positive Z axis indicator 383 would change color or otherwise be indicated. At the same time, the Y axis indicators 386 and 387 would return to their normal color or state.

[0056] The coordinate overlay 380 is further dynamic to indicate the direction of movement that the user is currently commanding. This functionality is illustrated in Fig 3. In this example, the mouse pointer 390 is hovering over the negative direction arrow in the movement controls 394 of the X axis control area 330. As a consequence, the coordinate overlay 380 is modified graphic overlay app 252 such that the negative X axis indicator 384 is now indicated by a different color. In a similar vein, when the user manipulates the UI device 235 to move the mouse pointer over any of the arrow controls of the control areas 330, 332, 334, and 336, the corresponding arrow or arrows of the coordinate overlay 380 is or are graphically indicated or highlighted.

[0057] Figs. 4 and 5 further show how the coordinate overlay 380 is updated by graphic overlay app 252 in dependence upon the state and angle of the theta stage 152. Specifically, as shown in Fig 4, with the theta stage at an angle of- 90 degrees, the negative X axis extends to the left and the positive X axis as extends to the right. In contrast, the negative Z direction is in the aft direction and the positive Z direction is in the fore direction within the frame of reference of the system 200 and optical camera 210.

[0058] In turn, Fig. 5 shows the coordinate overlay 380 when the theta stage 152 is set to - 45 degrees.

[0059] Figs. 6A, 6B, 6C show more detailed views of a portion of the theta control 336. Specifically, it includes graphic representation 350 of the sample holder 112. In detail, this graphic representation 350 rotates with the rotation of the theta stage.Specifically, Figs. 6A, 6B, 6C show the graphic representation 350 corresponding to the theta stage angles of Figs. 2, 3, 5, respectively.

[0060] While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.

Claims

CLAIMSWhat is claimed is:

1. A user interface rendered on a display of a microscopy system including a sample camera for capturing images of a sample loaded into and object stage subsystem of the microscopy system and a computer that processes image data from the camera, the user interface comprising: sample motion controls for moving the object stage subsystem; and an image region in which the image data from the camera is displayed along with a graphic overlay indicating different directions of movement provided by the sample motion controls.

2. The user interface as claimed in claim 1, wherein the sample motion controls include controls for a 3-axis stage of the object stage subsystem and the graphic overlay includes directions for the 3-axis stage.

3. The user interface as claimed in either of claims 1 or 2, wherein the sample motion controls include controls for a theta stage of the object stage subsystem and the graphic overlay includes directions for the theta stage.

4. The user interface as claimed in any of claims 1-3, wherein the graphic overlay is updated when a user hovers over different ones of the sample motion controls.

5. The user interface as claimed in any of claims 1-4, wherein the graphic overlay includes indicators for three axes of a 3-axis stage of the object stage subsystem.

6. The user interface as claimed in any of claims 1-5, wherein the graphic overlay includes indicators for a theta stage of the object stage subsystem.

7. The user interface as claimed in any of claims 1-6, wherein the graphic overlay includes a graphic representation of a sample holder.

8. The user interface as claimed in claim 7, wherein the graphic representation rotates with a rotation of a theta stage.

9. An X-ray microscopy system, comprising: an X-ray source subsystem of generating X-rays;an object stage subsystem for holding a sample in the X-rays; a detector subsystem for detecting the X-rays after interaction with the sample; a sample camera for capturing images of a sample loaded into the object stage subsystem; and a computer for receiving projections from the detector subsystem and generating a user interface including sample motion controls for moving the object stage subsystem and an image region in which the image data from the camera is displayed along with a graphic overlay indicating different directions of movement provided by the sample motion controls.

10. The system as claimed in claim 9, wherein the user interface is as described in any of claims 2-8.

Images