Medical image display system

Abstract

The invention provides a medical image display system, which comprises a glasses part for receiving and recognizing a voice command, a gesture command and a staring command of a user, selecting a gesture command of the user from the recognized gesture commands by using the staring command, correcting the voice command by using the gesture command of the user, and displaying the corrected voice command on the glasses part. Forming a request instruction; and the server is used for receiving the request instruction and providing the medical image of the shot object to the glasses part according to the request instruction.

Description

Technical Field

[0001] This invention relates to a medical image display system, specifically to a medical image display system that accurately generates request instructions and displays medical images within the user's field of vision in a manner that fully overlaps with the subject being photographed. Background Technology

[0002] With the development of medical technology, numerous medical imaging techniques have emerged, including fluorescence imaging, optical coherence tomography (OCT), CT, MRI, and ultrasound. Different medical imaging techniques have their own advantages and specific applications. Therefore, to gain a comprehensive understanding of a patient's condition, doctors typically employ multiple medical imaging techniques to conduct a thorough examination of the affected area, making a comprehensive judgment on the condition and obtaining an accurate diagnosis.

[0003] Fluorescence imaging, in particular, utilizes autofluorescent substances found in the human body or pre-injected fluorescent contrast agents to clearly reveal tissues, organs, and lesions that are invisible to other medical imaging techniques. Typically, fluorescence imaging produces planar images. To obtain three-dimensional fluorescence images, the sample must be scanned layer by layer, a cumbersome and time-consuming process that cannot be completed quickly or in real-time. Therefore, it cannot be integrated with other types of medical imaging in real-time to comprehensively reflect the actual situation.

[0004] On the other hand, with the development of technologies such as AR (Augmented Reality) and VR (Virtual Reality), more and more smart display devices and interactive devices are being applied in the medical field. In existing technologies, some AR or VR devices already have the function of being controlled by voice or gestures, but there is still a problem that interference from other people's voices, movements, or gestures cannot be eliminated.

[0005] Therefore, there is an urgent need for a medical imaging display system that can simultaneously correct and determine request instructions based on multiple commands, and based on this, integrate real-time stereofluorescence tomography images and other types of medical images into an image that is superimposed on the corresponding parts of the human body and directly displayed to the user through glasses. Summary of the Invention

[0006] Technical issues

[0007] One object of the present invention is to provide a medical image display system that can accurately generate request instructions for medical images.

[0008] Another objective of this invention is to provide a medical imaging display system that can generate real-time stereoscopic fluorescence tomography images and display them in the user's field of vision in a manner corresponding to the size, angle, etc. of the object being photographed.

[0009] Another objective of this invention is to provide a medical imaging display system that can integrate real-time stereofluorescence tomography images with other types of medical images to form a real-time integrated image, and display it in the user's field of vision in a manner corresponding to the size, angle, etc. of the photographed object.

[0010] The technical problems that this invention aims to solve are not limited to those described above. Those skilled in the art can understand other technical problems that this invention aims to solve based on the description in this specification.

[0011] Technical solution

[0012] The present invention provides a medical image display system, comprising: an eyeglass unit that receives and recognizes a user's voice commands, gesture commands, and gaze commands, selects a user's gesture command from the recognized gesture commands using the gaze commands, and corrects the voice commands using the user's gesture commands to form a request command; and a server that receives the request command and provides the eyeglass unit with a medical image of the subject to be photographed according to the request command.

[0013] In one embodiment, the glasses may include: a voice unit for receiving and recognizing the voice commands; a vision unit for observing and recognizing objects in front of the user and the gesture commands; an eye-tracking unit for tracking the user's eyes to recognize the user's gaze and the gaze commands; and a display unit for displaying the medical images through the lenses.

[0014] As one embodiment, a stereofluorescence tomography apparatus may be further included, the apparatus being used to generate a real-time stereofluorescence tomographic image of the subject being imaged, the medical image including the stereofluorescence tomographic image.

[0015] In one implementation, the server can receive the stereofluorescence tomography image from the stereofluorescence tomography device, and when it receives a request instruction for the stereofluorescence tomography image from the eyeglasses, it provides the stereofluorescence tomography image to the eyeglasses. When the user's gaze is on the subject being photographed, the eyeglasses can display the stereofluorescence tomography image in a manner that overlaps the subject being photographed within the user's field of vision.

[0016] As one implementation, the medical image may further include at least one of computed tomography (CT) images, magnetic resonance imaging (MRI) images, X-ray images, and ultrasound images, and the server may provide at least one of the stereofluorescence tomography images, CT images, MRI images, X-ray images, and ultrasound images to the eyeglasses according to the request instruction.

[0017] In one implementation, the server can integrate two or more medical images to form an integrated image, and when it receives a request instruction for the integrated image from the glasses, it provides the integrated image to the glasses. When the user's gaze is on the subject being photographed, the glasses can display the integrated image in a manner that overlaps the subject being photographed within the user's line of sight.

[0018] As one implementation, the integrated image may include an image formed by integrating the stereofluorescence tomography image with at least one of computed tomography, magnetic resonance imaging, X-ray image and ultrasound image.

[0019] In one embodiment, the stereofluorescence tomography device may include: an imaging unit for capturing images of a subject to obtain a fluorescence image and an optical coherence tomography image; an image processing unit for processing the images captured by the imaging unit to obtain a stereofluorescence tomography image; a plurality of excitation light emitting units for emitting excitation light beams with a field of view angle toward the subject to excite fluorescence in the subject; a tomography light emitting unit for emitting tomography light beams to perform optical coherence tomography on the subject; a first reflecting mirror for reflecting the tomography light beam onto the principal optical axis of the imaging unit; a beam splitter disposed on the principal optical axis for splitting the tomography light beam reflected by the first reflecting mirror into a first beam directed toward the subject along the principal optical axis and a transmitted second beam; and a second reflecting mirror. The second beam is reflected back to the beam splitter; a multi-wavelength light source unit, connected to the plurality of excitation light emitting units and the tomographic light emitting unit, is used to provide excitation light sources of different wavelengths to the plurality of excitation light emitting units, so that the plurality of excitation light emitting units emit excitation light beams of different wavelengths to excite fluorescence at different depths of the photographed object, and at the same time, to provide tomographic light sources for forming the tomographic light beams to the tomographic light emitting units; and a wavelength-tunable filter, disposed in front of the field of view of the photographing unit, filters the light incident on the photographing unit according to the different wavelengths of the excitation light sources and the tomographic light sources, so that the photographing unit simultaneously obtains fluorescence images of the photographed object at different depths based on the first fluorescent developer and optical coherence tomographic images of the photographed object as a whole.

[0020] In one embodiment, the image processing unit can calculate the depth corresponding to each of the fluorescence images based on the depth information in the optical coherence tomography image, and combine each of the fluorescence images according to its depth to form a stereofluorescence tomography image of the photographed object corresponding to the first fluorescent developer.

[0021] In one embodiment, the image processing unit can extract a first separation image corresponding to a first fluorescent developer and a second separation image corresponding to a second fluorescent developer from each of the fluorescence images, calculate the depth corresponding to each of the first separation image and the second separation image based on the depth information in the optical coherence tomography image, and combine each of the first separation image and the second separation image according to their depths to form a stereofluorescence tomography image of the photographed object corresponding to the first fluorescent developer and a stereofluorescence tomography image corresponding to the second fluorescent developer.

[0022] Beneficial effects

[0023] The medical image display system of the present invention can accurately generate request instructions for medical images.

[0024] The medical imaging display system of the present invention can form real-time stereoscopic fluorescence tomography images and display them in the user's field of vision in a manner corresponding to the size, angle, etc. of the photographed object.

[0025] The medical imaging display system of the present invention can integrate real-time stereofluorescence tomography images with other types of medical images to form a real-time integrated image, and display it in the user's field of vision in a manner corresponding to the size, angle, etc. of the photographed object.

[0026] The beneficial effects that can be achieved by the present invention are not limited to those described above. Those skilled in the art can understand other beneficial effects that the present invention can achieve based on the description of the present invention. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a medical imaging display system according to one embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of a stereofluorescence tomography apparatus according to one embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram of a stereofluorescence tomography apparatus according to another embodiment of the present invention.

[0030] Figure 4 This is a bottom view of a plurality of excitation light emitting units according to one embodiment of the present invention.

[0031] Figure 5 This is a schematic diagram of the overlapping configuration of multiple excitation beams according to one embodiment of the present invention.

[0032] Figure 6 yes Figure 3 An enlarged view of part A in the image.

[0033] Figure 7 This is a bottom view of the first adjustment device according to one embodiment of the present invention.

[0034] Figure 8 This is a front sectional view of the first adjusting device according to one embodiment of the present invention.

[0035] Figure 9 This is a schematic diagram of a stereofluorescence tomography apparatus according to another embodiment of the present invention.

[0036] Figure 10 yes Figure 9 An enlarged view of part B in the image.

[0037] Figure 11 This is a bottom view of the light-emitting part according to one embodiment of the present invention.

[0038] Figure 12 This is a schematic diagram of fluorescence imaging according to one embodiment of the present invention.

[0039] Figure Labels

[0040] 1000: Medical Imaging Display System

[0041] 100: Eyeglasses Department

[0042] 200: Server

[0043] 10: Stereo-fluorescence tomography device

[0044] 1: Filming Department

[0045] 11: Main optical axis

[0046] 2: Light-emitting part

[0047] 21: Excitation light emitting section

[0048] 211: Excitation beam

[0049] 212: Overlapping region

[0050] 213: Overlapping Space

[0051] 214: Maximum overlap region

[0052] 22: Tomographic light emitting section

[0053] 221: Tomographic beam

[0054] 222: First Beam

[0055] 223: Second Beam

[0056] 224: Irradiation area

[0057] 3: Multi-wavelength light source section

[0058] 4: Wavelength-tunable filter

[0059] 5: Distance measuring unit

[0060] 6: Diameter Adjustment Section

[0061] 61: First regulating device

[0062] 611: Fixing part

[0063] 612: Telescopic section

[0064] 613: Through hole

[0065] 614: Visible light source

[0066] 62: Second regulating device

[0067] 71: First reflecting mirror

[0068] 72: Beam splitter

[0069] 73: Second reflecting mirror

[0070] 81: First Angle Adjustment Section

[0071] 82: Second Angle Adjustment Section

[0072] 83: Third Angle Adjustment Section

[0073] S: Subject of the photograph

[0074] D1: First diameter

[0075] D2: Maintain distance

[0076] d1: Depth 1

[0077] dn: the nth depth

[0078] E1: First excitation beam

[0079] En: The nth excitation beam

[0080] F1: First fluorescence

[0081] Fn: nth fluorescence

[0082] T1: Time 1

[0083] Tn: Time n

[0084] t1: The first time period

[0085] tn: the nth time interval

[0086] λ1: First wavelength

[0087] λn: the nth wavelength Detailed Implementation

[0088] The present invention will now be described in detail through various embodiments, but these are merely examples and the present invention is not limited thereto.

[0089] Furthermore, several terms are used in this invention, defined in a manner that provides a detailed description of various embodiments of the invention. These terms are not limited to their meanings as commonly understood by those skilled in the art, nor are the inventions limited thereto. Specifically, the terms "first," "second," etc., are used only to distinguish different elements and do not constitute a ranking of the elements; the term "wavelength" refers to a band of light with a specific wavelength as its center and a defined range; the term "excitation light" refers to light with a wavelength capable of emitting fluorescence; the term "tomography light" refers to light with a wavelength capable of being used for optical coherence tomography; the term "coherent light" refers to light generated by interference at a beam splitter; and the term "stereochromatographic tomographic image" refers to a stereoscopic image formed by stacking fluorescence images and optical coherence tomographic images at different depths along the depth direction.

[0090] Furthermore, the proportions and dimensions of the various components have been altered or even exaggerated in the accompanying drawings to enable those skilled in the art to understand the various components in this invention. The actual dimensional relationships of the structure of this invention are not limited thereto.

[0091] The medical imaging display system of the present invention will now be described in detail with reference to the accompanying drawings.

[0092] Figure 1 This is a schematic diagram of a medical imaging display system according to one embodiment of the present invention.

[0093] Reference Figure 1The medical imaging display system 1000 of the present invention may include an eyeglass unit 100 and a server 200. The eyeglass unit 100 is used to receive and recognize user voice commands and action commands, and can use the action commands to correct the voice commands to form request commands. The server 200 can receive the request commands and provide the eyeglass unit 100 with medical images of the subject to be photographed (e.g., stereofluorescein images, computed tomography images, magnetic resonance imaging, X-ray images, and ultrasound images, etc.) according to the request commands. Furthermore, the voice commands and action commands may be pre-set and stored in at least one of the eyeglass unit 100 and the server 200.

[0094] As one implementation, the action command may include a gesture command. In this case, the glasses 100 can receive and recognize the voice command and the gesture command, and use the gesture command to correct the voice command to form the request command.

[0095] Different gesture commands can be set for each voice command. Of course, the correspondence between the voice commands and the gesture commands can be one-to-one or one-to-many, as long as each gesture command corresponds to only one specific voice command. Furthermore, the gesture commands can have the same or different meanings depending on whether the user makes a fist with their left or right hand. For example, a "fist" made with the left hand and a "fist" made with the right hand can have the same or different meanings. Additionally, the gesture command can be a single hand gesture or multiple consecutive hand gestures, such as a "fist" or consecutive "fist" and "paper." These can be appropriately set by those skilled in the art according to the actual situation.

[0096] For example, the voice command could be "show stereofluorescence tomography image," and the corresponding gesture command could be "fist." When the glasses 100 receives the voice command "show stereofluorescence tomography image" and recognizes the "fist" gesture command, it can generate and send the "show stereofluorescence tomography image" request command to the server 200. At this time, the server 200 can receive the request command and provide the glasses 100 with a stereofluorescence tomography image of the subject being photographed. The stereofluorescence tomography image can be a real-time stereofluorescence tomography image of the subject being photographed, generated by the stereofluorescence tomography imaging device 10 of the present invention, described later.

[0097] Therefore, it is possible to easily and quickly correct the user's voice commands, preventing the formation of incorrect request commands or the failure to form request commands due to user mispronunciation or the glasses 100's misrecognition of voice commands. This allows for the accurate and efficient formation of request commands, thereby improving the display efficiency of medical images.

[0098] Furthermore, in environments involving multiple people such as surgery, consultations, and outpatient clinics, the glasses 100 may receive and recognize multiple voice commands. For example, when a user says the voice command "Show stereofluorescence tomography images," others nearby may simultaneously or sequentially say other pre-set voice commands such as "Retrieve medical records" or "Show CT scans." In this case, the glasses 100 can easily and accurately determine the user's voice command "Show stereofluorescence tomography images" based on the user's fist gesture, and form the correct request command accordingly, thereby quickly and accurately displaying the medical images the user wishes to view.

[0099] Alternatively, the glasses 100 can also determine the voice command from the user based on voice recognition; however, considering that there may be others with similar voices to the user, this may have limitations in a multi-person conversation environment as described above. Therefore, it is preferable to use the gesture command to correct the voice command.

[0100] More preferably, the action command may further include a gaze command, in which case the gaze command can be used to select the user's gesture command from the recognized gesture commands. Specifically, in environments with multiple people, such as surgery, consultations, and outpatient clinics, the glasses 100 may simultaneously observe and recognize multiple hands, thereby recognizing multiple gesture commands. For example, when a user gives the voice command "Show stereofluorescence tomography images" and makes a "fist" gesture, the glasses 100 may, in addition to recognizing the user's gesture command, also recognize other pre-set gesture commands such as "scissors" (i.e., only the index and middle fingers are extended) or "paper" (i.e., all fingers are fully extended). At this time, the glasses 100 can select the user's gesture command from the recognized multiple gesture commands based on the user's gaze command (e.g., the user's gaze looks at their hand and stays for a predetermined time), thereby quickly and accurately determining the gesture command from the user, simply and accurately determining the user's voice command, and forming the correct request command based on this, thereby quickly and accurately displaying the medical images that the user wishes to view.

[0101] Preferably, the glasses unit 100 may include: a voice unit for receiving and recognizing the voice commands; a vision unit for observing and recognizing objects in front and the gesture commands; an eye-tracking unit for tracking the user's eyes to recognize the user's gaze and the gaze commands; and a display unit for displaying the medical images through the lenses.

[0102] The voice unit may include, for example, a voice input unit for receiving the user's voice; a voice-to-text conversion unit for converting the voice into text; a natural language processing unit for processing and analyzing the semantics and information of the text; and an information extraction unit for extracting information such as the patient's name, body part, image type, and shooting time from the semantics and information.

[0103] In the speech unit, the speech input unit may be a miniature microphone, and the speech-to-text conversion unit, the natural language processing unit, and the information extraction unit may be functional modules integrated into a microprocessor or programs stored in a memory; however, the present invention is not limited thereto.

[0104] In addition, the vision unit and the eye-tracking unit may include a miniature camera and may include a microprocessor with machine vision capabilities. The display unit may be a display module in an optical see-through head-mounted display, but the present invention is not limited thereto.

[0105] Therefore, without the need for additional equipment, the glasses 100 can collect and process the voice commands and the action commands to form the request command and display the medical images.

[0106] Refer again Figure 1 The medical imaging display system 1000 of the present invention may further include a stereofluorescence tomography device 10, which is used to form a real-time stereofluorescence tomography image of the subject being photographed.

[0107] The stereofluorescence tomography apparatus 10 in the medical imaging display system of the present invention will be described in detail below.

[0108] Figure 2 This is a schematic diagram of a stereofluorescence tomography apparatus according to one embodiment of the present invention.

[0109] Reference Figure 2 The stereofluorescence tomography apparatus 10 of the present invention may include an imaging unit 1, a light-emitting unit 2, a multi-wavelength light source unit 3, and a wavelength-tunable filter 4.

[0110] The imaging unit 1 is used to photograph the subject S to obtain a fluorescence image and an optical coherence tomography (OCT) image of the subject S. The imaging unit 1 may include existing medical imaging cameras (e.g., 3CCD, 3CMOS, 3InGaAs image sensors, etc.), as long as they can obtain fluorescence and OCT images; the present invention is not limited thereto. Furthermore, the subject S may be a patient's body or other biological samples. When the subject S is capable of autofluorescence, it is not necessary to inject a fluorescent contrast agent; otherwise, a fluorescent contrast agent can be pre-injected for fluorescence development. On the other hand, in Figure 2 In this invention, the main optical axis 11 of the imaging unit 1 is oriented vertically downwards, meaning the imaging unit 1 captures images from top to bottom. However, the invention is not limited to this; the imaging unit 1 can be configured to capture images in any direction depending on the actual situation. For example, when it is necessary to capture the front of a human body lying flat on the testing table or the upper surface of a biological sample placed on the experimental table, the imaging unit 1 can be configured as follows: Figure 2 As shown, the camera takes pictures from top to bottom; when it is necessary to take pictures of the side of a person lying flat on the testing table or the front, back and side of a person standing, the camera 1 can take pictures in the horizontal direction; when it is necessary to take pictures of the oblique side of a person lying flat on the testing table, the camera 1 can take pictures from the oblique top at an inclined angle.

[0111] The light-emitting unit 2 is used to emit excitation light and tomographic light to the object being photographed S, so as to excite fluorescence in the object being photographed S and perform optical coherence tomography on the object being photographed S, thereby enabling the imaging unit 1 to obtain a fluorescence image and an optical coherence tomographic image. Specifically, the light-emitting unit 2 may include a plurality of excitation light emitting units 21 for emitting excitation light beams 211 and tomographic light emitting units 22 for emitting tomographic light beams 221.

[0112] The multi-wavelength light source unit 3 can be connected to the plurality of light-emitting units 2. Specifically, the multi-wavelength light source unit 3 can be connected to the plurality of excitation light emitting units 21 to provide the plurality of excitation light emitting units 21 with an excitation light source for forming the excitation light beam 211, and can be connected to the tomographic light emitting unit 22 to provide the tomographic light emitting unit 22 with a tomographic light source for forming the tomographic light beam 221.

[0113] The wavelength-tunable filter 4 can be disposed in front of the field of view of the imaging unit 1 to filter the light incident on the imaging unit 1, so that the imaging unit 1 can simultaneously obtain a fluorescence image and an optical coherence tomography image to construct a stereoscopic fluorescence tomography image. For example, the wavelength-tunable filter 4 can adjust the transmitted wavelength as needed to simultaneously transmit fluorescence used to form the fluorescence image and coherent light used to form the optical coherence tomography image, and can filter out background light to improve the signal-to-noise ratio of the fluorescence image and the optical coherence tomography image, thereby improving the imaging effect.

[0114] Although not shown, the stereofluorescence tomography apparatus 10 of the present invention may include an image processing unit. The image processing unit processes the image captured by the imaging unit 1 to obtain a stereofluorescence tomography image. The image processing unit may use image processing devices commonly used in the art, such as computers and processors; the present invention is not limited thereto.

[0115] Figure 3 This is a schematic diagram of a stereofluorescence tomography apparatus according to another embodiment of the present invention. Figure 4 This is a bottom view of a plurality of excitation light emitting units according to one embodiment of the present invention.

[0116] Reference Figure 3 and Figure 4 The plurality of excitation light emitting units 21 can be arranged in a ring on an arc with the main optical axis 11 of the imaging unit 1 as the center and having a first diameter D1, so as to avoid blocking the field of view of the imaging unit 1. At the same time, it is ensured that the distance between the plurality of excitation light emitting units 21 and the subject S is the same, so that the illumination effect of each excitation light emitting unit 21 on the subject S is more uniform.

[0117] Furthermore, the plurality of excitation light emitting units 21 can emit excitation light beams 211 towards the object S in a tilted manner toward one side of the main optical axis 11, so that the plurality of excitation light beams 211 converge on the main optical axis 11, thereby forming a point in the field of view of the imaging unit 1 that is jointly illuminated by the plurality of excitation light beams 211. Furthermore, the plurality of excitation light emitting units 21 can emit excitation light beams 211 with a field of view angle (i.e., diffusion angle), so that each excitation light beam 211 has a predetermined field of view, thereby forming an area in the field of view of the imaging unit 1 that is jointly illuminated by the plurality of excitation light beams 211, that is, an overlapping area 212 formed by the overlapping fields of view of the plurality of excitation light beams 211, the overlapping area 212 having a predetermined shape and size. When the object S falls into the overlapping area 212, each of the plurality of excitation light beams 211 can illuminate the object S, therefore, the imaging unit 1 can obtain a fluorescent image composed of the fluorescence excited by one or more of the plurality of excitation light beams 211 on the object S.

[0118] The field of view (FOV) of the excitation beam 211 can be adjusted or set according to actual needs, and the present invention is not limited thereto. As one embodiment, the FAV of the excitation beam 211 can be formed based on the numerical aperture (NA) of the optical fiber. For example, the multi-wavelength light source 3 can be connected to multiple optical fibers and the excitation light source can be provided outward through these multiple optical fibers. In this case, the excitation light emitting unit 21 can be the end of the optical fiber that is not connected to the multi-wavelength light source 3. Based on the numerical aperture, the beam emitted from the end of the optical fiber has a diffusion angle (i.e., FAV). That is, in this way, the multiple excitation light emitting units 21 can emit an excitation beam 211 with a FAV (i.e., diffusion angle). Furthermore, to make the excitation beam 211 more uniform, a diffuser can be further provided on the excitation light emitting unit 21, and other optical devices can also be provided to adjust other optical properties of the excitation beam 211, but the present invention is not limited thereto.

[0119] Figure 5 This is a schematic diagram of the overlapping configuration of multiple excitation beams according to one embodiment of the present invention.

[0120] Reference Figure 5 (a) and Figure 5In (b), the plurality of excitation beams 211 overlap in space to form an overlapping space 213. Within the overlapping space 213, the field of view of the plurality of excitation beams 211 includes the overlapping region 212. Furthermore, the overlapping space 213 may have an approximately double-conical shape. Therefore, when the object S falls from above into the overlapping space 213, as the distance between the object S and the plurality of excitation beam emitters 21 gradually increases, the size of the overlapping region 212 gradually increases from zero and then gradually decreases back to zero. That is, a maximum overlapping region 214 exists, and the maximum overlapping region 214 is located on the common base of the upper and lower cones of the double cone shape. The distance between the maximum overlapping region 214 and the plurality of excitation beam emitters 21 can be called the "working distance".

[0121] On the other hand, the overlapping space 213 and the maximum overlapping region 214 change according to the first diameter D1. Specifically, in comparison... Figure 5 (a) and Figure 5 In (b), the smaller the first diameter D1, the smaller the distance between the overlapping space 213 and the plurality of excitation light emitting units 21, the smaller the size of the overlapping space 213, the smaller the distance between the maximum overlapping region 214 and the plurality of excitation light emitting units 21 (i.e., the working distance), and the smaller the size of the maximum overlapping region 214; the larger the first diameter D1, the larger the distance between the overlapping space 213 and the plurality of excitation light emitting units 21, the larger the size of the overlapping space 213, the larger the distance between the maximum overlapping region 214 and the plurality of excitation light emitting units 21, and the larger the size of the maximum overlapping region 214.

[0122] That is, there is a direct proportional relationship between the first diameter D1, the distance between the overlapping space 213 and the plurality of excitation light emitting parts 21, the size of the overlapping space 213, the distance between the maximum overlapping region 214 and the plurality of excitation light emitting parts 21, and the size of the maximum overlapping region 214. This relationship can be calculated by conventional geometric methods in combination with specific parameters such as the angle at which the plurality of excitation light emitting parts 21 tilt toward the main optical axis 11 and the field of view angle, which will not be elaborated here.

[0123] The stereofluorescence tomography apparatus 10 of the present invention can precisely adjust the size and position of the overlapping region 212 so that the overlapping region 212 completely covers the object being photographed, thereby enabling the imaging unit 1 to obtain a fluorescence image of the entire surface of the object being photographed S in one go.

[0124] Specifically, refer again Figure 3The stereofluorescence tomography apparatus 10 of the present invention may further include a ranging unit 5. The ranging unit 5 may be disposed on the plane of the arc and is used to measure the distance D2 between the plurality of excitation light emitting units 21 and the object being photographed S. The ranging unit 5 may use a rangefinder commonly used in the art, for example, a rangefinder based on photoelectric or acoustic waves, but the present invention is not limited thereto.

[0125] Figure 6 yes Figure 3 An enlarged view of part A in the image.

[0126] Reference Figure 6 The stereofluorescence tomography apparatus 10 of the present invention may further include a diameter adjustment unit 6. The diameter adjustment unit 6 is combined with the plurality of excitation light emitting units 21 to adjust the first diameter D1 according to the separation distance D2 measured by the ranging unit 5, so that the field of view of the plurality of excitation light beams 211 forms an overlapping region 212 at the separation distance D2 that completely covers the object S being photographed. Thus, the excitation light beams 211 can completely illuminate the entire surface of the object S at once, thereby simultaneously performing fluorescence imaging on the entire surface of the object S.

[0127] Specifically, when the separation distance D2 measured by the ranging unit 5 is less than the distance between the overlapping space 213 formed under the current first diameter D1 and the plurality of excitation light emitting units 21, the diameter adjustment unit 6 can reduce the first diameter D1 to reduce the distance between the overlapping space 213 and the plurality of excitation light emitting units 21, thereby causing the subject S to fall into the overlapping space 213, so that the overlapping area 212 accurately falls on the subject S.

[0128] When the separation distance D2 measured by the ranging unit 5 is greater than the distance between the overlapping space 213 formed under the current first diameter D1 and the plurality of excitation light emitting units 21, the diameter adjustment unit 6 can increase the first diameter D1 to increase the distance between the overlapping space 213 and the plurality of excitation light emitting units 21, so that the subject S falls into the overlapping space 213, so that the overlapping area 212 falls precisely on the subject S.

[0129] Furthermore, the diameter adjustment unit 6 can adjust the first diameter D1 so that the entire subject S falls into the overlapping space 213, so that the overlapping area 212 completely covers the subject S, thereby forming an overlapping area 212 that completely covers the subject S at the distance D2 between the plurality of excitation light beams 211.

[0130] Preferably, the diameter adjustment unit 6 can adjust the first diameter D1 according to the spacing distance D2 so that the field of view of the plurality of excitation light beams 211 forms the maximum overlap area 214 at the spacing distance D2 (that is, the working distance is adjusted to be the same as the spacing distance D2), thereby facilitating the user to appropriately adjust the spacing distance D2 according to the actual formed maximum overlap area 214 to form a maximum overlap area 214 consistent with the size of the shooting object S, thereby maximizing the utilization rate of the plurality of excitation light beams 211.

[0131] In one embodiment, the diameter adjustment unit 6 can use a drive mechanism commonly used in the art, such as a moving mechanism composed of one or more of a controller, motor, gear, rack, guide wheel, guide rail, etc., but the present invention is not limited thereto. In a preferred embodiment, the diameter adjustment unit 6 can be the first adjustment device described later.

[0132] Figure 7 This is a bottom view of the first adjustment device according to one embodiment of the present invention. Figure 8 This is a front sectional view of the first adjusting device according to one embodiment of the present invention.

[0133] Reference Figure 7 and Figure 8 The first adjusting device 61 may include: a fixing part 611, which is in the shape of an annular disc; a telescopic part 612, which is formed of piezoelectric material, is in the shape of an annular disc, and is fixed to the inner circumferential surface of the fixing part 611, and has a plurality of through holes 613 formed on an arc concentric with it and having the first diameter D1; and a power supply part (not shown), which is connected to the telescopic part 612 and is used to apply current to the telescopic part 612 to deform the telescopic part 612 so as to change the first diameter D1.

[0134] At this time, the first adjustment device 61 can be configured to be concentric with the main optical axis 11, the plurality of excitation light emitting parts 21 can be respectively arranged in each of the through holes 613 in a manner inclined toward one side of the main optical axis 11, and the ranging part 5 can be arranged in at least one of the telescopic part 612 and the fixed part 611 to measure the distance D2 between the plurality of excitation light emitting parts 21 and the shooting object S.

[0135] When it is necessary to photograph the object S, the power supply unit calculates the target diameter based on the measurement result of the ranging unit 5 (i.e., the separation distance D2), and then applies a corresponding current to the telescopic unit 612 according to the target diameter to adjust the first diameter D1 to the target diameter. This causes the fields of view of the plurality of excitation light beams 211 to overlap at the object S and form an overlapping area 212 that completely covers the object S. The target diameter is a value or range of the first diameter D1, which allows the fields of view of the plurality of excitation light beams 211 to form an overlapping area 212 that completely covers the object S at the separation distance D2.

[0136] Additionally, the first adjustment device 61 may be further configured with a visible light source 614, which is used to illuminate the object S with visible light to provide illumination, thereby facilitating user operation. The visible light source 614 may be, for example, as shown below. Figure 7 The annular LED light source shown is configured between the fixed part 611 and the telescopic part 612. However, the present invention is not limited to this. The shape, number, size, power, brightness, etc. of the visible light source 614 can be selected and / or adjusted according to actual needs, as long as it can provide sufficient illumination.

[0137] On the other hand, the stereofluorescence tomography apparatus 10 of the present invention may further include a distance adjustment unit (not shown). The distance adjustment unit is used to adjust the separation distance D2 so that the diameter adjustment unit 6 adjusts the first diameter D1 according to the adjusted separation distance D2, so that the field of view of the plurality of excitation light beams 211 forms an overlapping area 212 at the separation distance D2 that completely covers the object S being photographed.

[0138] The distance adjustment unit can use a drive mechanism commonly used in the art. For example, it can use a moving mechanism composed of one or more of a controller, motor, gear, rack, guide wheel, guide rail, etc. The present invention is not limited thereto.

[0139] Furthermore, the distance adjustment unit can adjust the spacing distance D2 according to the size of the object being photographed S, and the diameter adjustment unit 6 can adjust the first diameter D1 according to the adjusted spacing distance D2, so that the field of view of the plurality of excitation light beams 211 forms a maximum overlap area 214 at the spacing distance D2 that is consistent with the size of the object being photographed S. The size of the object being photographed S can be manually input by the user or calculated using a visual detection method, and the present invention is not limited thereto.

[0140] As a specific implementation, firstly, the target distance can be calculated based on the size of the object being photographed S. The target distance can be a value of the separation distance D2. When the separation distance D2 is adjusted to the target distance, the fields of view of the plurality of excitation light beams 211 can form a maximum overlap area 214 at the separation distance D2 that is consistent with the size of the object being photographed S. The target distance can be calculated using a separate controller or computer, etc., and the present invention is not limited thereto.

[0141] Subsequently, the distance adjustment unit can adjust the separation distance D2 to the target distance, and the diameter adjustment unit 6 can adjust the first diameter D1 according to the adjusted separation distance D2, so that the field of view of the plurality of excitation light beams 211 forms the maximum overlap area 214 at the separation distance D2 (that is, the working distance is adjusted to be the same as the adjusted separation distance D2).

[0142] Therefore, by coordinating the diameter adjustment unit 6 and the distance adjustment unit, at least one of the first diameter D1 and the separation distance D2 can be appropriately adjusted, thereby maximizing the illumination accuracy of the excitation light beam 211 on the photographed object S and the utilization rate of the excitation light beam 211.

[0143] The field of view (Field of View) angles of the plurality of excitation beams 211 may differ from one another. For example, the plurality of excitation light emitting units 21 may be formed with different Field of View angles, or there may be slight differences in the Field of View angles of the excitation beams 211 emitted by different excitation light emitting units 21 due to unavoidable errors or tolerances in the manufacturing process of each of the plurality of excitation light emitting units 211. For another example, when the excitation light emitting unit 21 is the end of an optical fiber as described above, since different wavelengths of light have different numerical apertures in the same optical fiber, there may be slight differences in the Field of View angles of excitation beams 211 of different wavelengths.

[0144] This difference may affect the overlap rate of the plurality of excitation beams 211, that is, it may affect the sharpness and accuracy of the outline boundary of the overlapping region 212. The requirements for the sharpness and accuracy of the outline boundary of the overlapping region 212 vary depending on the size of the object being photographed S. Specifically, the smaller the size of the object being photographed S, the smaller the size of the overlapping region 212 formed accordingly, and the greater the impact caused by the misalignment of the outline boundary of the overlapping region 212. Therefore, the requirements for the sharpness and accuracy of the outline boundary of the overlapping region 212 are higher.

[0145] The stereofluorescence tomography device 10 of the present invention can precisely adjust the contour boundary of the overlapping region 212 to ensure the clarity and accuracy of the contour boundary of the overlapping region 212.

[0146] Specifically, refer again Figure 6 The stereofluorescence tomography device 10 of the present invention may further include a plurality of first angle adjustment units 81.

[0147] The plurality of first angle adjustment parts 81 may be disposed on the diameter adjustment part 6 and combined with the plurality of excitation light emitting parts 21 for adjusting the angle of inclination of the plurality of excitation light emitting parts 21 toward the main optical axis 11, so as to adjust the overlapping area 212, specifically, to adjust the contour boundary of the overlapping area 212.

[0148] In one embodiment, the plurality of first angle adjustment units 81 may use a rotating mechanism commonly used in the art, for example, a rotating mechanism composed of one or more of a controller, motor, shaft, gear, etc., and the present invention is not limited thereto. In a preferred embodiment, the plurality of first angle adjustment units 81 may be a plurality of second adjustment devices described later.

[0149] Refer again Figure 8 The plurality of second adjustment devices 62 may be formed of piezoelectric material and may be disposed on the telescopic portion 612 of the first adjustment device 61, and respectively coupled one-to-one with the plurality of excitation light emitting portions 21, and connected to the power supply. When it is necessary to adjust the contour boundary of the overlapping region 212, current can be applied to the second adjustment device 62 coupled with the excitation light emitting portion 21 to be adjusted through the power supply, so that the second adjustment device 62 deforms, thereby adjusting the angle at which the excitation light emitting portion 21 tilts towards the main optical axis 11, and finally adjusting the position and angle of overlap between the excitation light beam 211 emitted by the excitation light emitting portion 21 and other excitation light beams 211. Thus, the contour boundary of the overlapping region 212 can be finely adjusted.

[0150] Figure 9 This is a schematic diagram of a stereofluorescence tomography apparatus according to another embodiment of the present invention.

[0151] Reference Figure 9The stereofluorescence tomography apparatus 10 of the present invention may further include: a tomography light emitting unit 22, connected to the multi-wavelength light source unit 3, for emitting a tomography light beam 221 to perform optical coherence tomography on the photographed object S; a first reflecting mirror 71 for reflecting the tomography light beam 221 onto the principal optical axis 11; a beam splitter 72, disposed on the principal optical axis 11, for splitting the tomography light beam 221 reflected by the first reflecting mirror 71 into a first beam 222 directed along the principal optical axis 11 toward the photographed object S and a transmitted second beam 223; and a second reflecting mirror 73 for reflecting the second beam 223 back to the beam splitter 72. Thus, the stereofluorescence tomography apparatus 10 of the present invention can perform optical coherence tomography on the photographed object, thereby obtaining an optical coherence tomography image having depth information of the entire photographed object S.

[0152] Figure 10 yes Figure 9 An enlarged view of part B in the image.

[0153] Reference Figure 10 The stereofluorescence tomography imaging device 10 of the present invention may further include: a second angle adjustment unit 82, disposed on the diameter adjustment unit 6 and combined with the tomographic light emitting unit 22, for adjusting the angle of the tomographic light emitting unit 22; and / or a third angle adjustment unit 83, combined with the first reflector 71, for adjusting the angle of the first reflector 71. Through the second angle adjustment unit 82 and / or the third angle adjustment unit 83, the optical path of the tomographic light beam 221 can be precisely adjusted so that the first beam 222 accurately illuminates the entire surface of the object S being photographed, thereby obtaining the optical coherence tomographic image. The second angle adjustment unit 82 and the third angle adjustment unit 83 may use the same rotating mechanism commonly used in the art as the first angle adjustment unit 81. For example, a rotating mechanism composed of one or more of a controller, motor, shaft, gear, etc., may be used, or it may be formed of a piezoelectric material; the present invention is not limited thereto.

[0154] Figure 11 This is a bottom view of the light-emitting part according to one embodiment of the present invention.

[0155] Reference Figure 11 The tomographic light emitting unit 22 can be disposed on the arc. Furthermore, the tomographic light emitting unit 22 can be arranged together with the plurality of excitation light emitting units 21 in a ring on an arc having a first diameter D1 centered on the main optical axis 11 of the imaging unit 1, so as to avoid obstructing the field of view of the imaging unit 1.

[0156] Furthermore, when the diameter adjustment unit 6 is the first adjustment device 61, the tomographic light emitting unit 22 can be disposed in one of the plurality of through holes 613. In this case, the second angle adjustment unit 82 can use the second adjustment device 62. Therefore, not only can the structure of the stereofluorescence tomography imaging device of the present invention be made more compact to improve space utilization, but the overall number and types of parts constituting the stereofluorescence tomography imaging device of the present invention can also be reduced, thus improving assembly efficiency and maintenance convenience.

[0157] The hardware structure of the stereofluorescence tomography apparatus 10 of the present invention and the operation of overlapping the fields of view of the plurality of excitation beams 211 have been described in detail above. Hereinafter, the operation of obtaining a fluorescence image by the stereofluorescence tomography apparatus 10 of the present invention and the operation of forming the stereofluorescence tomography image by combining the optical coherence tomography image will be described in detail.

[0158] The stereofluorescence tomography device 10 of the present invention can simultaneously obtain fluorescence images at different depths of the photographed object S and optical coherence tomography images of the photographed object S as a whole. Based on the depth information in the optical coherence tomography images, the depth of each fluorescence image is calculated, and the fluorescence images are combined according to their depths to form the stereofluorescence tomography image.

[0159] Specifically, the multi-wavelength light source unit 3 can provide excitation light sources of different wavelengths to the plurality of excitation light emitting units 21, so that the plurality of excitation light emitting units 21 emit excitation light beams 211 of different wavelengths respectively, thereby exciting fluorescence at different depths of the photographed object S. At the same time, the multi-wavelength light source unit 3 can also provide tomographic light emitting units 22 with tomographic light sources for forming the tomographic light beam 221, so as to perform optical coherence tomography on the photographed object.

[0160] Correspondingly, the wavelength-tunable filter 4 can filter according to the excitation light source and the tomographic light source of different wavelengths, so that the imaging unit 1 can simultaneously obtain fluorescence images of different depths of the subject S and optical coherence tomographic images of the subject S as a whole.

[0161] Figure 12 This is a schematic diagram of fluorescence imaging according to one embodiment of the present invention.

[0162] Reference Figure 12 In (a), the multi-wavelength light source unit 3 can, for example, provide different wavelengths of excitation light sources to n excitation light emitting units 21, that is, from the first excitation light emitting unit to the nth excitation light emitting unit, namely, excitation light sources with wavelengths from the first wavelength λ1 to the nth wavelength λn. Here, n is a natural number greater than 1.

[0163] Specifically, the multi-wavelength light source 3 can sequentially provide the first excitation light emitting unit to the nth excitation light emitting unit with an excitation light source of the first wavelength λ1 to the nth excitation light emitting unit respectively, so that the first excitation light emitting unit to the nth excitation light emitting unit sequentially emits the first excitation light beam E1 of the first wavelength λ1 to the nth excitation light beam En of the nth wavelength λn respectively, thereby sequentially exciting the first fluorescence F1 to the nth fluorescence Fn of the object being photographed S from the first depth d1 to the nth depth dn respectively.

[0164] Correspondingly, the wavelength-tunable filter 4 can transmit the first excitation light source to the nth excitation light source provided by the multi-wavelength light source unit 3, respectively transmitting the first fluorescence F1 to the nth fluorescence Fn, so that the imaging unit 1 sequentially obtains the first fluorescence image of the first depth d1 composed of the first fluorescence F1 to the nth fluorescence image of the nth depth dn composed of the nth fluorescence Fn.

[0165] Preferably, the operation of the multi-wavelength light source 3 providing the excitation light source of different wavelengths can be performed according to a specific time, and the operation of the wavelength-tunable filter 4 filtering the light incident on the imaging unit 1 can also be performed according to the specific time, so that the multi-wavelength light source 3 and the wavelength-tunable filter 4 operate synchronously, thereby enabling the imaging unit 1 to accurately obtain fluorescence images of different depths for the subject S.

[0166] Specifically, refer to Figure 12 In (b), the multi-wavelength light source unit 3 may, for example, provide the first excitation light source with wavelength λ1 to the nth excitation light source with wavelength λn to the first excitation light emitting unit to the nth excitation light emitting unit in the first time period t1 to the nth excitation light emitting unit in the nth time period tn.

[0167] Correspondingly, the wavelength-tunable filter 4 can transmit the first fluorescence F1 to the nth fluorescence Fn during the first time period t1 to the nth time period tn, respectively, from the first time period T1 to the nth time period Tn.

[0168] The time difference between each adjacent time point from the first time point T1 to the nth time point Tn may be the same or different. The (n-1)th time period tn-1 may be less than or equal to the time difference between the (n-1)th time point Tn-1 and the nth time point Tn. The first time period t1 to the nth time period tn may be the same or different from each other. However, each time period from the first time period t1 to the nth time period tn should be within a predetermined time period. The predetermined time period should be appropriately selected based on factors such as computing power and resolution. The total duration from the first time point T1 to the nth time point Tn should be within the predetermined duration. The predetermined duration is sufficient to enable the stereofluorescence tomography imaging device 10 of the present invention to construct and display the stereofluorescence tomography image in real time.

[0169] When the imaging unit 1 obtains the first to the nth fluorescence images of the subject S, the image processing unit can calculate the depth corresponding to each fluorescence image based on the depth information in the simultaneously obtained optical coherence tomography images, stack the fluorescence images according to their depths, and then, based on each fluorescence image, form a filling image corresponding to the depths between and / or outside the depths corresponding to each fluorescence image through interpolation, thereby forming the stereoscopic fluorescence tomography image of the entire subject S. Therefore, the stereoscopic fluorescence tomography imaging device 10 of the present invention can quickly obtain a stereoscopic fluorescence tomography image without scanning the subject S layer by layer.

[0170] The above description assumes that the subject S contains only one type of autofluorescent substance or fluorescent developer. However, in practice, it may be necessary to obtain stereofluorescence tomographic images of different parts of the subject S that require different fluorescent developers. In such cases, it is necessary to inject multiple fluorescent developers into the subject S.

[0171] The following is a detailed description of the operation for obtaining stereofluorescence tomography images when the subject S contains multiple autofluorescent substances or fluorescent developers, but content that is repeated above will not be repeated.

[0172] At this time, the multi-wavelength light source unit 3 can provide excitation light sources of different wavelengths to the plurality of excitation light emitting units 21, so that the plurality of excitation light emitting units 21 emit excitation light beams 211 of different wavelengths respectively. At the same time, the multi-wavelength light source unit 3 can also provide tomographic light emitting units 22 with tomographic light sources for forming the tomographic light beam 221, so as to perform optical coherence tomography on the photographed object.

[0173] Correspondingly, the wavelength-tunable filter 4 can filter according to the different wavelengths of the excitation light source and the tomographic light source, so that the imaging unit 1 can simultaneously obtain fluorescence images of the subject S at different depths corresponding to various autofluorescent substances and / or fluorescent developers, as well as an optical coherence tomographic image of the subject S as a whole.

[0174] Specifically, the multi-wavelength light source unit 3 can sequentially provide p types of excitation light sources to p excitation light emitting units 21 for exciting fluorescence of the first fluorescent developer, so as to excite fluorescence based on the first fluorescent developer at p depths of the photographed object S, thereby obtaining p fluorescent images corresponding to the first fluorescent developer. The multi-wavelength light source unit 3 can also sequentially provide q types of excitation light sources to q excitation light emitting units 21 for exciting fluorescence of the second fluorescent developer, so as to excite fluorescence based on the second fluorescent developer at q depths of the photographed object S, thereby obtaining q fluorescent images corresponding to the second fluorescent developer. There may be overlap between the p types of excitation light sources for exciting fluorescence of the first fluorescent developer and the q types of excitation light sources for exciting fluorescence of the second fluorescent developer; that is, there may be the same excitation light emitting unit 21 between the p excitation light emitting units 21 and the q excitation light emitting units 21. This is determined by the excitation wavelength of the first fluorescent developer and the second fluorescent developer themselves, and the present invention is not limited thereto.

[0175] When the imaging unit 1 obtains a fluorescence image of the subject S corresponding to the first fluorescent developer and a fluorescence image corresponding to the second fluorescent developer, the image processing unit can extract a first separated image corresponding to the first fluorescent developer and a second separated image corresponding to the second fluorescent developer from each fluorescence image. Based on the depth information in the optical coherence tomography image obtained simultaneously, the unit calculates the depth corresponding to each first separated image and the second separated image, and combines each first separated image and the second separated image according to their depths to form a stereofluorescence tomography image of the subject S corresponding to the first fluorescent developer and a stereofluorescence tomography image corresponding to the second fluorescent developer.

[0176] Specifically, the image processing unit can stack each of the first separated images and the second separated images according to their depths, and based on each of the first separated images and the second separated images, it uses interpolation to form a first filling image corresponding to the depths between and / or outside the depths corresponding to each of the first separated images, and a second filling image corresponding to the depths between and / or outside the depths corresponding to each of the second separated images, thereby forming a stereofluorescence tomographic image of the entire photographed object S corresponding to the first fluorescent developer and a stereofluorescence tomographic image corresponding to the second fluorescent developer. Therefore, the stereofluorescence tomographic imaging device 10 of the present invention can quickly obtain stereofluorescence tomographic images corresponding to different fluorescent developers without scanning the photographed object S layer by layer.

[0177] Furthermore, the image processing unit can combine the stereofluorescence tomographic image of the photographed object S corresponding to the first fluorescent developer with the stereofluorescence tomographic image corresponding to the second fluorescent developer. That is, the image processing unit can correlate the first separated images and / or the first filled images of the same depth of each of the first separated images and each of the first filled images to form the stereofluorescence tomographic image of the entire photographed object S. Thus, the stereofluorescence tomographic imaging apparatus 10 of the present invention can quickly obtain stereofluorescence tomographic images of different fluorescent developers located at different locations simultaneously without scanning the photographed object S layer by layer.

[0178] The above provides a detailed description of the operation of the stereofluorescence tomography apparatus 10 of the present invention in forming the stereofluorescence tomography image. The stereofluorescence tomography apparatus 10 can continuously form the stereofluorescence tomography image. Therefore, the stereofluorescence tomography apparatus 10 of the present invention can not only assist doctors in observing a wide range of areas at once, but also allow doctors to observe the current condition of the surgical site (i.e., the object being photographed S) in real time during surgery.

[0179] As one implementation, the medical imaging display system 1000 of the present invention can display the stereofluorescence tomography image to a user.

[0180] Specifically, the server 200 can receive the stereofluorescence tomography image from the stereofluorescence tomography device 10. Of course, the server 200 can receive the stereofluorescence tomography image in real time and store it in its memory.

[0181] When a request command for the stereofluorescence tomography image is received from the eyeglasses 100, the server 200 can provide the stereofluorescence tomography image to the eyeglasses 100. For example, when the request command is "display real-time stereofluorescence tomography image", the server 200 can forward the real-time stereofluorescence tomography image received from the stereofluorescence tomography imaging device 10 to the eyeglasses 100. Of course, during this process, the user can issue request commands such as "pause" or "playback". At this time, the server 200 can perform corresponding operations according to the request command. For example, it can pause the stereofluorescence tomography image at the frame when the "pause" request command is received, or it can retrieve and provide the stereofluorescence tomography image frame by frame to the eyeglasses 100 in reverse chronological order from the moment the "playback" request command is received. The present invention is not limited to these.

[0182] On the other hand, the eyeglasses 100 can display the stereofluorescence tomography image at a specific position on the lens. However, in order to facilitate the user to accurately understand the correspondence between the image and the subject being photographed, it is preferable to adjust the position, angle, size, etc. of the stereofluorescence tomography image displayed on the lens according to the relative positional relationship between the subject being photographed and the eyeglasses 100.

[0183] Specifically, when the user's gaze is directed toward the subject being photographed, the glasses 100 can display the stereoscopic fluorescence tomography image in a manner that overlaps the subject being photographed within the user's field of vision.

[0184] At this time, the eyeglasses 100 can confirm whether the subject being photographed is in front of the eyeglasses 100 and specifically at which position in front of it, i.e., the relative positional relationship, through the visual part.

[0185] Furthermore, the eyeglasses 100 can also calculate the relative positional relationship using the stereofluorescence tomography device 10. Specifically, the eyeglasses 100 can calculate its relative positional relationship with the subject being photographed using the relative positional relationship between the subject and the stereofluorescence tomography device 10 (hereinafter referred to as the "first relationship") and the relative positional relationship between the stereofluorescence tomography device 10 and the eyeglasses 100 (hereinafter referred to as the "second relationship"). In this case, a positioning tag can be provided at the imaging part 1 or other parts of the stereofluorescence tomography device 10, and a sensing part (e.g., a positioning sensor) for sensing the positioning tag can be provided in the eyeglasses 100. The first relationship can be the relative positional relationship between the positioning tag and the subject being photographed, and the second relationship can be the relative positional relationship between the positioning tag and the eyeglasses 100. The first relationship can be calculated using the separation distance D2 and the specific position of the positioning tag on the stereofluorescence tomography device 10, while the second relationship can be directly measured. The glasses 100 can quickly and accurately calculate its relative positional relationship with the photographed object by adding the spatial vectors of the first relationship and the second relationship.

[0186] Then, the eyeglasses 100 can calculate the position and angle of the user's gaze on the lens when the user looks at the subject being photographed, so that the display unit can display the stereoscopic fluorescence tomography image in the user's field of vision in a manner corresponding to the size and angle of the subject being photographed.

[0187] In addition to the stereofluorescence tomography image, the medical imaging display system 1000 of the present invention can further display computed tomography (CT) images, magnetic resonance imaging (MRI) images, X-ray images, and ultrasound images. In other words, the server 200 can provide at least one of the stereofluorescence tomography image, CT image, MRI image, X-ray image, and ultrasound image to the eyeglasses 100 according to the request instruction. The CT image, MRI image, X-ray image, and ultrasound image can be obtained by performing corresponding medical imaging on the subject beforehand using appropriate medical imaging devices and stored in the server 200. Of course, the medical imaging display system 1000 of the present invention may also directly include at least one of a computed tomography imaging device, an MRI imaging device, an X-ray imaging device, and an ultrasound imaging device; the present invention is not limited thereto.

[0188] For example, if the subject has been X-rayed and the resulting X-ray image is stored in the server 200 beforehand, the glasses 100 can send a request instruction to the server 200 to "display the X-ray image," and the server 200 can provide the X-ray image of the subject to the glasses 100 according to the request instruction. The process of forming the request instruction is as described above and will not be repeated here.

[0189] For example, if the subject has undergone X-ray and MRI imaging beforehand and the resulting X-ray and MRI images are stored in the server 200, the eyeglasses 100 can send a request command to the server 200 to "display the X-ray images and MRI images." The server 200 can then provide the eyeglasses 100 with the X-ray and MRI images of the subject according to the request command. When the eyeglasses 100 displays more than two medical images, these medical images can be displayed separately at different positions on the lenses.

[0190] Furthermore, the medical image display system 1000 of the present invention can display to the user an integrated image composed of two or more medical images.

[0191] Specifically, the server 200 can integrate two or more medical images, such as stereofluorescence tomography, computed tomography, magnetic resonance imaging, X-ray, and ultrasound, into a single integrated image based on algorithms such as SIFT, ORB, and SURF, so as to clearly display the tissues, organs, lesions, bones, etc. that the user wants to focus on.

[0192] Those skilled in the art can select the medical images to be fused as needed, but it is preferred to integrate the real-time stereofluorescence tomography image of the subject obtained by the stereofluorescence tomography device 10 of the present invention with at least one other medical image such as computed tomography, magnetic resonance imaging, X-ray image and ultrasound image to form the integrated image, so as to reflect the current state of tissues, organs, lesions, bones, etc. in real time.

[0193] When a request instruction for the integrated image is received from the eyeglasses unit 100, the integrated image can be provided to the eyeglasses unit 100.

[0194] On the other hand, the eyeglasses 100 can display the integrated image at a specific position on the lens. However, in order to facilitate the user to accurately understand the correspondence between the image and the subject being photographed, it is preferable to adjust the position, angle, size, etc. of the integrated image displayed on the lens according to the relative positional relationship between the subject being photographed and the eyeglasses 100.

[0195] Specifically, when the user's gaze is directed towards the subject being photographed, the glasses 100 can display the integrated image in a manner that overlaps the integrated image with the subject within the user's field of vision. This display method is similar to that described above and will not be elaborated upon further here.

[0196] The medical image display system of the present invention has been described in detail above, but these are merely multiple embodiments of the present invention, and the present invention is not limited thereto. Those skilled in the art can make various modifications within the scope of the technical analysis of the present invention, and all such modifications fall within the protection scope of the claims of the present invention.

Claims

1. A medical imaging display system, characterized in that, include: The glasses unit (100) receives and recognizes the user's voice commands, gesture commands, and gaze commands, selects the user's gesture commands from the recognized gesture commands using the gaze commands, and corrects the voice commands using the user's gesture commands to form a request command; as well as The server (200) receives the request instruction and provides the eyeglasses (100) with the medical image of the subject (S) according to the request instruction.

2. The medical imaging display system according to claim 1, characterized in that, The eyeglasses section (100) includes: The voice unit is used to receive and recognize the voice commands; The vision unit is used to observe and recognize the object in front and the gesture command; An eye-tracking unit is used to track the user's eyes to identify the user's gaze and the gaze command; and The display unit shows the medical images through lenses.

3. The medical imaging display system according to claim 1, characterized in that, The system further includes a stereofluorescence tomography apparatus (10) for forming a real-time stereofluorescence tomographic image of the subject (S). The medical images include the stereofluorescence tomography images.

4. The medical imaging display system according to claim 3, characterized in that, The server (200) receives the stereofluorescence tomography image from the stereofluorescence imaging device (10), and when it receives a request instruction for the stereofluorescence tomography image from the eyeglasses unit (100), it provides the stereofluorescence tomography image to the eyeglasses unit (100). When the user's gaze is directed toward the subject (S), the glasses (100) display the stereofluorescence tomography image in such a way that the stereofluorescence tomography image overlaps with the subject (S) within the user's line of sight.

5. The medical imaging display system according to claim 3, characterized in that, The medical images further include at least one of computed tomography (CT) images, magnetic resonance imaging (MRI) images, X-ray images, and ultrasound images. The server (200) provides the eyeglasses (100) with at least one of the following: stereofluorescence tomography image, computed tomography image, magnetic resonance imaging, X-ray image, and ultrasound image, according to the request instruction.

6. The medical imaging display system according to claim 5, characterized in that, The server (200) integrates two or more medical images to form an integrated image, and when it receives a request instruction for the integrated image from the eyeglasses unit (100), it provides the integrated image to the eyeglasses unit (100). When the user's gaze is directed toward the subject (S), the glasses (100) display the integrated image in such a way that the integrated image overlaps with the subject (S) within the user's line of sight.

7. The medical imaging display system according to claim 6, characterized in that, The integrated image includes an image formed by integrating the stereofluorescence tomography image with at least one of the following: computed tomography image, magnetic resonance imaging, X-ray image, and ultrasound image.

8. The medical imaging display system according to claim 3, characterized in that, The stereofluorescence tomography device (10) includes: The imaging unit (1) is used to photograph the subject (S) to obtain fluorescence images and optical coherence tomography images; The image processing unit is used to process the images captured by the imaging unit (1) to obtain stereofluorescence tomography images; Multiple excitation light emitting units (21) are used to emit excitation light beams (211) with a field of view to the photographed object (S) to excite fluorescence in the photographed object (S); The tomographic light emitting unit (22) is used to emit a tomographic light beam (221) to perform optical coherence tomography on the photographed object (S); The first reflecting mirror (71) is used to reflect the tomographic light beam (221) onto the main optical axis (11) of the imaging unit (1); A beam splitter (72) is disposed on the main optical axis (11) to split the tomographic beam (221) reflected by the first reflector (71) into a first beam (222) directed toward the photographed object (S) along the main optical axis (11) and a second beam (223) transmitted. The second reflector (73) reflects the second beam (223) back to the beam splitter (72); A multi-wavelength light source unit (3) is connected to the plurality of excitation light emitting units (21) and the tomographic light emitting unit (22), and is used to provide excitation light sources of different wavelengths to the plurality of excitation light emitting units (21) respectively, so that the plurality of excitation light emitting units (21) emit excitation light beams (211) of different wavelengths respectively, so as to excite fluorescence at different depths of the photographed object (S), and at the same time, provides the tomographic light emitting unit (22) with a tomographic light source for forming the tomographic light beam (221); and A wavelength-tunable filter (4) is disposed in front of the field of view of the imaging unit (1). Based on the excitation light source and the tomographic light source of different wavelengths, the filter filters the light incident on the imaging unit (1) so that the imaging unit (1) can simultaneously obtain a fluorescence image of the subject (S) based on the first fluorescent developer at different depths and an optical coherence tomographic image of the subject (S) as a whole.

9. The medical imaging display system according to claim 8, characterized in that, The image processing unit calculates the depth corresponding to each of the fluorescent images based on the depth information in the optical coherence tomography image, and combines each of the fluorescent images according to its depth to form a stereofluorescence tomography image of the photographed object (S) corresponding to the first fluorescent developer.

10. The medical imaging display system according to claim 8, characterized in that, The image processing unit extracts a first separation image corresponding to the first fluorescent developer and a second separation image corresponding to the second fluorescent developer from each of the fluorescent images. It calculates the depth corresponding to each of the first separation image and the second separation image based on the depth information in the optical coherence tomography image, and combines each of the first separation image and the second separation image according to their depth to form a stereofluorescence tomography image of the photographed object (S) corresponding to the first fluorescent developer and a stereofluorescence tomography image corresponding to the second fluorescent developer.

Images