SYSTEM AND METHOD FOR MONITORING AND PERFORMING LABORATORY PROCEDURES

Description

TECHNICAL FIELD

[0001] The present invention relates to a system and method for following and conducting laboratory procedures. More particularly, the invention relates to such a system comprising a microscope configured for examining a specimen; a camera connected to the microscope, wherein the camera is capable of taking images of the specimen obtained by the microscope and transmitting thereof in an electronic manner to augmented reality glasses; a computer connected to the camera, wherein the computer is capable of processing the images, at least one pair of augmented reality glasses connected to the computer and being capable of having the images projected thereon in real time, such that a user does not need to use an eyepiece of the microscope to view the images, wherein the computer has laboratory protocol files for In Vitro Fertilization (IVF) procedures installed thereon, and the computer is capable of projecting images of the protocols onto the glasses, and wherein the projected images do not interfere with the user's natural vision, wherein the laboratory procedures for IVF comprise Ovum Pickup procedure, Oocyte screening and In Vitro Maturation, wherein each of the procedures is time dependent and wherein the glasses are configured to notify the user of the time to perform a next procedure, thereby the system prevents errors and fatigue of the user.BACKGROUND OF THE INVENTIONIn Vitro Fertilization (IVF)

[0002] IVF is a process by which an egg is fertilized by sperm outside the body. The process involves monitoring and stimulating a woman's ovulatory process, removing an ovum or ova (egg or eggs) from the woman's ovaries and letting sperm fertilize them in a liquid in a laboratory. The fertilized egg (zygote) is cultured for 2-6 days in a growth medium and is then transferred to the same or another woman's uterus, with the intention of establishing a successful pregnancy. The lab processes for achieving a successful IVF demand strict discipline, experience and knowledge. Procedures such as Ovum Pick Up (OPU) which require high-skills and vast knowledge are done by doctors and embryologists only. Typically, a stereo microscope is used for IVF laboratory procedures. Looking through the eyepiece of the microscope for long periods, as one does when performing IVF, is very tiring. The solution is to have the microscope connected to a screen, so that the lab worker can look at the screen instead of the eyepiece. This solution is not perfect, since the workers still find the need to use the eyepiece.

[0003] Another challenge that an IVF lab worker faces, is the many regulations and procedures that the work entails. For instance, it is very crucial that the correct specimens of egg and sperm are taken for a single IVF procedure, and that they are not mixed-up with other specimens. There are many steps and regulations that need to be met in order to ensure that the worker does not become confused. In spite of the many regulations, the "human error" factor still exists. There is a need to provide a method for further preventing human error.Augmented Reality (Wikipedia)

[0004] Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is modified (possibly even diminished rather than augmented) by a computer. As a result, the technology functions by enhancing one's current perception of reality. By contrast, virtual reality replaces the real world with a simulated one. Augmentation is conventionally in real time and in semantic context with environmental elements, such as sports scores on TV during a match. With the help of advanced AR technology (e.g. adding computer vision and object recognition) the information about the surrounding real world of the user becomes interactive and digitally manipulable. Information about the environment and its objects is overlaid on the real world. This information can be virtual or real, e.g. seeing other real sensed or measured information such as electromagnetic radio waves overlaid in exact alignment with where they actually are in space. Augmented reality brings out the components of the digital world into a person's perceived real world. AR displays can be rendered on devices resembling eyeglasses. Versions include eyewear that employ cameras to intercept the real world view and re-display its augmented view through the eye pieces and devices in which the AR imagery is projected through or reflected off the surfaces of the eyewear lens pieces. Background art includes U.S. Publication No. US 2014 / 0314300, which discloses systems and methods for facilitating classification of cytological by acquiring or importing image data of a cytological specimen to identify one or more objects of interest in a respective specimen image dataset, including feature attributes for the identified objects.SUMMARY OF THE INVENTION

[0005] According to an aspect of the present invention, there is provided a system for following and conducting laboratory procedures (30), said system comprising: a microscope (36) configured for examining a specimen; a camera connected to said microscope (36), wherein said camera is capable of taking images of said specimen obtained by said microscope (36) and transmitting thereof in an electronic manner to augmented reality glasses; a computer (34) connected to said camera, wherein said computer (34) is capable of processing said images; at least one pair of augmented reality glasses (32) connected to said computer (34) and being capable of having said images projected thereon in real time, such that a user does not need to use an eyepiece of said microscope to view said images; wherein said computer (34) has laboratory protocol files for In Vitro Fertilization (IVF) procedures installed thereon, and said computer (34) is capable of projecting images of said protocols onto said glasses (32), and wherein said projected images do not interfere with a user's natural vision, wherein said laboratory procedures for IVF comprise Ovum Pickup procedure, Oocyte screening and In Vitro Maturation, wherein each of said procedures is time dependent and wherein said glasses are configured to notify the user of the time to perform a next procedure, thereby said system (30) prevents errors and fatigue of the user.

[0006] According to some embodiments, said microscope (36) comprises a stereomicroscope.

[0007] According to some embodiments, said microscope (36) connection to said camera may be any one of a wireless and wired connection.

[0008] According to some embodiments, said computer connection to said at least one augmented reality glasses (32) may be any one of a wireless and wired connection.

[0009] According to some embodiments, said augmented reality glasses (32) can have both said laboratory protocol image and said microscope (36) image projected thereon simultaneously.

[0010] According to some embodiments, said connection is wireless and a person situated away from a laboratory location may wear said glasses (32) and by so, supervise a lab workers steps.

[0011] According to some embodiments, the system further comprises a Smartphone application transmitting protocols of various procedures including photos, video and audio files, to said glasses (32) via a cable or RF transition.

[0012] According to a further aspect of the present invention, there is provided a method for following and conducting IVF laboratory procedures, said method comprising: wearing augmented reality glasses (32) and connecting them to a computer (34); projecting a written laboratory protocol for IVF procedures comprising Oocyte screening and In Vitro Maturation to said augmented reality glasses (32) for a user to read so that a user can follow the steps of said protocol while conducting said laboratory procedure; connecting a microscope (36) to a camera, wherein said microscope (36) examines a specimen and wherein said camera produces images of the specimen; connecting said camera to said computer (34), wherein said computer (34) analyzes said images produced by said camera; and projecting said images of the specimen onto said augmented reality glasses (32) in real time such that said user does not need to use an eyepiece of said microscope to view said images; wherein said projections do not interfere with a user's natural vision, wherein said laboratory procedures for IVF comprise Oocyte screening and In Vitro Maturation, wherein each of said procedures is time dependent and wherein said glasses are configured to notify the user of the time to perform a next procedure, wherein each of said procedures is time dependent and wherein said glasses are configured to notify the user of the time to perform a next procedure, thereby preventing fatigue of the user.

[0013] According to some embodiments, the method further comprises the step of: identifying the specimen situated in a petri dish (40) inspected by said microscope by screening information associated with a barcode (42) attached to said petri dish with said augmented reality glasses (32), and preventing fertilizing wrong specimens.

[0014] According to some embodiments, the produced images are a video sequence and the specimen is being micro-manipulated while viewing it through said augmented reality glasses (32).

[0015] Accordingly, it is a principal object of the present invention to overcome the limitations of prior art systems for conducting laboratory procedures.

[0016] Generally speaking, the present invention is directed for use of augmented reality glasses (32) in preparation of IVF laboratory procedures.

[0017] The inventive system is intended to ease the strain of a user working on a microscope for long periods of time and preventing mistakes.

[0018] The system as disclosed herein provides convenience for the user analyzing specimens using input means such as a microscope, by using augmented reality glasses capable of having images of the specimen projected thereon so that the user is not forced to look at the specimen through an eyepiece of a microscope. Using the augmented reality glasses prevents fatigue caused by using a microscope eyepiece for long periods of time.

[0019] Other objects and advantages of the invention will become apparent as the description proceeds.

[0020] The following publications are considered as prior art: https: / / www.eti.uni-siegen.de / ubicomp / papers / es ubicomp2015.pdf to Philipp M. Scholl et. al and US8848289B2 to Amirparviz et al. No one of these publications, alone or in combination with the other, teach on a camera attached to a microscope that passes a video stream of the microscope presentation to an augmented reality glasses while carrying out an IVF process.

[0021] The reference numbers have been used to point out elements in the embodiments described and illustrated herein, in order to facilitate the understanding of the invention. They are meant to be merely illustrative, and not limiting.BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Embodiments, features, aspects and advantages of the present invention are described herein in conjunction with the following drawings: Fig. 1 shows a prior art example of a user viewing a specimen through a microscope eyepiece, also having a digital screen connected to the microscope; Fig. 2 shows a system for following and conducting laboratory procedures; Fig. 3 shows a petri dish with a barcode; Fig. 4 shows a specimen under a microscope under the treatment of a micromanipulator; Fig. 5 shows an augmented reality glasses of the present invention showing an image of a specimen being micro manipulated; Fig. 6 shows the augmented reality glasses, showing a specimen image and a protocol image projected onto the glasses; and Fig. 7 shows a schematic block diagram of the system and method of the present invention.

[0023] It should be understood that the drawings are not necessarily drawn to scale.DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The present invention will be understood from the following detailed description . For the sake of brevity, some well-known features, methods, systems, procedures, components, circuits, and so on, are not described in detail.

[0025] In Fig. 1 there is shown a prior art example of a user viewing a specimen through a microscope eyepiece. The microscope is connected to a digital screen which shows an image of the specimen.

[0026] In Fig. 2 there is shown a system for following and conducting laboratory procedures 30. System 30 includes augmented reality glasses 32 which are connected to computer 34 via a wireless or wired connection and a microscope 36, connected to computer 34. A screen 35 may also be connected to computer 34. A video and stills camera (not shown) is connected to microscope 36. A specimen 38 is placed under microscope 36 for viewing, manipulating or treating specimen 38. The camera can photograph images, or shoot videos, of specimen 38 and project those images and movies onto glasses 32. The videos and photographed images may be saved for backup or evidence. A user (typically a lab worker or scientist) wearing glasses 32 can view specimen 38 without using an eyepiece of microscope 36, which can be very tiring after using the eyepiece for a long period of time. The glasses 32 have clear lenses so that the user has a regular view of his surroundings in addition to a view of the projected images.

[0027] In addition to having images from specimen 38 projected onto glasses 32, images of lab protocols can be projected. images of computer documents of various types, such as .pdf, .gif, .jpg etc., may be projected. The advantage of having protocols projected onto a users glasses 32, is that the user can carefully follow the steps of the laboratory protocol while conducting the protocol, and therefore will not forget any step, and errors will be prevented.

[0028] In addition glasses 32 may be adapted to reflect the visual characteristics of a user, such that if the user is near sighted or far sighted, the image presented on glasses 32 is adapted accordingly.

[0029] Glasses 32 may have a camera installed thereon for taking photos and videos.

[0030] In another embodiment of the system 30, a Smartphone features applications which transmit protocols of various procedures including photos, video and audio files, to the glasses via a cable or RF transmission.

[0031] In Fig. 3 there is shown a petri dish having a barcode 42 attached to its side, for scanning to a Matching System. Barcode 42 holds information regarding the identity of the individual whose specimen 38 (not shown here) belongs to. Glasses 32 have the ability to scan a barcode 42 and read it so that the information regarding the individual will appear on glasses 32 for the user to view, and the information will be compared to a database. The Matching System alerts the user if there is a mismatch by projecting a warning onto glasses 32. This feature is very important for any lab, but particularly for an IVF lab which handles both sperm specimens and egg specimens and needs to "mix" them in order for fertilization to take place. It is obviously crucial for a sperm specimen of a man to fertilize the egg specimen of his female spouse. The results of a mistake in this stage are devastating. Therefore, many precautions are to be taken to prevent such a mistake. Before working with a certain specimen, the user must scan the barcode 42, with glasses 32, on petri dish 40 so that the Matching System of glasses 32 can make sure the correct specimen is being handled.

[0032] In Fig. 4 there is shown microscope 36 having petri dish 40 placed under microscope's 36 objective 44, and a micromanipulator 46 is shown treating a specimen in petri dish 40. A camera (not shown) is connected to microscope 36 and glasses 32 (not shown here) are connected to the camera so that images taken by the camera are projected onto glasses 32, still and videos alike. The embryologist micromanipulating the specimen does not need to see it via the eyepiece of microscope 36, he can see everything on glasses 32, in real time. Using this method, IMSI (sperm morphology) can be done, as well as Intra Cytoplasmatic Sperm Injection (ICSI), laser assisted hatching or any other specific micromanipulation. For example, Oocytes vitrification can be performed by projecting the Oocyte vitrification protocols of different companies (such as Origio, Kitazato or Vitrolife, inclusive of preparation of the various solutions and devices). Imaging of the various stages of equilibration and vitrification can be seen. Because each step is time dependent, glasses 32 are able to notify the embryologist of the time to perform the next procedure.

[0033] In vitro culture (IVC) and embryo monitoring by time-lapse systems such as Embryoscope, Primovision, Eeva, Gavi, etc., can be connected to glasses 32, and embryo images are projected thereon.

[0034] Using glasses 32 is a very convenient method for training new employees. A simulation of the procedures can be displayed on the glasses as well.

[0035] In Fig. 5 there is shown augmented reality glasses 32 showing an image of a specimen (38) being micro manipulated. Any common IVF procedure may benefit from using glasses 32, also a procedure known as Ovum Pick Up (OPU) which is a procedure for extracting eggs from a woman's ovaries. An OPU procedure is executed by inserting a needle through the vaginal wall and into an ovarian follicle. This procedure is done under the guidance of an Ultrasound (U.S) device, and the physician needs to keep his eye on a screen showing the U.S images, during the entire procedure. By Using glasses 32, there is no need to look at the screen, the U.S images are projected to glasses 32, so that the physician need not glance away from the patient to the screen. Using this method, the entire procedure is smoother, quicker and safer.

[0036] Embryo transfer (ET) into a patients uterus can also be done using glasses 32. Ultrasound images of the endometrium will be projected onto glasses 32. Additionally, the physician performing the transfer can evaluate the embryos as seen in the streomicroscope and will be read the details of the couple.

[0037] An additional example of a procedure that can be done using glasses 32 is sperm collection and evaluation; images of the analyzed sperm can appear on the glasses, as well as images of the sperm parameters from the microscope. Digital magnification and morphology evaluation additionally can be seen.

[0038] In Fig. 6 there is shown augmented reality glasses 32 showing a specimen image and a protocol image projected onto glasses 32. The wearer of glasses 32 can see both the image of the specimen and the written protocol of the procedure he is conducting.

[0039] In Fig. 7 there is shown a schematic block diagram of the architecture of the system and method of the invention. Block 50 represents the various input means that may be used for system 30, such as microscope, stereomicroscope, barcode scanner, U.S transducer, etc. Arrow 52 represents the processing of the input information of block 50, and block 54 represents the various processing operations done with the input of block 50. Arrow 56 represents the output resulted from processing operation 54, which is projected onto either / both screen 35 and augmented reality glasses 32.

[0040] In the figures and / or description herein, the following reference numerals (Reference Signs List) have been mentioned: The system for following and conducting laboratory procedures 30 Augmented reality glasses 32 Input means such as a microscope 33 Computer 34 Computer screen 35 Microscope 36 Specimen 38 petri dish 40 barcode 42 Microscope objective 44 Micromanipulator 46

[0041] Any term that has been defined above and used in the claims, should to be interpreted according to this definition.

[0042] The reference numbers in the claims are not a part of the claims, but rather used for facilitating the reading thereof. These reference numbers should not be interpreted as limiting the claims in any form.

Claims

1. A system for following and conducting laboratory procedures (30), said system comprising: a microscope (36) configured for examining a specimen (38); a camera connected to said microscope (36), wherein said camera is capable of taking images of said specimen obtained by said microscope (36) and transmitting thereof in an electronic manner to augmented reality glasses (32); a computer (34) connected to said camera, wherein said computer (34) is capable of processing said images; at least one pair of augmented reality glasses (32) connected to said computer (34) and being capable of having said images projected thereon in real time, such that a user does not need to use an eyepiece of said microscope to view said images; wherein said computer (34) has laboratory protocol files for In Vitro Fertilization (IVF) procedures installed thereon, and said computer (34) is capable of projecting images of said protocols onto said glasses (32), and wherein said projected images do not interfere with a user's natural vision, wherein said laboratory procedures for IVF comprise Ovum Pickup procedure, Oocyte screening and In Vitro Maturation, wherein each of said procedures is time dependent and wherein said glasses are configured to notify the user of the time to perform a next procedure, thereby said system (30) prevents errors and fatigue of the user.

2. The system (30) of claim 1, wherein said microscope (36) comprises a stereomicroscope.

3. The system (30) of claim 1, wherein said microscope (36) connection to said camera may be any one of a wireless and wired connection.

4. The system (30) of claim 1, wherein said computer connection to said at least one augmented reality glasses (32) may be any one of a wireless and wired connection.

5. The system (30) of claim 1, wherein said augmented reality glasses (32) can have both said laboratory protocol image and said microscope (36) image projected thereon simultaneously.

6. The system (30) of claim 3, wherein said connection is wireless and a person situated away from a laboratory location may wear said glasses (32) and by so, supervise a lab workers steps.

7. The system (30) of claim 1, further comprising a Smartphone application transmitting protocols of various procedures including photos, video and audio files, to said glasses (32) via a cable or RF transition.

8. A method for following and conducting IVF laboratory procedures, said method comprising: wearing augmented reality glasses (32) and connecting them to a computer (34); projecting a written laboratory protocol for IVF procedures comprising Oocyte screening and In Vitro Maturation to said augmented reality glasses (32) for a user to read so that a user can follow the steps of said protocol while conducting said laboratory procedure; connecting a microscope (36) to a camera, wherein said microscope (36) examines a specimen and wherein said camera produces images of the specimen; connecting said camera to said computer (34), wherein said computer (34) analyzes said images produced by said camera; and projecting said images of the specimen onto said augmented reality glasses (32) in real time such that said user does not need to use an eyepiece of said microscope to view said images; wherein said projections do not interfere with a user's natural vision, wherein said laboratory procedures for IVF comprise Oocyte screening and In Vitro Maturation, wherein each of said procedures is time dependent and wherein said glasses are configured to notify the user of the time to perform a next procedure, wherein each of said procedures is time dependent and wherein said glasses are configured to notify the user of the time to perform a next procedure, thereby preventing fatigue of the user.

9. The method of claim 8, further comprising the step of: identifying the specimen situated in a petri dish (40) inspected by said microscope by screening information associated with a barcode (42) attached to said petri dish with said augmented reality glasses (32), and preventing fertilizing wrong specimens.

10. The method of claim 8, wherein said produced images are a video sequence and the specimen is being micro-manipulated while viewing it through said augmented reality glasses (32).