AR module based on microscope

Abstract

The utility model discloses an AR module based on a microscope, and relates to the technical field of biomedical instrument design, and the AR module comprises an image collection module, an AR projection module, an AI analysis module and a module base. The image acquisition module, the AR projection module and the AI analysis module are all arranged on the module base, and the module base is detachably connected with the corresponding position of the microscope; the image acquisition module comprises a camera, a reflector, a first polarizer, a first lens and a beam splitter prism; the AR projection module comprises a second lens, a second polaroid and a micro display; and the AI analysis module is connected with the camera and the micro display. According to the invention, the dual polaroids are adopted to prevent light in the direction of the micro display from entering the camera, the image of the display screen can be prevented from entering the camera to influence image capture of the camera, and the beam splitter prism with the reflectivity greater than the light transmittance is adopted to increase the light reflected to the camera and the eyepiece of the projection microscope of the micro display.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of biomedical instrument design, and particularly relates to an AR module based on a microscope. BACKGROUND

[0002] Among lymphoid and hematopoietic system diseases, malignant tumors account for a large proportion, and the incidence increases year by year. At present, bone marrow cell morphology, immunotyping, cytogenetics and molecular biology are complementary to each other, and are indispensable four test methods in the diagnosis and treatment of blood system diseases. Through immunotyping, cytogenetics and molecular biology, curative effect judgment and prognosis analysis are helpful to further improve the accuracy of diagnosis, but bone marrow cell morphology is still the cornerstone and gold standard for the diagnosis of lymphoid and hematopoietic system diseases. Because the bone marrow cell morphology is diverse, and different diseases can appear in a variety of cell morphology changes, therefore, the bone marrow cell morphology examination needs experienced testers, and the subjectivity is strong.

[0003] At present, some bone marrow cell image analysis systems have the functions of digital image acquisition, display, processing (such as image segmentation and editing, etc.) and tissue cell structure parameter measurement analysis, which are suitable for hospital blood laboratories, clinical laboratories and other related departments involving medical image analysis, medical colleges and biomedical morphology research laboratories of scientific research units. The bone marrow cell image analysis system can be applied to morphometric analysis, support bone marrow cytology examination report, peripheral blood smear report and chromosome analysis report, etc., and can also support remote bone marrow cell image consultation and single, multi-terminal or large screen projection morphology teaching. In terms of cell counting, a special cell counter is provided, the counting result is directly imported into the software system and automatically analyzed and calculated, manual input and manual calculation are replaced, the work intensity and error probability are greatly reduced, and the work efficiency is improved, but due to the high complexity and diversity of blood system malignant tumor cells, the identification of related blood system disease diagnostic bone marrow cells still needs manual identification of test technicians.

[0004] At present, with the help of image processing and artificial intelligence (AI) technology, objective and automatic blood cell recognition, classification and statistics can be partially realized, but AI recognition needs full-digital scanning specimens, needs to invest full-automatic microscopic scanning instrument, the instrument is expensive, and cannot meet the needs of primary hospitals and large-scale popularization.

[0005] Blood cell morphology analysis needs to observe cell particles, nucleoli and chromatin under a high-power oil lens of a microscope, and the full-digital slide scanning device is affected by the stability of high-speed movement of a loading table and the flatness of a slide surface. Compared with the real field of view under a microscope, the digital picture after scanning has differences in cell color, size and internal structure, especially for atypical cell morphology, the difference is larger, so manual review needs to observe under the microscope again. The utility model discloses a microscope-based AR module, adopts double polarizer to block the light of micro display direction to enter the camera, can avoid the image of display screen to enter the camera and influence the camera to capture the image, and adopts the reflectivity greater than the light transmittance of the light splitting prism to increase the light of reflection to the camera and micro display projection microscope ocular lens.

[0006] The utility model discloses a microscope-based AR module, adopts double polarizer to block the light of micro display direction to enter the camera, can avoid the image of display screen to enter the camera and influence the camera to capture the image, and adopts the reflectivity greater than the light transmittance of the light splitting prism to increase the light of reflection to the camera and micro display projection microscope ocular lens.

[0007] To reach the above-mentioned purpose, the utility model provides a microscope-based AR module, include: image acquisition module, AR projection module, AI analysis module and module base, wherein, image acquisition module, AR projection module and AI analysis module all set up on module base, and the corresponding position of module base with microscope is detachable connection, image acquisition module includes: camera, reflector, first polarizer, first lens and light splitting prism, AR projection module includes: second lens, second polarizer and micro display, and the polarization direction of second polarizer is perpendicular with the polarization direction of first polarizer, and AI analysis module is connected with camera and micro display, and sample light passes through light splitting prism and divides into two light, one light directly enters microscope ocular lens, and the other light passes through first lens, first polarizer and reflector in proper order and enters camera, and is converted into electric signal through camera and is input to AI analysis module, and AI analysis module carries out the analysis to electric signal, obtains the identification result, and the identification result is imaged to micro display, and obtains display screen image, and the display screen image of micro display passes through second polarizer and second lens in proper order and divides into two light through light splitting prism, one light enters microscope ocular lens, and the other light is prevented to enter camera under the interaction of first polarizer and second polarizer.

[0008] As above, wherein the light splitting prism and the microscope optical axis direction form a preset angle.

[0009] As above, wherein the light splitting prism and the microscope optical axis direction form a 45° angle.

[0010] As above, wherein the light splitting prism has a light transmittance N and a reflectivity M, and M>N.

[0011] As above, wherein the light splitting prism has a light transmittance N=30% and a reflectivity M=70%.

[0012] As above, wherein the micro display is an OLED, an LCD, a digital micro-lens array, or a laser scanner projection, and the micro display is located at an imaging focal plane of the second lens.

[0013] As above, wherein the AI analysis module is connected to the camera through a data line, and the AI analysis module is connected to the micro display through a data line.

[0014] The AI analysis module communicates with the camera through the wireless communication module; and the AI analysis module communicates with the micro display through the wireless communication module.

[0015] The camera, the reflector, the first polarizer, the first lens, the light-splitting prism, the second lens, the second polarizer and the micro display are sequentially arranged on the module base from left to right; the module base is provided with at least one sliding rail; the camera, the reflector, the first polarizer, the first lens, the light-splitting prism, the second lens, the second polarizer and the micro display are connected to the module base through the sliding rail.

[0016] The camera, the reflector, the first polarizer, the first lens, the light-splitting prism, the second lens, the second polarizer and the micro display are sequentially arranged on the module base from left to right; the module base is provided with a plurality of mounting holes; the camera is detachably connected to the mounting hole through the camera support, the reflector is detachably connected to the mounting hole through the reflector support, the first polarizer is detachably connected to the mounting hole through the first polarizer support, the first lens is detachably connected to the mounting hole through the first lens support, the light-splitting prism is detachably connected to the mounting hole through the light-splitting support, the second lens is detachably connected to the mounting hole through the second lens support, the second polarizer is detachably connected to the mounting hole through the second polarizer support, and the micro display is detachably connected to the mounting hole through the micro display support.

[0017] The application achieves the following beneficial effects:

[0018] (1) The application adds an augmented reality (AR) module to a general microscope, so that the analysis results of artificial intelligence can be integrated into the operator's field of view in real time, superimposed on the sample field of view seen by the operator through the eyepiece, and the see is obtained (i.e. the real field of view can be seen, and the cell classification result can be automatically obtained), without the need for digital scanning, without slowing down the sample inspection or modifying the standard operation process, and without reducing the efficiency of the sample inspection or modifying the standard operation process, so that the application can be integrated into the artificial intelligence auxiliary system in real time, and the application is convenient for review.

[0019] (2) As long as there is a microscope, the AR module based on the microscope can realize intelligentization, i.e. a morphological expert can be configured in each primary hospital, which can provide software and hardware support for the five-level disease management system and the three-dimensional prevention and treatment management system of malignant diseases established in China, and can lay a foundation and provide support for screening hematopoietic system diseases, improving the medical treatment method of urban and rural residents, and improving the efficiency of the medical service system.

[0020] (3) The microscope-based AR module of the present application adopts two polarizers (i.e., a first polarizer and a second polarizer) to block light from the micro display from entering the camera, which can avoid the image of the display (i.e., the display image) from entering the camera and affecting the camera's capture of the image.

[0021] (4) The microscope-based AR module of the present application adopts a light splitting prism with a reflectivity greater than a transmittance to increase the light reflected to the camera and the micro display projection microscope eyepiece. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings described below are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0023] Figure 1 Structure diagram of an embodiment of the microscope-based AR module. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0025] As Figure 1As shown, the application provides a microscope-based AR module, comprising: an image acquisition module, an AR projection module, an AI analysis module, and a module base. Among them, the image acquisition module, the AR projection module, and the AI analysis module are all arranged on the module base, and the module base is detachably connected with the corresponding position of the microscope. The image acquisition module comprises: a camera 1, a reflector, a first polarizer 2, a first lens, and a light splitting prism 3. The AR projection module comprises: a second lens, a second polarizer 4, and a micro display 5; the polarization direction of the second polarizer 4 is perpendicular to the polarization direction of the first polarizer 2. The AI analysis module is connected with the camera 1 and the micro display 5. The sample light 6 is split into two paths of light by the light splitting prism 3, one path of light directly enters the microscope eyepiece 7, and the other path of light sequentially passes through the first lens, the first polarizer 2, and the reflector, then enters the camera 1, and is converted into an electrical signal by the camera 1 and then input to the AI analysis module. The AI analysis module analyzes the electrical signal to obtain a recognition result, and images the recognition result on the micro display 5 to obtain a display screen image. The display screen image of the micro display 5 sequentially passes through the second polarizer 4 and the second lens, and is split into two paths of light by the light splitting prism 3, one path of light enters the microscope eyepiece, and the other path of light is prevented from entering the camera 1 under the interaction of the first polarizer 2 and the second polarizer 4.

[0026] Specifically, the second polarizer 4 allows light of a specific direction to pass through, and the polarization direction of the first polarizer 2 is perpendicular to the polarization direction of the second polarizer 4, so the first polarizer 2 can block the light passing through the second polarizer 4. The microscope-based AR module of the application uses double polarizers (i.e., the first polarizer 2 and the second polarizer 4) to block the light from the micro display 5 from entering the camera 1, which can avoid the image of the display screen from entering the camera 1 and affecting the capture of the image by the camera 1.

[0027] The image acquisition module, the AR projection module, and the AI analysis module can be fixedly connected or detachably connected with the module base, and the application preferably is detachably connected, which facilitates adjusting the setting position of the image acquisition module, the AR projection module, and / or the AI analysis module or replacing the image acquisition module, the AR projection module, and / or the AI analysis module. Among them, the specific model or parameters of the camera 1, the reflector, the first polarizer 2, the first lens, the light splitting prism 3, the second lens, the second polarizer 4, and the micro display 5 are set according to actual conditions.

[0028] Further, as an embodiment, the camera 1, the reflector, the first polarizer 2, the first lens, the light splitting prism 3, the second lens, the second polarizer 4 and the micro display 5 are sequentially arranged on the module base from left to right; wherein the module base is provided with a plurality of mounting holes; the camera 1 is detachably connected with the mounting hole through the camera support, the reflector is detachably connected with the mounting hole through the reflector support, the first polarizer 2 is detachably connected with the mounting hole through the first polarizer support, the first lens is detachably connected with the mounting hole through the first lens support, the light splitting prism 3 is detachably connected with the mounting hole through the light splitting support, the second lens is detachably connected with the mounting hole through the second lens support, the second polarizer 4 is detachably connected with the mounting hole through the second polarizer support, and the micro display 5 is detachably connected with the mounting hole through the micro display support.

[0029] Specifically, the specific number of mounting holes is set according to the actual situation. By adjusting the camera support, the reflector support, the first polarizer support, the first lens support, the light splitting support, the second lens support, the second polarizer support and / or the micro display support to connect with the mounting holes at different positions, the setting positions of the camera 1, the reflector, the first polarizer 2, the first lens, the light splitting prism 3, the second lens, the second polarizer 4 and / or the micro display 5 can be adjusted. For example, by adjusting the axial position of the first lens relative to the target surface of the camera 1, the sample image can be clearly imaged to the target surface of the camera 1; by adjusting the axial position of the second lens relative to the micro display 5, the display screen image of the micro display 5 can be clearly observed under the microscope ocular.

[0030] Further, as another embodiment, the camera 1, the reflector, the first polarizer 2, the first lens, the light splitting prism 3, the second lens, the second polarizer 4 and the micro display 5 are sequentially arranged on the module base from left to right; wherein the module base is provided with at least one sliding rail; the camera 1, the reflector, the first polarizer 2, the first lens, the light splitting prism 3, the second lens, the second polarizer 4 and the micro display 5 are connected with the module base through the sliding rail, so as to adjust the setting positions of the camera 1, the reflector, the first polarizer 2, the first lens, the light splitting prism 3, the second lens, the second polarizer 4 and the micro display 5.

[0031] Further, the reflector, the first polarizer 2, the first lens and the light splitting prism 3 and the microscope objective form an infinite distance imaging light path, which can image the sample image on the target surface of the camera 1.

[0032] Specifically, the first lens and the microscope objective in the microscope light path form an infinite distance imaging light path, which can magnify the sample image to the target surface of the camera 1.

[0033] Further, when the light splitting prism 3 is arranged on the module base, and the microscope-based AR module is arranged on the microscope, the light splitting prism 3 and the optical axis direction of the microscope form a preset angle.

[0034] Further, the specific value of the preset angle is set according to the actual situation, and the application preferably is that the light splitting prism 3 and the optical axis direction of the microscope form a 45° angle.

[0035] Further, the light transmittance of the light splitting prism 3 is N, and the reflectivity of the light splitting prism 3 is M, and M>N.

[0036] Specifically, since the use of the polarizer will cause the light reflected to the camera 1 and the light projected to the microscope eyepiece by the micro display 5 to be weakened, the number of layers of the light splitting prism is increased to achieve the N / M light splitting effect, thereby enhancing the light reflected to the camera 1 and the light projected to the microscope eyepiece by the micro display 5.

[0037] Further, the specific value of the light transmittance N is set according to the actual situation. The specific value of the reflectivity M is set according to the actual situation. As an embodiment, the light transmittance N of the light splitting prism 3 is 30%, and the reflectivity M of the light splitting prism 3 is 70%.

[0038] Further, as another embodiment, the light transmittance N of the light splitting prism 3 is 20%, and the reflectivity M of the light splitting prism 3 is 80%.

[0039] Further, the micro display 5 is an OLED, LCD, digital microlens array, or laser scanner projection display device, but is not limited to an OLED, LCD, digital microlens array, or laser scanner projection display device. The micro display 5 is located on the imaging focal plane of the second lens.

[0040] Specifically, the micro display 5 can be directly placed in the microscope light path of the infinite imaging system to realize the augmented reality display function.

[0041] Further, as an embodiment, the AI analysis module is connected to the camera 1 through a data line.

[0042] Specifically, the AI analysis module has an external input interface.

[0043] Further, as another embodiment, the AI analysis module communicates with the camera 1 through a wireless communication module.

[0044] Specifically, the wireless communication module can be realized by using an existing module with wireless communication function.

[0045] Further, as an embodiment, the AI analysis module is connected to the micro display 5 through a data line.

[0046] Specifically, the micro display 5 has an external input interface, and can display external data on the screen through the external input interface.

[0047] Further, as another embodiment, the AI analysis module communicates with the micro display 5 through the wireless communication module.

[0048] Specifically, the wireless communication module can be implemented by using an existing module with wireless communication function.

[0049] Further, the camera 1 of the image acquisition module, the micro display 5 of the AR projection module, and / or the AI analysis module are also connected to the control module, and the working state of the camera 1 of the image acquisition module, the micro display 5 of the AR projection module, and / or the AI analysis module are controlled by the control module.

[0050] Specifically, the control module can be implemented by using an existing module with the function of controlling the working state of the device and transmitting and receiving working instructions, which can improve the automation degree of the microscope-based AR module.

[0051] The microscope-based AR module of the present application can be applied to clinical diagnosis, and can also be used as a teaching tool for medical colleges, and can also be applied to medical fields such as pathological diagnosis, chromosome karyotype analysis, and microbial morphology identification, and has strong clinical practical value. The microscope-based AR module of the present application is used for blood film re-examination and student teaching in some hospitals, which can improve the efficiency of film reading; the accuracy rate of peripheral blood leukocyte classification reaches more than 90%, which can greatly alleviate the problems of labor shortage and high cost. In addition, the microscope-based AR module of the present application realizes the local deployment of AI, and also solves the problem of data security.

[0052] The beneficial effects achieved by the present application are as follows:

[0053] (1) The present application adds an augmented reality (AR) module to a general microscope, so that the analysis result of artificial intelligence can be integrated into the operator's field of view in real time, superimposed on the sample field of view seen by the operator through the eyepiece, and the see is obtained (i.e. can see the real field of view, and also can automatically obtain the cell classification result), without the need for digital scanning, in the case of not slowing down the sample inspection or modifying the standard operation process, it can be "integrated in real time" in the artificial intelligence auxiliary system, which is convenient for rechecking.

[0054] (2) As long as there is a microscope, the microscope-based AR module of the present application can be intelligent, that is, a "morphology expert" can be configured in each primary hospital, which can provide software and hardware support for the five-level disease management system and the three-dimensional prevention and treatment management system of malignant diseases established in China, and lay a foundation and provide support for screening hematopoietic system diseases, improving the way of medical treatment for urban and rural residents, and improving the efficiency of the medical service system.

[0055] (3) The microscope-based AR module of the present application adopts a double polarizer (i.e., a first polarizer and a second polarizer) to block light from the micro display from entering the camera, which can avoid the image of the display screen (i.e., the display screen image) from entering the camera and affecting the camera's capture of the image.

[0056] (4) The microscope-based AR module of the present application uses a light splitting prism with a reflectivity greater than a transmittance to increase the light reflected to the camera and the micro display projection microscope eyepiece.

[0057] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the scope of protection of the present application is intended to include the preferred embodiments and all changes and modifications falling within the scope of the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and changes of the present application fall within the scope of the present application and its equivalent technology, the present application also intends to include these modifications and changes.

Claims

1. A microscope-based AR module, characterized in that, The application relates to a microscope module. The microscope module comprises an image acquisition module, an AR projection module, an AI analysis module and a module base. The image acquisition module, the AR projection module and the AI analysis module are arranged on the module base, and the module base is detachably connected to a corresponding position of a microscope. The image acquisition module comprises a camera, a reflecting sheet, a first polarizer, a first lens and a light splitting prism. The light splitting prism has a light transmittance N and a reflectivity M, and M>N. The AR projection module comprises a second lens, a second polarizer and a micro display. The camera, the reflecting sheet, the first polarizer, the first lens, the light splitting prism, the second lens, the second polarizer and the micro display are sequentially arranged on the module base from left to right. The micro display is located on the imaging focal plane of the second lens. The AI analysis module is connected to the camera and the micro display. Sample light is split into two paths by the light splitting prism. One path of light directly enters the microscope eyepiece, and the other path of light sequentially passes through the first lens, the first polarizer and the reflecting sheet, enters the camera, and is converted into an electrical signal by the camera and then input to the AI analysis module.

2. The microscope-based AR module of claim 1, wherein, The AI analysis module analyzes the electrical signal, obtains an identification result, and images the identification result on the micro display to obtain a display screen image.

3. The microscope-based AR module of claim 1, wherein, The display screen image of the micro display sequentially passes through the second polarizer and the second lens, and is split into two paths by the light splitting prism.

4. The microscope-based AR module of claim 1, wherein, One path of light enters the microscope eyepiece, and the other path of light is prevented from entering the camera under the interaction of the first polarizer and the second polarizer.

5. The microscope-based AR module of claim 1, wherein, The light splitting prism and the optical axis direction of the microscope form a 45-degree angle.

6. The microscope-based AR module of claim 1, wherein, The light transmittance N of the light splitting prism is 30%, and the reflectivity M of the light splitting prism is 70%.

7. The microscope-based AR module of claim 1, wherein, The micro display is an OLED, an LCD, a digital microlens array or a laser scanner projector. The AI analysis module is connected to the camera through a data line, and is connected to the micro display through a data line.

8. The microscope-based AR module of claim 1, wherein, The AI analysis module communicates with the camera through a wireless communication module, and communicates with the micro display through a wireless communication module. The module base is provided with at least one sliding rail. The camera, the reflecting sheet, the first polarizer, the first lens, the light splitting prism, the second lens, the second polarizer and the micro display are connected to the module base through the sliding rail. The module base is provided with a plurality of mounting holes. The camera is detachably connected to the mounting hole through a camera support, the reflecting sheet is detachably connected to the mounting hole through a reflecting sheet support, the first polarizer is detachably connected to the mounting hole through a first polarizer support, the first lens is detachably connected to the mounting hole through a first lens support, the light splitting prism is detachably connected to the mounting hole through a light splitting support, the second lens is detachably connected to the mounting hole through a second lens support, the second polarizer is detachably connected to the mounting hole through a second polarizer support, and the micro display is detachably connected to the mounting hole through a micro display support.

Images